Somerton Castle has now disappeared. Late in the 16th century a portion of the surrounding walls was standing together with a round bastion tower. The materials were used to construct a goal on its site somewhere in close proximity to the present White Hart Inn (Barnett).
Externally there are no remains of the castle walls in either the White Hart Inn or the adjoining buildings (OS record 1966).
"There is a tradition that its (the castles) dungeons once imprisoned French Royalty and remain under the Market Square" (Hayter). (Somerset HER)
From 1280 until c.1371, when Somerton's gaol was holding county prisoners, the clerks recording the proceedings at assizes regularly referred to the gaol as Somerton castle. The building adjoined a hall of pleas or court house and the churchyard, and a visitor in 1579 recorded 'an old tower embattled about castle-like'. Later antiquarians sought to place the castle in the market place, but that was a medieval house confused with a real castle at Somerton in Lincolnshire. (Dunning 1995)
Somerton's brief period as a county town began in 1278 when the shire courts were transferred there from Ilchester. The county gaol was established in the town in 1280, and itinerant justices began to deliver it in the same year. Early in 1366 the justices met again at Ilchester and, later in that year, in order to relieve Ilchester's economic depression, both shire and circuit courts were again permanently established there. By 1371 the gaol at Somerton was no longer holding county prisoners.
The gaol and its adjacent hall of pleas, where a riot had taken place in 1344, then went out of use. In 1434 John Harper leased a parcel of the house formerly called the court hall (aula curie), probably the hall of pleas, which stood 'by the churchyard of the church of Somerton'. Four years later Richard Smyth held a waste site within the lord's gaol on the west of this hall