I first came to Lidgate on a beautiful, crisp February morning towards the end of the last millennium. I was near the start of a bike ride from Newmarket to Ipswich. There was hardly a cloud in the sky, and it felt like the first proper springlike day of the year. The birds thought so too. Although the years have passed since that occasion, it has remained strong in my memory. I came back nine years later in May, on the hottest day of the year so far. This time I arrived from the south, and discovered what I hadn't noticed before: the pretty cottages of this otherwise lovely village line a real ratrun of a road, which took the pleasure out of the bike ride. I made a mental note to myself to arrive from the north next time I visited. Here, the border with Cambridgeshire is delightful; gentle woodland enfolds the little River Kennet, and the villages are invariably lovely; Dalham and Moulton particularly so. Given their proximity to Cambridge, the houses must be worth a fortune.

On that bright February day I soon found the road was climbing up onto the agricultural plain. The low sun striped the hedgeless lane ahead of me through a comb of poplars, and the settlements became preoccupied; working villages, at the heart of their land. On this February morning, a skein of mist still clung to the furrows, and rising among them was something most un-East Anglian. A castle mound, with houses and a church clambering around it. This was Lidgate.

I cycled off the main road, and into the village. A wide pond spread at the foot of the lane to the church, a few cars parked around it, their owners stretching in the pale light. A young couple had brought their infant son to feed the ducks. The high, cloudless sky clamoured brilliantly in the surface of the still water, and soon it will be spring again, I thought, the whole cycle starting over.

Although the windows are Decorated, and it doesn't look as though this was a prosperous place in the 15th century, all is not what it seems, for the nave walls are Norman, and may well have been the castle chapel. The castle was probably one of those raised by King Stephen against the revolt of Geoffrey de Mandeville in 1143, later becoming a manorial hall. Not much is left, now. The poet John Lidgate may also have come from this village; he was a monk at Bury Abbey.

The church sits up on the hill beside the scant ruins of the castle, and one can't help but be reminded of the same juxtaposition on a grand scale at Durham and Lincoln. The churchyard was trim, but the site itself seems ancient and primitive. All is very simple, very humble. A proper village church this, which makes the Stations of the Cross and the other evidence of the 19th century revival all the more romantic.

Apart from the nave walls and later porch, all is 13th and 14th century. No clerestory was built here, making the church seem rather square and gloomy inside. Mortlock directed me to some extraordinary grafitti, including 14th century fragments of music, and the head of a Madonna. Impossible to photograph, of course. Medieval benches sit beneath the 19th century roof, and the rood screen is also medieval; but the gate, probably inspired by those at neighbouring Cowlinge, are not.

The most famous feature of this church is the small brass to Thomas Atwelle, one of only four in Suffolk that show a priest wearing Eucharistic vestments. However, it isn't something you can easily see, as it is covered by the long blue carpet which runs the length of the church. This is unfortunate, because, as at Bawburgh in Norfolk, the carpet draws the moisture from the stone floor, which then collects in the indentation of the brass, with the effect of ruining both brass and carpet. The carpet is railed down; I tried removing one of the rails, but when I came to the next one it would hardly budge. I thought that, at a pinch, I could probably get it out, but probably not put it back again, so in the end I reluctantly decided to leave it. I did lift the side of the carpet to look at the poor little figure, its latten oxidising into murky green. So much Catholic imagery was destroyed by the 16th and 17th century reformers; so many brasses were melted down by 18th century scrap metal dealers. So little survives, and everything is precious.