The name of Lakenheath always seems singularly inappropriate. There isn't a lake, and you left the heath behind four miles ago on the other side of Mildenhall. In fact, the name is a corruption of lyking-hythe, meaning the landing place of the fen dwellers. Lakenheath is a fen edge village, like so many places across the border in Cambridgeshire. The fen starts immediately to the west of the high street - you step down into it. Lakenheath has the feel of a frontier town. I almost expect to see tumbleweeds blowing along the high street. Lakenheath is the second biggest parish in Suffolk, by quite a long way. Unlike Mildenhall, the biggest, it doesn't have settlements scattered throughout it. Here, everyone lives along the ancient road from Eriswell. In 1844, White's Directory could observe that this was between the fen and the great warren, but today the great warren has gone, replaced by one of the biggest American airbases in Europe. This is the main employer for miles around, and as at Mildenhall the village centre suffers from this; if you work on the base you can shop on the base, and everybody does. Here on the edge of Suffolk, non-conformism set in early, and several of the villages had Baptist and Methodist churches before the Victorians brought them Anglican ones. Now, the Church of England is in retreat, surviving best where it has fully embraced evangelical protestant theology. But for many years, St Mary remained one of Suffolk's few Anglo-catholic shrines, and while that is not the case today there is still a feel of the High Church wing about it.

From the street, the most striking feature of St Mary is what appears to be a large porch to the west of the tower. This is so singular that it has inspired a variety of interpretations. Some say that it is a galilee porch as at Bottisham, just over the border in Cambridgeshire. Others suggest a chapel (there is an upper room with niches in the eastern wall). The window in the west face suggests a 15th century origin, although if that is the case the doorway below has obviously been renewed. In fact, it is none of these things. It was built in the years after the Reformation, probably in the early 18th century, as a schoolroom. The niches in the upper room are actually the formerly external niches on the tower's west face. However, the building appears older simply because the masonry and window were taken from the former parish church of Eriswell St Peter, the ruins of which still sit at the southern end of Lakenheath high street, transformed in the 18th century into a dovecot for Eriswell Hall.

The graveyard is a fascinating one, and a great text for understanding the changes in graveyard fashions; the 18th century graves are hard against the street, and you travel through time as you head eastwards, finally reaching the 1890s against the back fence. The 13th century tower is older than we are used to on big churches, and there was a general rebuilding at the time, but the core of the building is Norman, and, wonderfully, still remains so inside. You step in to a lovely, mysterious space, full of colour. Turning east, the Norman chancel arch is the best in Suffolk, the beautiful nave altar beneath it complementing it perfectly. Beyond, the high altar with its big six candles and Sarum screen are simple and splendid.

There are four remarkable medieval survivals at Lakenheath, and the one which is most striking is the sequence of wall paintings on the north arcade. There are a little difficult to decipher at first because there are at least three separate sets overlaying each other, the last of which is a nasty diaper and curly tree-pattern probably dating from the 18th century. The first figure you see as you enter is St Edmund low down on the column. He wears a crown and holds three arrows. Above, the later tree pattern overlies scenes from the life of Christ; the carrying of the cross on the right is easily discernible.

The upper levels are clearer, and the figures larger. Most obvious of all is the Annunciation scene. On the pillars below, life-sized figures stand like ghosts. As well as the sequence on the north arcade, there is a fine Christ in Majesty to the south of the chancel arch. You can examine it at close quarters, and see that the quality is really something special. I assume that it was the last in a sequence that once spread along the south arcade.

While in the south aisle you may notice, if you haven't done so already, some of Suffolk's finest bench ends. The most famous is the Lakenheath tigress. She stoops, beguiled by a mirror. The medieval bestiary recorded that the only way to steal a tigress's cubs is to place a mirror in the path of her pursuit. She then stops to look at her reflection, and the hunter makes off with her babies. The bestiary explained that the tigress is like humanity, the cubs are her soul, the hunter is the devil, and the mirror is worldly pleasure. Although damaged, of equal fame is the representation of two wrestlers, who have got themselves into a position that I shall only describe as intimate. There's also a contortionist, and a whale eating another one.

The third great survival here is Suffolk's best 13th Century font. It was perhaps that of the original church, and looks so much like the kind of thing the Victorians copied that you have to give it a second glance to make sure. It is topped by a stunningly beautiful font cover. And, finally above your head is another tremendous angel roof. Many writers have said that it is by the same craftsman as the one at Mildenhall. I don't know enough to say, except that here the feeling is so very different. At Mildenhall the angels are remote and magnificent; here, they are intimate and lovely.

Wandering around, there are still plenty of reminders of this building's High Church days, as well as a curiosity of a kind, a memorial to Earl Kitchener, who had family roots in the parish. There is a very big set of royal arms to Charles II.

Back at the turn of the century, English Heritage began to cast their beady eye on Lakenheath. As well as advising on the restoration of the existing wall paintings, their infrared cameras discovered considerable wall painting in the north aisle. Perhaps there are more exciting discoveries still to be made, and that potential may well play a significant part in St Mary's future. For, as lovely as this church is, it has fallen on hard times. The congregation is tiny, and the church is vast. For some reason, don't ask me to explain, this corner of Suffolk and Norfolk, where the two counties meet, is prey to more vandalism than anywhere else in the region. Now, everything here is well-kept, and obviously much loved by a congregation who are proud of their church and want you to love it too. But is the situation sustainable?

And these days, the problem of upkeep is compounded by the threat of lead theft. We were surprised to find the church open when we came back in the summer of 2011, because ordinarily this church is kept locked, like its neighbours at Brandon and Eriswell. You have to get the key from the village shop, but now the shop will no longer have the key. We'd found the church open because a meeting was taking place to discuss an incident the previous day. Someone had borrowed the key from the shop and then taken it back a little while later. That evening, someone walking their dog noticed that the priest door in the chancel had been unbolted from the inside and left on the latch - fortunately, it was a windy evening, and the door was rattling slightly. I knew the reason why the door had been opened at once. At a church in east Suffolk earlier that week, a churchwarden had told me the same thing had happened to her church - the villains had then returned in the middle of the night, let themselves into the church, put up a ladder in the south aisle, made a hole and then removed the roof lead from the inside, quite undisturbed by the risk of anyone seeing them. It seemed that Lakenheath was being prepared for a similar job.

In all honesty, I don't see how some churches will survive the current round of metal thefts. If Lakenheath church was the beating heart of its community, if locals were in and out of the church all day long, if local people felt they had a personal stake in the building, then it would be different. But it isn't, they aren't, they don't, and it won't be. For that reason I can only foresee redundancy for this church in the long term, and English Heritage's interest will no doubt ensure that it is taken on by the Churches Conservation Trust to receive more tender loving care than can be possible while it is still in use by its parish. It grieves me to write this, but I believe it to be the case; and, even, to be the best possible outcome. Fortunately for most churches, the local diocese frowns on redundancy these days, and most medieval parish churches have their future ensured by the benefice system. But this is such a big church, and such an important one, and it has such a potential for neglect, that I suspect that an exception may be eventually made. I only hope that it can happen before it is too late.