I came down out of St Margaret, the lanes narrowing and straightening. Sad-eyed cows huddled in the lea of trees, as the clouds thickened thunderously, and the air hung heavy like a dulling wine. I had cycled through sunshine and cloudburst across the Saints, soaked near St Peter, warmed and uplifted at All Saints, and I felt sure that this was the next wave, but in fact it would not rain again for nearly a week. My wheels spun as the landscape changed, and I was rolling downhill now into the Waveney valley. I was near the end of my journey. I had set out to visit all twelve of the Saints in one bike ride, and this was the last, although of course it is not really one of the Saints' churches at all. I first came here in the winter of 1998. Then, it had rained for several days, and a fierce, blustery wind was buffeting across the wide open plains of the Saints. My bike had got a puncture a couple of miles away at St Michael, so I abandoned it, to collect it later. I'd walked into the weather, along the narrow, hedgeless lanes. The flat, ploughed fields gleamed with wet flints. There was not a soul about. It was like walking on the moon.

After a couple of miles, I'd come to South Elmham Hall, in the parish of St Cross. Here, the fields and paths were dissolving beneath the previous days' deluge, and I was glad I was wearing sensible shoes. I picked my way through the farmyard mud, a couple of hens surveying me miserably from beneath a tractor. Ahead of me, the five bar gate seemed a way into another world. As I opened it, and stepped out into the field, the winter sun lit the tree-filled valley below. Here I was aimed. The footpaths across the Hall farm were clearly marked, generally running along hedgerows (hedgerows in the Saints!) and the whole area was designated South Elmham Walks. I came down into the valley, where the brook marked on my OS map was a lazy flood, spreading out beyond the ends of the flat concrete cattle bridge. But I was impatient to get on, so I swirled through the shallows, on my way to something I had wanted to see since moving to Suffolk more than ten years before.

Coming back now in summer, more than ten years later, I thought about the changes that Suffolk had seen since then. In 2001, the foot and mouth epidemic had changed rural England dramatically, closing it off for the summer and wrecking the rural economy. It was as if a great flock of locusts had spread across the countryside, eating livelihoods rather than crops. It left behind a barer, sadder landscape. And then there was the more insidious rise to power of the supermarkets. Now, the East Anglian farming economy was at the beck and call of the giants of Tesco, Sainsbury and Wal-Mart. They dictated the prices and the fashions in food. With one word they could wipe out demand for a crop in mid-season, and slowly they encouraged the industrialisation of meat production. The old England was going for ever. In the late 1990s, I had cycled through every Suffolk village, visiting every Suffolk parish church. Repeating the journey ten years on, I was amazed by how empty the countryside had become, and particularly by the absence of cattle, which seemed to have been marginalised onto specialist farms. Instead, great wastes of land had been given over to pig production, and fields which had been left fallow thanks to the absurdity of the Common Agricultural Policy were now intensively and rapaciously farmed for short-term gain. So much of Suffolk looked like a temporary landscape. I remembered cycling past dairy farms, many of them with 18th and 19th century barns and outbuildings, smelling the silage and hearing the mournful lay of cattle, but now these same farms were empty apart from piles of silent, rotting machinery. It was depressing. What will become of us, I wonder?

I cycled down to the gates of South Elmham Hall, and was pleased to discover that it was still in business. I locked my bike, something I rarely do when I am out cycling in the East Anglian countryside, but I had a walk ahead of me, and I didn't want to find myself stranded at the end of it. I climbed the gate as I had ten years before. There was no one about, no other walkers in sight. Some beautiful cows came across to look at me. They jostled for position, but the wire of an electric fence held them back. There were new laminated signs explaining how to avoid crossing fields of cows in the Walks. This may sound like another case of 'Health & Safety' gone mad, but in fact a woman had been trampled to death on this very path just a few months before. Cows are gentle, but they are inquisitive, very big and very stupid. If one of them thinks you are interesting, it will follow you, and other cows will follow her. If you walk faster, they will walk faster to follow you, and if you run, they will run. And you will tire or trip before they do. The best thing to do is to stand your ground, wave your arms, shout and clap, and shoo them away. They'll soon forget you.

The sun came out, the clouds evaporating as I strolled on across the undulating fields. Ahead of me in the woods was a mystery. An ancient flint core ruin of uncertain origin, dated by some as early as the 7th century, and by others as late as the 15th. It is set among towering oaks and beeches, and surrounded by a wide, low, rectangular rampart, which is clearly something much older, most likely a Roman encampment. Here, half a mile from the nearest road, in an area where the word 'road' is itself a bit of an exaggeration, I saw for the first time the remains of the building generally described as South Elmham Minster, although I have also heard it referred to as a Cathedral, a castle, a church and even a pagan temple.

Fortunately, thanks to archeologists, we know rather more about it now; but in some some ways, they have presented us with more questions than they have answered. I stepped through the narrow gate, and made my way across an open space into the trees. Ahead of me, the ruins glowed in the vivid sunlight beneath the trees.

Bales of straw were stacked haphazardly among the ghostly shapes. I walked among them, trying to get my bearings. This is as wild a place as any in Suffolk, although as a result of the time when the South Elmham Walks project received European funding a park bench sits incongruously in the clearing. These ruins date, almost certainly, from slightly before the Conquest - say, about AD 1000. But the story behind it goes back to the evangelisation of Suffolk by St Felix some 350 years earlier.

The eccesiastical organisation of late Roman Britain is lost in the mists. The foundations of a 4th century church have been uncovered in Colchester, 40 miles away, but tell us little about the bigger picture. What we do know, though, is that in about 630 St Felix, the first Catholic Bishop of East Anglia, established his See at Dunmoc, probably Walton Castle on the Suffolk coast. However, his successors soon moved on, and there may never have been a Cathedral building at Walton, in the sense we understand it today. Instead, the East Anglian Bishops styled themselves 'Bishop of Elmham'; the See then moved on to Thetford, and at last to Norwich. Today, the restored Catholic Bishop of East Anglia still has his see in Norwich, and the Anglicans, who inherited the medieval Cathedral at the Reformation, have their Bishop of Norwich too.

Such simple history suggests, of course, that this building in the woods was Elmham cathedral, and some guidebooks describe it as such. However, they are wrong. For there is another Elmham in Norfolk, known as North Elmham, and it is there that the successors of St Felix sat and ministered ex cathedra. But South Elmham DOES have some Episcopal connections, and they come much later. For it was here that that the Bishops of Norwich had their summer retreat, and South Elmham Hall is a surviving section of that building. It is not unlikely that the 'Minster' had a connection with this house, and we can fully discount the possibility that this was ever East Anglia's cathedral.

A more pausible theory, but again one that doesn't bear much examination, is that this really was a Minster - that is to say, a kind of Cathedral outstation, leading a group of parishes. This would neatly explain why there are so many villages with the name South Elmham. But it almost certainly isn't true. Such a small-scale Ecclesiastical structure would be unique in England; other Minsters were homes to religious communities, carrying out the Bishop's work in a large number of parishes (so, for instance, Beverley, Ripon, Wakefield and Southwell Minsters doing the Archbishop's work in the huge diocese of York). Here, we are only 15 miles from Norwich, and there is no evidence that there was ever a religious community in the Saints; there was one two miles away, at Flixton, but that was an Augustinian convent.

The most likely explanation is that this was an extraparochial chapel, perhaps even a chapel of ease for a now vanished community. John Sanderson of South Elmham Hall tells me that most archaeologists now consider it to be the work of Herbert de Losinga, first Bishop of Norwich, and that it was a private chapel that may also have been the centre of a community at some point in its life. It may have been on the site of an earlier church, which was on the site of a Roman temple, hence the location within a Roman fortification (you'll find the same thing at Burgh Castle, in Norfolk). The villages of South Elmham take their name from the Deanery of South Elmham, all the parishes of which had been under the ownership of the Bishop of East Anglia.

But there are features of this building which call for something more in the way of an explanation.

The church is basically in three parts; you approach from the east, and the further west you go, the more substantial are the remains. Not much survives of the chancel at all, except for a curious, square structure set in the north wall, at an angle to the rest of the building. I had puzzled over this for several years, but John Sanderson explains that it is the top of the north east corner, which had fallen into the building at some point and not been carted off for reuse elsewhere. The other two parts are separated by a wall with two apparent doorways in it. Some books describe the western section as the base of a tower, which may be so, although it wouldn't explain the partition wall. A more exciting theory is that the western part was a narthex, a common feature of Christian basilicas in the early years; on a grand scale, you can see what might have been here at Vezelay, in Burgundy.

Here in the narthex was where the uninitiated (that is to say, the unbaptised) could take part in the Ministry of the Word. However, at the Creed, a curtain would be drawn across, so that they could not witness the most solemn mystery of the Christian faith; the consecration and celebration of the Mass. It is awsome enough to think of this at Vezelay. It is a spine-tingling thing indeed to stand in these remote ruins, and imagine it happening here. Oddly in this woodland clearing, no birds sang. There was a deep silence, and it could have been any time. I stood for a while, just imagining. There was a sudden chill from the tree shadows, and I knew it was time to go. It was a relief to step out of the woods and into the sunshine again. The lazy cows watched me as I walked back along the edges of their fields to my bike, half a mile away. The clover hummed with bees flitting clumsily as I climbed the hill, and then I was off again, cycling north, as a skylark erupted from the pasture beside the lane, and began to sing.