If round-towered Holy Trinity around the corner is a glass of chilled white wine, then this building is an exotic cocktail of a church. One of those cocktails with a miniature umbrella, a couple of sparklers, and chunks of unfamiliar fruit. The immense tower of St Mary rears up between the two, the three churches together forming a kind of ecclesiologist's theme park. St Edmund sits on Bungay's busy High Street in a neat graveyard behind a neat hedge. It is actually on the site of the former medieval church of St Thomas of Canterbury, which once shared a graveyard with St Mary. It is worth noting in passing that this church and St Benedict at nearby Beccles are the only urban Suffolk Catholic churches with their own graveyards. By the time Suffolk's catholic churches were built, the Burial Acts of the 1860s had been passed, prohibiting urban churchyard burials. But Bungay and Beccles were too small for the Acts to apply to them.

The church is served by the Benedictine community, and has a thriving school immediately next door, giving Bungay a peculiarly Catholic presence on its High Street. Externally, it is a remarkable building. Most striking of all is the massive octagonal structure on the north side of the front. This is the baptistery, and what it contains is even more worthy of note. We will come back to it in an moment. The frieze above the main entrance shows the martyrdom of St Edmund, and you step beneath it into a quiet porch, with no indication of the treasures beyond.

The contents of Catholic churches are usually brought together over the years, as and when their often poor communities managed to afford them. Not so here. Not only was St Edmund all built in one go, it was also completely furnished in the most opulent style at the same time. You step through into what is probably the most ornate church in all East Anglia. The walls are entirely panelled and carved in wood. Set into them are the Stations of the Cross. Stone statuary above looks down. The windows are full of gorgeous coloured glass by Hardman & Co. The sanctuary glistens with gilt. It takes your breath away.

The man who made all this possible is commemorated to the north of the entrance. His name was Frederick Smith, and he died in 1903. The parish is very fortunate to have a church that, from the start, met its every liturgical and devotional need. Mind you, it must leave the Knights of St Columba twiddling their thumbs a bit. Around the corner to the baptistery, then. You pull the curtain aside, and step into a high, gated room, with the most stunning font set in the middle of it. The font is carved in coloured marble, and the font cover rises above it. Clearly, Frederick Smith was not short of a bob or two, and he even bequeathed a trust to pay for future repairs. If you come here, you will be standing in the most significant Art Nouveau space in Suffolk.