This is a tricky issue, and one where only you can decide. As you have found, it is only Audi(UK) who stipulate the 5 year change interval. Audi Germany state only the mileage limit, and this applies in Germany and many other countries. However, they then go on to say that individual countries can make their own recommendations based on local conditions. Since the weather and road conditions in Germany are not that different to the UK the 5 year change interval is hard to understand, other than as a money maker for Audi(UK). One determined forum member took this up with Audi Germany, but despite several exchanges they would not/could not give any logical explanation. As I recall, after much wriggling, Audi finally conceded that even in the UK a belt change at 5 years was not mandatory, but that an annual condition check should be made.
When I had my Passat, like you I covered very few miles and kept the car garaged. In common with Audi(UK), VW(UK) said it needed a belt change after 12ok miles or 5 years, despite there being no such requirement in Germany. Well, after 5 years I had only covered 15k miles, so I decided to leave it longer. After 8 years (and 39k miles) I reasoned that I would be selling the car in another couple of years, and a purchaser would be looking for a recent belt change, so I cracked and had the change. The old belt was like new when it was removed, but I was right in that one of the first questions from the chap I sold the car to was "has the belt been changed recently?" So the £500 cost of the belt change was largely offset by the improved selling price I got for the car.
My 1.4tfsi is currently on 16k miles coming up to 4 years old, and I expect the mileage will climb at a rate of no more than 3k miles per year. Audi Germany say that the belt is a special design which lasts the life of the car. Audi(UK) say it should be changed ever 5 years. I will start visually checking the belt after 5 years, but I think it is very unlikely I will need to change it due to deterioration. I intend keeping the car for another 5 years at least, so as with the Passat I will probably have an (uneccessary) belt change a year or two before I sell it just to make it easier to sell for a good price.