Hi,
I have a 2004 2.0TDI BKD and i had similar issues...
It could be lots of things, i would suggest you find a VAG specialist (Not Audi) and get the car scanned.
I had diagnostics at Audi and they found nothing.. waste of money.
I went to a local VAG specialist and their diagnistics found a Faulty Cam Shaft Sensor, becuase this sensor is behind the timing belt and it was due anyway i had waterpump, timing belt bearings pulleys and the camshaft sensor replaced all with genuine parts. £500 ish all in.
It is a lot better however i do still have some slight issues in starting (Rarely)...
Suspect Lazy starter motor.
I have also changed all Glow Plugs, Temp Sensors, air filter,Battery etc....
If i was you, to save blindly buying new parts, get it scanned and hopefully you will have a better idea of what is wrong.