I was turning ss off manually with the button when it was activating as it should - now I don't bother/need to as for some unknown reason it rarely works/cuts in! - took it to Audi and they could not find any issues but I did not make any more of a fuss - despite meeting the various criteria to function (according to the handbook) mine very rarely ever cuts in although I do not know why!
Sounds like your battery is having the same issues as mine. Beware as it may just die like mine did. There are a few threads on here about a dodgy batch of VW/Audi batteries