i was in the oposite camp mate had an S3 always wanted an S4... once i wrapped the S3 the choice was made for me but if you want a true 'drivers car' i think you'll feel you need to tweak either model. i spent nearly 8 grand gettin my s3 the way i wanted it, suspension, chips, heldex controllers, adj tie bars, poly buts here an there uprated ARB's... it just goes on i think to assume you can sell your a4 and find everything you want in an s3 might leave you unimpressed.
i would say if your a fan of the 'true quattro' you wont like the haldex, i never did an now we the s4 i wouldnt go back.
end of the day mate we all want cars we dont have an take what we drive for granted...you want a black one till you have one its only then you fancy a different colour! we've all got the same problem on here!
good luck either way!