I've driven to France, Germany, Poland and Spain over the years when the children (3) were young. The roads are not as congested as in the UK and you can cover 100's of miles quickly but the wife and kids like lots of comfort breaks so you do take a little longer to get to your destination than you first thought. Best thing is the stops and sights you see on the way.
In France the Viaduc de Millau took about 3 hours to go over and also went around to the viewing points nearby.
I'm trying now to convince the wife we should drive to Sweden via the Øresund Bridge in the Westfield, she's not keen on that but it will happen one way or another.
For piece of mind use AA European Ins. cover, make your tyres are newish and a complete health check of the car before you go. Use the Channel Tunnel, it's quicker and doesn't suffer problems that the ferries have. At worst ,if the Channel Tunnel is closed you are given tickets for the Ferries sameday.