I do not recommend new zealand with kids, at least not the south side which is all I've been to.
Yes, Milford Sound is one of the most beautiful places in the world. We were told that we must drive from Queenstown (which is the ski town) to Milford as it is one of the most beautiful drive in the world. We headed out (this was in August) and after a while it started snowing and got worse and worse. We came to a hill where a bunch of cars got stuck and my wife did not want us to drive anymore as it seemed way too dangerous.
I had been to Milford on a previous occasion and we got there via single engine plane from Queenstown which was almost the freakiest plane ride ever (possibly matching the short flight from st maarten to st barths). The flight is awesome, the scenery from the plane is breathtaking, we were snapping pics the whole way through. We then took a boat ride in Milford Sound.
There is a very cool speedboat ride near Queenstown in the canyons, very wild and very recommended (could not do it the second time since my wife was pregnant).
Let me know if you want more