ya know... I did this drive once, and I gotta say passing through wine country and in-and-over mountains was very pretty, and the sights while driving were an attraction all their own- I didn't even feel the need to stop and do anything!
Problem is, with kids you're gonna have to make frequent stops, and you'll probably find a portion of your trip will be in the middle of nowhere, with the road you're driving on the only sign of civilization (I felt like I was in a Road Runner cartoon... literally barren land in all directions, and just this 2-lane road in front of me). It was creepy... then you pass some mountains, and suddenly there's people again.
We did stop to eat our bagged lunches at a place called Casa de Fruita, which is a cute little place with some markets and stuff. Also driving through Gilroy with the windows down was an experience of itself. There's isn't much Kosher food, let alone civilization along much of this route, so prepare yourself before you go!