If you use your Google Voice number exclusively (or at least primarily) and place calls through the normal phone dialer, you'll still need to have your Google Voice app installed at this point. You see, the Hangouts app right now isn't declaring itself as a "phone dialer," so when you tap a phone number in another app, say on a restaurant's website in Chrome or an email in Gmail, it still opens up your phone's dialer to make that call. And unless you leave the Google Voice app installed to route that call through the Google Voice system, you'll be calling out from the number associated with your SIM — and you may not want that.
But I don't have the old Google voice app and I don't see where I can install it
ETA: I found it