Why switch? I think the Moto X is a great device but besides being more comfortable in the hand I don't see how it outweighs the One.
Did you use both? The One is much bigger and heavier. The Moto X is so much more comfortable in the hand it's ridiculous. Holding the One in one hand, my thumb does not reach the top left of the screen. That the screen size is still the same is incredible.
Aside from that, the camera shortcut and active display are killer features. Touchless control is amazing, and something I use often as a someone who bikes. And battery life - it outlasts my One by a long shot. Then there are smaller improvements like the dimple in the back, Motorla Connect, Motorola Assist, and others.
The HTC One is a great phone, it really is. I would highly recommend it, especially over Samsung (too busy for my liking). It looks beautiful, the speakers in the front is a great touch, it's fast and I even liked Blinkfeed (Which Flipboard doesn't truly replicate).
But -IMHO -the Moto X is simply a more comfortable, more intuitive, longer lasting, and, above all, more practical device than the One - and that is saying a lot.