It wasn't a snarky remark. It was accurate as I see it. Banning HCM's is not taking away any liberty and yes it would make some unhappy. I don't see making someone unhappy as a downside. Feelings get hurt all the time.
I believe you are incorrect. Me not being able to own a tank, is taking away my liberty, but is done so for obvious reasons, that outweigh said "infringements" on my liberty. While HCMs are a Fer less extreme example, banning them, whether right or wrong would be taking away liberty to some extent. That leaves us with the same question as in the case of the tank, is it a liberty that should be taken away, due to the problems it can/does cause? Obviously you feel the answer is yes, and I for the most part agree.
ETA: If you don't like the tank example, replace it with a less extreme example. I used it to make a point.