I wouldn't go so far as to say they were directly led astray, .
As a non chabadsker, you should please properly study the Torah, the directives, and the actions of the leadership. - I doubt you have properly done this. - an unbiased person would objectively see the truth. - there are hundreds of strange and suspect sayings, actions that directly caused the confusion.etc etc. Etc.
I would guess that your familiarity of chabad world view is limited to what you heard from the opposition.