Another thing that gets people nervous: Infallibility.
My impression is that the Rebbe is infallible isn’t limited to meshichists, CMIIW. Being fallible is part of being mortal.
Perhaps this isn’t specific to Lubavitch, it can be an issue by other groups, even litvish by the extreme “Daas Torah” types. But my impression is that it’s more of a developed idea by Lubavitchers.