The stigma is not the yarmalke per se. Yarmalke type / hat type is a way of identifying yourself as part of a group. Among 'black hatters' the kippa sruga represents Modern Orthodox Zionist. The stigma / resentment / antagonism is against the group, not the headgear itself.
100%.
People of different persuasions often dress somewhat similarly due to their similar identification. Hippies have long hair, some african americans wear their pants at half mast, etc.
When someone wears or does that thing they are implying that they are affiliated with that group. In our circles it's manifested itself with most 'Ultra' Orthdox Jews wearing Velvet and modern Orthdox wearing knitted/leather.
Wearing it in and of itself doesn't make you a different person but it sends a message to others about who you are.
Therefore, if you wear it people will assume that you associate yourself with Modern Orthodoxy.
Nothing wrong with being Modern Orthodox - but don't expect them to think you're Chasidish, or insulted if they assume you're not Yeshivish.