I think the people who are against it aren't against religion, but rather, everything has its time and place...
Exactly, the fact that they're building mosques is not really what bothers me because on the flipside, as jews, we know that we are free to build shuls whenever/wherever we need them. I just simply think that its a little gallish to build a massive mosque so close to ground zero when it's islam that promoted the attacks in the first place. Remember how happy the muslims were after 9/11? They were dancing in the streets out of joy! (i was young but i still remember seeing it on the news)
There is a time and a place for everything, and a massive mosque right outside ground zero post 9/11 is whats giving people a hard time NOT the concept of having freedom to build religious venues
But thats just my opinion....