Are you in favor of removing them?
I'm not a historian, but I think that not all of them are equal. I can hear the argument that some should be removed. But just because a guy hated Jews doesn't mean his statue should be removed. But if someone did more than just say things, there's more reason to take it down.
I would say that the distinction should be based on why statues are being made for them. If the actions they are being honored for are bad, I think they should be taken down. If, however, they are being honored for good that they did, I don't think they should be torn down because of their (unrelated) sins.
For example, I think a statue of Robert E. Lee at West Point (assuming they have statues of all their Superintendents) is fine in my eyes. That same statue at the South Carolina state house is inappropriate.