I think this has been discussed some times before.
You may be downvoted because the Downvoter doesn't agree with your point or because he (or she) thinks your comment is rubbish. There is no way to tell the reason, but so far it seems to work fine anyhow.