So @
Jaxel, we actually did run an ELO based ranking system last year and the results were this:
1) No one cares enough about their rating to actually stop going to tournaments because of it. We had absolutely zero problems with that, and I can't see someone being willing to sabotage their own winnings and enjoyment of the game because of it. And even they did, the beauty of a couple year's worth of IGAU or MK events is that people go to enough tournaments for the ratings to mean something, but not enough that they can just rest on their laurels.
If you simply stop going to events, someone will pass you up as the year goes on. It worked beautifully as it was intended to work. Also it's not that hard to code in an expiration or say that your ranking only stays visible if you attend an event once every X number of months.
And if anything, the fact that you can't "fake" your way up the rankings by beating less experienced/skilled opponents encourages people to come out and go after the other top players. But realistically, I don't know anyone who made travel plans based on the ranking system.
2) As far as punishing people for having a bad day -- to me, that's not a punish; it just reflects your actual results. If it's tweaked properly, you aren't swinging that drastically in points from one event to another. But what it does is reward consistency and reflect which players are actually playing well overall.
Would you tell a football team who missed the wildcard playoff spot by losing their last couple games that they were 'punished'? Nope -- if another team performed more consistently throughout the season, then they earned it. Each win and loss is accounted for.
It prevents people from resting on their laurels -- if they want to rank well, they have to play well. And it helps deal with the problem of people with easy access to tourneys simply attending a lot of events to "erase" their bad performances. You're held accountable for your performance at every ranked event.
3) The fact that players are assigned points based on their wins naturally adjusts the rewards according both the size of the tournament and the density of good/experienced players. If you go to something like EVO and win, the fact that you had to beat more and better players to get there vs. winning a smaller event means you end up with more points at the end. But it does this objectively, based on facts only -- without assigning an arbitrary point value to the entire event. And if tweaked/setup correctly, you are still rewarded most for getting farther in the tournament. Consistent players that win tournaments do end up farther than people that aren't, because it requires wins (and only allows 1 loss max) to get there. But winning a tournament and then going out in pools at your next couple, doesn't allow you to unfairly stick at the top by only counting your best nights.
This is important for Injustice-community-style events because unlike the tennis tour, we don't have a Masters or Grand Slam series of mandatory big events, and not everyone can make as many of them. So the density/quality of each major is determined by who actually decides to show up. (And currently in this community, whether or not the event is even a considered a major depends on who decides to show up). The same 'major' can be absolutely stacked one year, and sparsely attended by top players the next. Thus it produces more accurate rankings to take into account what kind of players people actually had to beat to place.
Overall I had a blast running things, and in the end the rankings we constructed did a great job of identifying the players who were playing the best. Both ATP and ELO have their tradeoffs. I'm not married to either type of system, but my main concern is that whatever is used actually fit the situation at hand. So you have to choose well, put aside biases and tweak the system carefully in order to give the best results for the actual community you're using it on -- not for a community of professional athletes with totally different rules and travel constraints.
But in any case -- we've probably beaten this discussion to death
