Its clear that CrimsonShadow wants to do an ELO system... which is easy enough to do, but has its won fair share of issues... The problem with ELO is that it has flaws in tournament design. Its really made for 1 on 1 matches. It is considered a "race" system, it can end up rewarding people simply for attending the most events... and it has a problem where it can severely punish people for simply having a bad day (something very common in fighting games). The Association of Tennis Professionals Entry Ranking system was designed to combat a lot of the inherit issues in ELO.
ELO is also sensitive to the ORDER of matches... because each match is representative of itself. Its not just about who you beat, but also when you beat them. Yes, there are algorithms to take care of ELO ranking in single events, but not if events are submitted in the wrong order. For instance, I submit an event that happened yesterday. The rankings would be handled based on that date and would preclude the adding of new events before that date; since ELO is based on order. In order to add new events previous to the date; you would have to rebuild the entire ranking system after every single tournament that is submitted. The ATP system allows a rolling ranking, without a concern for the order in which events are submitted into the system.
The tennis ATP system is probably the best "neutral" ranking system; since it bases itself solely on tournament results. It doesn't care who you played against, just how you performed. Player A can be the best player in the world, who only ever loses to one person, his training partner Player B... but Player B has never won a tournament... with ELO, there is a chance Player B could be ranked higher than Player A. Since ELO would consider B beating A, as a "bad beat", and include it in the rankings. Meanwhile, B gets a large windfall because he beat the top ranked player.
The issue with the tennis ATP system is that it takes a bit more work to monitor to make sure TOs aren't gaming the system; while ELO just handles itself. I've been developing this ranking system since 2009. Its the best way to handle fighting game events in a non-league style fashion I have found.
ELO also suffers from the issue where it actually promotes top players into NOT attending events. If you are high ranked on the ladder, you may not want to attend anymore events because you put your ranking at risk. This is where you have issues of when a new game comes out and some player abuses the meta; in time people will learn how to beat him, but by then he has earned himself a top ELO rating. Then he drops out from tournaments and simply plays online and trolls the forums. Unless you do a yearly ELO rating reset, he's going keep that rating forever... and a yearly reset would upset the people who still attend events, by throwing their hard-earned work in the trash... for a non-league tournament structure, this is not recommended. In addition, you may have issues where tournament players will selectively choose what events they go to, depending on who else is going to be at that event. Maybe they don't want to play against a specific player and ruin their ranking?
The ATP system is designed to "stale" older rankings. So it promotes people to continue attending events to meet the minimum event requirement. In addition, it automatically rolls retired players out of the rankings. As well, since it doesn't care who is actually in the event, people don't have to be selective about what tournaments they attend. Basically, the issue with ELO is all social problems; not mathematical problems.
