ELO Ranking System

[SnuggleBunnies]

• ELO Ranking System

• o An ELO ranking system has been added for MP games.
  o This is optional and challenges can be set as ranked or unranked.
  o A green ranked game symbol appears in the MP lobby for ranked games.
  o Ranked challenges show the challenger’s current ELO rating in the “Accept Challenge” Tab.
  o On-going ranked challenges show both players’ rankings in the tooltip in the “My Games” tabs. (At present their current rankings are shown, but in a future update the tooltip will instead show their rankings at the time the challenge was accepted – which are what scoring depends on - see below).
  o A player’s ranking starts at 1000 and then goes up or down depending on victories or defeats in ranked games. The amount it goes up or down depends on the relative existing rankings of the two players at the time when the challenge was accepted.
  o Beta ELO rankings for FOG2 Medieval are shown here: https://www.slitherine.com/leaderboard/ ... i-medieval

o A player’s ranking starts at 1000 and then goes up or down depending on victories or defeats in ranked games. The amount it goes up or down depends on the relative existing rankings of the two players at the time when the challenge was accepted.

I assume there is no way to integrate past slitherine official tournament scoring into starting ELOs for players, to get a head start on having the numbers actually being helpful.

[rbodleyscott]

o A player’s ranking starts at 1000 and then goes up or down depending on victories or defeats in ranked games. The amount it goes up or down depends on the relative existing rankings of the two players at the time when the challenge was accepted.

I assume there is no way to integrate past slitherine official tournament scoring into starting ELOs for players, to get a head start on having the numbers actually being helpful.

I don’t think so.

[Jorgito78]

Are there plans to add a time limit to "ranked" matches?
Because nothing prevents a player from leaving a match unnatended and it never finishing. So, will there be ELO adjustments in such cases? Or perhaps, after x time has passed, a % of ELO points is subtracted from the player in the current turn and added to the other player.

[rbodleyscott]

Are there plans to add a time limit to "ranked" matches?
Because nothing prevents a player from leaving a match unnatended and it never finishing. So, will there be ELO adjustments in such cases? Or perhaps, after x time has passed, a % of ELO points is subtracted from the player in the current turn and added to the other player.

Good point. I will pass it on the those developing the ELO coding.

[tyronec]

Time limits are the perennial issue for MP games. I think all of us who are running competitions or campaigns are having to manage somehow the small number of players who don't complete their games on schedule. For me running the HOML this is more of a hassle than all of the other little problems and mistakes being made added together.
Not a problem however for the Slitherine tournaments because the award system there works fine.
I think if there were a way of resolving this then we would see more types of competition such as knockouts or Swiss being run by players.

So what should the time limit be before ranked matches go to resolution. If it has to be a fixed length then my feeling would be 4 weeks, but would be good to hear what other players think.
What would be really great would be for an extra field when the challenge is being set up to specify the time limit for the game. Enter 1 to 9 for the number of weeks allowed.

I know this might be some work to set up but if I were to ask for one enhancement to FoG then this would be it.

[Jorgito78]

So what should the time limit be before ranked matches go to resolution. If it has to be a fixed length then my feeling would be 4 weeks, but would be good to hear what other players think.
What would be really great would be for an extra field when the challenge is being set up to specify the time limit for the game. Enter 1 to 9 for the number of weeks allowed.

I know this might be some work to set up but if I were to ask for one enhancement to FoG then this would be it.

Or there could be different levels, for example, "Speed Challenges", "Normal Challenge" and "Unranked", having the first a time limit of, say, 2 weeks, the "Normal" having a limit of 3 to 9 weeks and "Unranked" not having limits at all.

[SimonLancaster]

I think two weeks is being extremely polite to someone who hasn’t taken their turn. One week too short, but I think two weeks is fair to all.

[rbodleyscott]

I think two weeks is being extremely polite to someone who hasn’t taken their turn. One week too short, but I think two weeks is fair to all.

I agree, and have said exactly that to the PBEM3 overlords.

And then the win is awarded to the non-defaulting player - that is what would happen with the current code, the only issue is when the game should be called.

Also please note, we are talking about a time limit on inactivity, not a time limit for the whole game.

[tyronec]

For a time limit on inactivity would think around 10-15 days. I have had had many league games that have gone several days or even over a week with no move and then have been completed within the timeline - need to allow for players going on holiday.
However what about if someone got timed out from it being a ranked game but were still within the competition timeline. For that reason perhaps it should be a month for ranked games as it is now for all MP games. IMO this is not great, could lead to ranking games potentially to go on for months if one side is playing very slowly.

This is going to help for some games in campaigns/competitions where a player disappears however there will be more games where a player is just playing slowly and doesn't get finished on time. The organisers will still have to adjudicate or resolve in some other way. And then there is the issue of a game being terminated for the campaign/competition but not for the ranking system - does the player who was playing at a good pace have to continue to not be penalised in the ranking system. Maybe this is not a big deal and that for the few games where a player was playing very slowly we would just have to accept that the ranking system could be unhelpfull.

It would all work much better if we could have a time limit for the whole game. Maybe that is not possible, if so then so be it. However I would have thought that all the data is there and it is already being done in the Slitherine tournaments.

[ericdoman1]

Hi TC

Thanks. Had a look. Not really much info on it at the mo.

It would be great if we could all use the same rating system.

TDC - now consists of flynnn5555, mh5064, wzfcns, Triarii, ulysisgrunt, angusosborne and Kroche

WTC - Triarii, XLegione, angusosborne and I

1. Is the rating system used for all Slitherine tournaments. That is from the very beginning to future and 3, 4 and 5 round tournaments

2. How is it going to distinguish between present players and past players. If you take note of earlier tournaments, say the first 2 years. How many of the same players are playing now?

3. What about those players who may only play in a few tournaments (you and I for example), compared to those who play in all. Is the rating system going to work on the average or total.
For example Player A has played in 6 tournaments, total points are 6000 so average is 1000. Player B has played in 12 tournaments, total points is 7200 but average is 600. If total points Player B with 7200 will be above Player A with 6000. If average, Player B would be above Player A.

4. Only using the best 5 or 6 tournament "scores", in the last 2 years, might be the better option.

5. I do like the idea that all tournaments use the same rating system but does that mean we all have to use the same scoring system.

6. What tournaments are to be included and should there be bonus points for the tougher ones. For example HOML is the toughest tournaments based on the opponents. TDC is probably second, Slitherine tournaments 3rd and then say WTC (although, the added pressure of playing for a team and country, might make it 3rd). I am not familiar with any other tournaments to make a decision on level of difficulty.

There is always more

Cheers


Eric

[kronenblatt]

If I understand this correctly, it’s an ELO ranking system? Then wins and losses will adjust the ranking positively for wins and negatively for losses; with the adjustment based on the difference in original (i.e., before the game) ranking between the two players. i.e., the scoring system of an individual tournament doesn’t really matter, since it’s only whether winning or losing that will adjust the ranking. It also doesn’t necessarily benefit players with numerous games, since only wins are beneficial to the ranking while losses are detrimental.

[Triarii]

I have just downloaded the open beta and am particularly interested in this.

• ELO Ranking System
o An ELO ranking system has been added for MP games.
o This is optional and challenges can be set as ranked or unranked.
o A green ranked game symbol appears in the MP lobby for ranked games.
o Ranked challenges show the challenger’s current ELO rating in the “Accept Challenge” Tab.
o On-going ranked challenges show both players’ rankings in the tooltip in the “My Games” tabs. (At present their current rankings are shown, but in a future update the tooltip will instead show their rankings at the time the challenge was accepted – which are what scoring depends on - see below).
o A player’s ranking starts at 1000 and then goes up or down depending on victories or defeats in ranked games. The amount it goes up or down depends on the relative existing rankings of the two players at the time when the challenge was accepted.
o Beta ELO rankings for FOG2 Medieval are shown here: https://www.slitherine.com/leaderboard/ ... i-medieval

Slitherine PBEM may or may not be aware that we have the ELO type rankings for the TDC games from TDC2 to TDC4 inclusive.
The calculations and records exist with the 500+ game results in files on a shareable drive and the mechanism of calculation may be of interest.

The system is a standard ELO and does this
1 gives a equal and opposite ELO change to the players
2 Will quantify that change on
a) difference between ELO scores
b) difference between the players % damage i.e. margin of victory or difference in a draw.

I am more than happy to share the working files if that would be useful.

[rbodleyscott]

1. Is the rating system used for all Slitherine tournaments. That is from the very beginning to future and 3, 4 and 5 round tournaments

As I understand it, the plan was to start the ELO rankings afresh, so of course it would take time for them to become representative and stable.

All MP games tagged as Ranked when the challenge is set up would be included. For the Slitherine tournament setup, there is an option in the tournament setup to set all the games as ELO ranked, which I would do when setting up the official Slitherine tournaments.

For manually run tournaments you would need to tell your tournament players to check the ranked checkbox when setting up their challenges. (And that game is not valid unless this was done).

It is possible that some "friendly" tournaments might choose not to use ranked games.

2. How is it going to distinguish between present players and past players. If you take note of earlier tournaments, say the first 2 years. How many of the same players are playing now?

4. Only using the best 5 or 6 tournament "scores", in the last 2 years, might be the better option.

As of the current time, the plan was to start the rankings from scratch when the release version goes live. Which, of course, avoids any issues as to which previous tournaments to include.

I don't know whether it is intended to have rankings degrade over time if players do not play for long periods.

3. What about those players who may only play in a few tournaments (you and I for example), compared to those who play in all. Is the rating system going to work on the average or total.
For example Player A has played in 6 tournaments, total points are 6000 so average is 1000. Player B has played in 12 tournaments, total points is 7200 but average is 600. If total points Player B with 7200 will be above Player A with 6000. If average, Player B would be above Player A.

Do you think perhaps you are taking this a bit too seriously? It isn't international chess rankings and it isn't intended just for tournaments.

It is more of a "ladder" system.

5. I do like the idea that all tournaments use the same rating system but does that mean we all have to use the same scoring system.

No. The game client program will decide whether the game is won or lost, which is the only thing (apart from the ELO ratings at the time that the challenge was accepted) affecting the ranking adjustments. The actual tournament scoring is a separate issue and can be done any way you want.

6. What tournaments are to be included and should there be bonus points for the tougher ones. For example HOML is the toughest tournaments based on the opponents. TDC is probably second, Slitherine tournaments 3rd and then say WTC (although, the added pressure of playing for a team and country, might make it 3rd). I am not familiar with any other tournaments to make a decision on level of difficulty.

See above. This is for all ranked MP games, not just tournaments.

The scoring adjustment depends solely on the two players' rankings at the time the challenge was accepted.

The Slitherine lead on the ELO system is David Sharrock, and I have asked him to look at this thread.

[tyronec]

The game client program will decide whether the game is won or lost, which is the only thing (apart from the ELO ratings at the time that the challenge was accepted) affecting the ranking adjustments. The actual tournament scoring is a separate issue and can be done any way you want.

That works fine for completed games, however there does need to be consideration of what happens when games have timed out - if the rating system calculates a timeout in a different way to say a campaign or a competition.

Also how are timed out games going to be scored ?
I would say that if a player has been timed out then they must be rated as 'lost'. Otherwise we could have players stopping playing games when they are about to lose to help their rating.
However if an opponent times out I would say no change to the rating. Otherwise players are going to have their ratings going up just because their opponent doesn't turn up, as happens in a lot of the Slitherine tournaments. With a consequent distortion to the rankings, garbage in - garbage out.

[kronenblatt]

For the sake of simplicity and because this is not a matter of life or death, maybe a timed-out game should simply for both players be viewed as a draw for the purpose of ELO Rankings (except in the case of Slitherine-run automated tournaments where timed-out games seemed to be measured and an outright winner and loser determined based on that).

All systems can be gamed in one way or another, and if a player does that once or at least a couple of times too many, then he won't be welcome (back) to our community-run tournaments anyway.

[Triarii]

I hope Slitherine do find a practical way to run a multi-player (mp) ELO ranking system and I would be happy to see one.

My initial assumption was that the ranking will apply in any mp game where the players opt in or possibly (in the case of Slitherine tournaments) are opted in. from Richard’s answers that seems to be the intention.
This could be wrong, but it is difficult to see any way in which such a system would change the way in which community tournaments are set up and chosen to be scored. The ELO ranking would be different to and independent of any community organisers choices. Organisers could choose to take note of rankings or not.

It is also difficult to see any way in which existing community rankings could or should inform Slitherine’s system other than an explanation of “this is how we did it”. The offer to share our current record is absolutely for illustration of a method and absolutely not to try to inform any Slitherine ranking that may happen.

There are valid questions about how it is intended that stalled games be dealt with. I had assumed but do not yet know if the intention is that where ranking is opted in, and a game becomes stalled then at the point at which the game can be ended the player who has not returned turns will be the default loser.

Also, of interest is what the intention is for multi-player games, not in Slitherine Tournaments, where there is an outlier map or mistaken set-up meaning an early cancellation of the game.

I am very interested to see how these things pan out in the open beta.

These would be my initial questions;

a) Is win/loss and the difference between existing ranking at the start of the game the only factors affecting the ELO changes? Is the difference between % losses intended to be used? It seems not from current answers but that isn’t entirely clear.

b) How is it intended stalled games will be treated – is it a default loss for the awol player?


c) In mp games outside official tournaments is it intended that there be a criterion for a minimum number of turns before ranking is applied (e.g. 6 turns/ 3 per player) to allow for rejection of outlier maps or a mistaken set up in a community tournament?

[kronenblatt]

Also: this is Medieval. Will the same apply to Ancients in the next patch? And the ELO Rating then be player based? Or divided upon between the two games?

[rbodleyscott]

Also: this is Medieval. Will the same apply to Ancients in the next patch?

That is the intention.

And the ELO Rating then be player based? Or divided upon between the two games?

There will be separate ELO rankings for each game.

[rbodleyscott]

These would be my initial questions;

a) Is win/loss and the difference between existing ranking at the start of the game the only factors affecting the ELO changes?

Yes

Is the difference between % losses intended to be used?

No. The ELO system is a generic one to use for all applicable Slitherine games (with separate rankings for each game).

There is no scope for tailoring the system to individual game systems.

(It could no doubt be done, if development and maintenance time was infinite, but it isn't. Like the Slitherine Tournament system, it is generic, and Slitherine want it to stay generic).

b) How is it intended stalled games will be treated – is it a default loss for the awol player?

Yes

c) In mp games outside official tournaments is it intended that there be a criterion for a minimum number of turns before ranking is applied (e.g. 6 turns/ 3 per player) to allow for rejection of outlier maps or a mistaken set up in a community tournament?

There is no such intention currently. I will put the point to Slitherine.

[ericdoman1]

3. What about those players who may only play in a few tournaments (you and I for example), compared to those who play in all. Is the rating system going to work on the average or total.
For example Player A has played in 6 tournaments, total points are 6000 so average is 1000. Player B has played in 12 tournaments, total points is 7200 but average is 600. If total points Player B with 7200 will be above Player A with 6000. If average, Player B would be above Player A.

Do you think perhaps you are taking this a bit too seriously? It isn't international chess rankings and it isn't intended just for tournaments.

It is more of a "ladder" system.

Come on Richard. If total or average is classed as being serious, then heaven help us. We have both been playing tournaments for a very long time now so we should know what are the best options.

Keeping things very simple is always a good idea but it will not be 100% accurate.

You could have everybody start on the same number and then 10 points for a win and 5 points for a draw.

You could then change that to say 12 points for a win in HOML, 11 in TDC, 10 in SLitherine and WTC tournaments.

You may not recall this as you weren't involved in playing many single TT tournaments but ratings were based on positions not on who you played. The top tournaments would score you more points. For example, Roll Call, BHGS Challenge and Britcon. Then I believe you had Warfare and then the various double tournaments. All used the Swiss system. An ELO system was used but dopped shortly after, it didn't work? This is probably the simplest method of rating players.

When we decided to use an ELO system for TDC, I think we copied FIFA's system. The formula and the final rating would take into account, who played whom (using their ratings) and I believe the score as well. Initially we worked out the base figure for players by using the Digital Leagues rating system and or the simple Slitherine rating system. Luckily we had karvon, carpenkm and Triarii to help us out.

So you have 2 options a simple system but not accurate or a more complicated and accurate system. Both are very easy to use, the formulae have been calculated so all you need to do is enter the data. Maybe you could have one of each, try them out and after a year or 2 decide on which is the best?

It would have been so much better if you had contacted the FOG II community from the start so it would have been a joint venture.

FAO David Sharrock - I'd recommend speaking with karvon, kronenblatt, carpenkm, tyronec and Triarii, if they are willing