Forum

Quoted from InstantMuffin

That’s like saying open source encryption is bad. Either it’s bad by design or it’s bad by implementation.But it is bad. In a way open-source (and by an extent free software) is even required tor a proof of reliability (I’m actually quoting Richard Stallman on this). Only thing the valve point does is showing that valve’s csgo ranking system is bad by their own estimation.
Now I fully understand aspects being trade secrets to prevent rip-offs, but saying reliability is fully dependent on the secrecy is like saying good encryption relies on secrecy of the encryption process and not secrecy of the key. It’s bs every computer science freshman knows. Just be honest then, no one is different in that regard. Work deserves recognition and in a way sometimes keeping stuff to yourself is the only way to get proper credit.

No one is claiming to have made a perfect system that could not be gamed, I’m struggling to see where any dishonesty is coming from in this regard. Obviously pointing out that it’s a concern means that it’s possible.

Quoted from Gentleman Jon

[…]
No one is claiming to have made a perfect system that could not be gamed, I’m struggling to see where any dishonesty is coming from in this regard. Obviously pointing out that it’s a concern means that it’s possible.

The problem for me personally is that there is no accountability or proof/indication that the rankings are accurate, which is a must if this system is to be adopted by people. I appreciate that a lot of research and investigations have gone into this, but there is an incredibly obscure black box in to how it works under the hood. Without giving away weightings, you could perhaps list *things* that are taken into account for each class – at least people could point out missing things that they think should be included.

No-one knows how to improve their ranking other than by winning (which I believe you have stated), and there’s no proof the stats are accurate.

I sound incredibly pessimistic and I don’t mean to be, but if you’re going for obscurity in actual implementation details (fair enough), the system could use some accountability to somehow prove the data is reasonably accurate.

Could stats be generated based on logs from last seasons prem/div 1 officials and see how things come out then?

Quoted from Spreijer

People got so obsessed by stats, so we got a ranking system that doesn’t make any sense. What’s the point of this? Why can’t we just gather logs and filter it by ka/d, dpm etc. That would say a lot more than these ranks.

No it wouldn’t… This ranking system analyzes logs and compare it to the people of the enemy team ranking, so it attributes you a rating or classification but those metrics like KD are kinda irrelevant in this aspect

My point is, you could have a better score with a KD of 1 if you win by 1 round that with a KD of 6 and winning 5-0, the difference here being that you are playing higher ranked players which will be much more relevant to the ranking score than any other metric I reckon.

So the way I see this, it’s that this is the ranking system more nearing perfection we could have, of course there could always be some values to be adjusted and logs never give enough information like about that person calling or attitude in-game, but then neither does virtually any ranking system out there, it’s kind of 1 more metric to analyze your game, compare it to someone else, and may in some case be used for recruitment purposes.

As for the guy criticizing this for not being open source, that’s bullshit, any system of this kind, like a lot of security systems, DRM systems, etc, require security to obscurity to be trustworthy and provide some fidelity.

Want an example? Look at dota mmr system which is completely closed source and no one knows the exact parameters of it. Even then some time ago people discovered average damage in a game would account a large amount in the mmr placing matches, so a lot of people would only play Zeus in the placement matches which has a ulti that does insane global damage making it the hero with the highest overall damage, so in short people used that weakness of the system to be placed 1 or 2k MMR above what they should be. And that is having in account a closed system, now think if it was open, it would be so much easier to game it.

Quoted from foxy_dave

The problem for me personally is that there is no accountability or proof/indication that the rankings are accurate, which is a must if this system is to be adopted by people.

You’re a tf2center guy, so for you ‘adoption’ means using the ratings to balance teams, right? Which is something you don’t currently do. So all a system has to do is beat a random shuffle to be worth adopting, right? (Not that you even use a random shuffle.) I explained this a year ago. Still confused why everyone at tf2center believes a team balancer needs to be verifiably perfect rather than simply useful.

How about we test Jon’s system by predicting results? It’s easiest if tf2pickup is changed slightly to allow viewing of the ‘current match’ pages without having to log in.

Quoted from RTC

Any sort of statistic system in a team-based game, especially one as dynamic as TF2, can only aspire to be a mediocre representation at the very best. I appreciate the sheer amount of effort that has gone into it, and it’s probably by far the best attempt to try to create such a system for the game, but to make even an adequate system is as impossible as an echo without a voice to start it.

I’ll make a longer post some time later to explain why.

I can’t wait

Quoted from RTC

Any sort of statistic system in a team-based game, especially one as dynamic as TF2, can only aspire to be a mediocre representation at the very best. I appreciate the sheer amount of effort that has gone into it, and it’s probably by far the best attempt to try to create such a system for the game, but to make even an adequate system is as impossible as an echo without a voice to start it.

I’ll make a longer post some time later to explain why.

Lots of ranking systems exist for team based games, not only sports but also for a lot of esports, dota has one, csgo has one, LoL has one, smite has one…

Some more accurate than others but overall its possible to create such a system despite the limitations and variables of ranking a team game. Dota system has shown to work time and time, and variables like champions and items are much higher.

Quoted from fraac

How about we test Jon’s system by predicting results? It’s easiest if tf2pickup is changed slightly to allow viewing of the ‘current match’ pages without having to log in.

When the disparity of ranking between two teams reaches the point of between 1 and 2 standard deviations from zero then it “predicts” the result in 85% of cases, but that means that even in highly unbalanced games it fails to predict them in 15% of cases. Obviously there is room for improvement, but it’s also true that predicting the result isn’t the main goal, in fact the opposite is one of the 2 primary goals – making the result completely unpredictable. The point is that the 85% cases of predictable games will be eradicated but because the classification of those games exist in a mathematical domain then it may not be easy to determine what they were, where they went and whether they’re really gone.

There are some problems, particularly with 6v6 and it’s inherent propensity to produce one sided games even between closely matched teams. Highlander is much more responsive as it’s slower paced. I continue to investigate ways to improve things, that’s why it’s in beta.

Quoted from InstantMuffin

You just said earlier that secrecy is part of the process. So I disagreed. (my first post)
If you want a system that is less likely to be gamed (or gamed at all) it needs to be open. That’s what I’m saying and that’s what I’ve proven on an argumentative basis which I also backed up by literature.
Independent of that I fully understand and respect you not wanting to “share” the work to properly get credit. It’s an understandable tradeoff.

I’m sorry I should have been clearer, I’ve applied secrecy because I know it’s potentially vulnerable and because I also know that I’m using stats in a way that has not entered public discourse, so secrecy is an excellent way to preserve their efficacy. I shouldn’t have abbreviated this to “this process”.

I’m not sure why you would attribute a desire for personal glory to this failure to anticipate your criticism, I’ve been trying to get something implemented in the background for at least 6 months, and running the site is nothing but a headache.