new idea for ratings

[d-man]

okay, i'm new here. lovely site, really, but the ratings seem just a bit of a shame. why not gain or lose ratings proportionally to how well the computer evaluates the moves, instead of one best answer. even titled players won't always get Toga's best evaluated move, so it  seems impractical for gaining real game-playing strength. It is discouraging to get "failed" for a move that wouldn't necessarily lose a real-life game. so you'd still get extra bonus for getting the very "best" computer move.

[tacto]

Well this isn't playing. So it will never measure playing strength.

While looking for Toga 2's (ELO 2800) best choice, we can all learn a lot. Giving points for 2nd best moves would be way too complicated to implement, especially right now.

Maybe something like the second or third best moves get some merit if they don't differ more than a pawns(1) value from the first best. Who knows...

[d-man]

thanks for your reply. i think i understand what you are saying, it would take a major overhaul of design. well, it's not terrible the way it is.

i dunno about others, but the only reason i do these is to get better in real games. in real games, if i play a move that wins a queen, it most often virtually the same as mate. since in a real game we deal with managing time, it is often a waste to think further after seeing a definite winning move. i know of a couple of titled players that got frustrated with this site for that very reason.

at least take away the "failed" stamp. something like "winning but not best" might have a softer impact on the pysche.

[richard]

at least take away the "failed" stamp. something like "winning but not best" might have a softer impact on the pysche.

Hi d-man,

You'll be happy to hear that the next problem set release will do two things
1) Do a better job of identifying alternative winning lines.
2) Allow the user to be told "Good move, but look for another" when they play an alternative.

I'm still processing the new set, but if no new bugs in the generator show up, the new set (and UI changes to allow alternatives) should be available in 1-2 weeks.

Regards,
Richard.

[d-man]

Great! very nice site, btw, I prefer it to the other tactic sites

[slacker00]

Welcome, d-man.  I agree with what  you are saying, it would be nice to get further automated support and guidance on problems, but richard is doing a lot of work on this right now.  So stick  around and you'll see constant improvements.  I've only been here about a month, and the improvements so far are like night and day.  I can't wait to see  what improvements come down the road.

[tmr]

Richard

What's your thought on rating inflation with the new alternate move algorithm?  Seems as if the higher rated problems are more likely to have alternate moves and thus higher rated players will more likely to not loose rating points for getting a good percentage of their problems "wrong" (that is higher rated players will see a higer percentage of these types of problems in their problem draws).  Not sure I see an easy solution to this.

[richard]

tmr:  I think there will definitely be some rating inflation.  I'm not that worried about rating inflation as long as it is bounded, i.e. if the highest standard rating say goes up by 200 then I don't have an issue with that, but if it keeps growing as blitz ratings have in the past then I'll be concerned.  In any case I'd guess that standard ratings are probably around 100-150 under the equivalent OTB rating at least at the top level.  The main reason I don't like constant changes that impact ratings is that it makes it hard for users to use their rating as a measurement of progress.  I'm hoping that the type of changes that tend to impact ratings will start to become a lot more rare after the alternatives are in place.  The only item I have on the todo list that should impact ratings is to tweak blitz ratings a little to get them a bit closer to standard ratings.  I've just reduced the reward for fast solving a little bit and I'll be interested to see how much effect that will have. If blitz ratings don't move much I might consider rewarding even less for fast solving as well as punishing a bit more for slow solving.

Having said all that, a number of the changes I made recently had a lot less impact on ratings that I expected. I'm hoping the impact of alternatives is relatively short term.  The problems that people were getting wrong due to alternative winning lines will drop in rating and users getting them right will increase in rating and get exposed to harder problems.  The trick will be generating enough difficult problems to fill the hole.

Regards,
Richard.

[tmr]

Well with the alternate move algorithm you'll should be able to expand the problem set by including all of the mate in N and mate in N+1 problems again.  There are probably a fair number of more difficult mate problems with multiple lines that were thrown out last time.  This should help at the high end.

Or are you only implimenting the alternate move algorithm for non-mate problems?

[richard]

tmr: The new generator does save a few problems that would have previously been thrown out due to alternatives. However as I'm also now a lot better at detecting alternatives , there are quite a few problems that get thrown out due to too many alternatives (I can only look at so many lines at a time so if the N best are all alternatives I have to throw the position away as  don't know if N+1 would have a winning alternative). The ones I am able to keep seem to be almost canceled out by the extra ones I am throwing away although It looks like overall I'll probably be keeping a little more now.

Regards,
Richard.