New Problem Selection Algorithm

[richard]

dlester (sorry dlester, I missed responding to this when you first posted) & tmr:

I think the idea of changing the width of the rating selection range at different rating levels is reasonably feasible.  I don't think the the issue of different levels of reward/penalty need to be dealt with explicitly as glicko does a good job of dealing with this.  I think this was seen when the problem selection process changed recently, there were big changes in the % correct being achieved by users, but that wasn't reflected in big changes in ratings (there was some change but not huge).

I'll need to have a bit more of think about this one.

Regards,
Richard.

[drahacikfm]

For my son, who is just starting to play and is six years old, this problem selection method is impossibly cruel. 

If you are worried about your son getting discouraged, the new premium features will let you create any kind of problem set you want.  For example, you can create a set of problems with ratings under 1000 standard.  Then when your son finishes all of those, make a problem set for 1000 to 1100.  Etc.  Or make a problem set with Mate in One.  After finishing that, your son can do Mate in Two, etc.  These are all unrated, of course.

1) a variable selection range so that as your rating increases the width of the range of problems gradually widens.  This will help prevent any discontinuities in the ratings.

I'm not sure I understand the reasoning behind having a narrow rating range of problems served to weaker players, and a wider rating range to strong players.  My thought is that the difference between an 1100 problem and a 1300 problem is not as much as the difference between a 2100 problem and a 2300 problem.   2300 problems are almost impossible for me to get right, but 2100 problems I have reasonable chance.  The range I get now seems wide enough.  I wouldn't want it to be a lot wider.

But anyway, the premium features that allow you to make all kinds of problem sets for unrated practicing should satisfy all your training wishes.

[dlester]

If you are worried about your son getting discouraged, the new premium features will let you create any kind of problem set you want.  For example, you can create a set of problems with ratings under 1000 standard.  Then when your son finishes all of those, make a problem set for 1000 to 1100.  Etc.  Or make a problem set with Mate in One.  After finishing that, your son can do Mate in Two, etc.  These are all unrated, of course.

I am not worried about him getting discouraged as I am right there with him to encourage him.  But, now that the rating of the problem is hidden until you attempt it, I end up solving it in my head and then deciding if I want him to concentrate on it a while based on my best guess of how difficult it is.  When the problem rating was displayed I would just skip until a sufficiently easy problem was on the screen for him to attempt.

The premium feature you are describing would be a step in the right direction, though I think more reasonable problem selection could be implemented (and fortunately the idea seems well received by Richard).

I'm not sure I understand the reasoning behind having a narrow rating range of problems served to weaker players, and a wider rating range to strong players.  My thought is that the difference between an 1100 problem and a 1300 problem is not as much as the difference between a 2100 problem and a 2300 problem.   2300 problems are almost impossible for me to get right, but 2100 problems I have reasonable chance.  The range I get now seems wide enough.  I wouldn't want it to be a lot wider.

You are a considerably more skilled player than I am, but respectfully the difference between a 2100 and a 2300 problem is theoretically much less than the difference between an 1100 and a 1300 problem.  The former represents a problem that is 9.5% more difficult ((2300-2100)/2100) whereas the latter represents a problem that is 18% more difficult ((1300-1100)/1100). 

You have probably not been a beginner for a very long time, so it is easy to lose touch with what it is like to start playing a game as complicated as chess.  I coach a scholastic chess club in WA State.  I get kids that are total beginners all the time and I get to see them struggle from learning to just move the pieces, to finding hung pieces, to starting to recognize basic tactics, until finally they are able to start maneuvering their opponents into tactically advantageous situations.  Each of these steps is huge in their development. 

Taking a student who is at the stage where they can really only manage to recognize very basic situations like hung pieces or back-rank checkmates (ie, someone in the U800 range) and giving them mate in 2 exercises rated over 1300 isn't going to help them develop.  They need easier puzzles until they are ready for more difficult challenges.  When they are ready, their rating will increase and they will start getting gradually more challenging puzzles.

I am not advocating widening the range for more skilled players.  When I said 'wider' with regard to that, I really just meant wider than for the lesser skilled players.  The range for those players is something that can be adjusted over time as deemed necessary, but it has a large player base to provide solid feedback.  Lesser skilled players, who are just developing their talents, are not going to get that kind of representation.  It would still be nice to take their needs into account when designing the site as they are exactly who you want to attract.  No other site caters to them at all.  If you can get them early and keep them happy and engaged, then you have a long-term customer.

[richard]

Hi dlester,

You make a number of good points there, I think that the beginners voice is certainly under represented in the forums. I'm much closer to a beginner than an experienced player myself so fortunately I'm not oblivious to their needs :-)

As I mentioned earlier, I think a narrower rating band at the lower end might be quite useful.  The only real reason against it that comes to mind is one of simplicity. The more complex the selection process the harder it becomes to analyse it's global effects.

At the moment user created problem sets are based on creating sets defined with search criteria, but I also plan to add support for ad-hoc problem sets where users can add specific problems to a set.  This would allow you to hand pick problems you think would be best suited for your child at their current level of play, rather than leaving it up to inhuman whims of the selection algorithm :-) This will not be available upon first release of the premium features but I do hope to add it not longer afterwards.


Regards,
Richard.

[drahacikfm]

You are a considerably more skilled player than I am, but respectfully the difference between a 2100 and a 2300 problem is theoretically much less than the difference between an 1100 and a 1300 problem.  The former represents a problem that is 9.5% more difficult ((2300-2100)/2100) whereas the latter represents a problem that is 18% more difficult ((1300-1100)/1100). 

Actually, the Elo rating system is designed so that only the difference between two ratings is significant.  When there is a 200-point difference, the higher player is expected to score about 75% over the course of a number of games.  To keep the same rating before and after, he should score one win and one draw in every two games (75%).  It doesn't matter if it's 1300 vs. 1100, or 2300 vs. 2100, or even 3300 vs 3100 (on some internet chess servers, ratings get that high).  You could add one million to every rating, and the rating system would still work the same as it does now.  We would be talking about ratings of 1,001,100 vs 1,001,300, a difference of only 0.02%, but still a difference of 200 points, which is the important part.  So the percents you mentioned aren't really measures of the differences in difficulty.

I've been teaching my 9-year-old son chess for several years now.  And a few other kids.  I remember when he was just starting out, the frustration when he couldn't get a problem.  Beginners don't need the added pressure of tactics ratings, so the customized unrated sets would be great for them.  After they get better, they can do rated.  Then the selection algorithm comes into play.

Right now the selection is centered on your rating, so you are expected to get about 50% right and 50% wrong.  (If you got more than 50% right, your rating would go up until your success drops to 50%),  I think 50% correct is not enough for kids.  To keep up their enthusiasm, they need to get about 75% right and 25% wrong.  A tighter rating range won't increase the success rate.  It will still be 50%.  But giving them more problems below their rating would increase it.  That's where the customized sets come in, because I don't think Richard would want to change the selection algorithm to serve 75% of problems below the user's rating.

Taking a student who is at the stage where they can really only manage to recognize very basic situations like hung pieces or back-rank checkmates (ie, someone in the U800 range) and giving them mate in 2 exercises rated over 1300 isn't going to help them develop.  They need easier puzzles until they are ready for more difficult challenges.  When they are ready, their rating will increase and they will start getting gradually more challenging puzzles.

You will be able to make a problem set for your kids with any kind of mix of parameters.  For example:  Mate in 2 problems with ratings below 1000.  Or back-rank-mate problems rated below 1100.  Etc.  Perfect for teaching.

[richard]

dlester: Drahacik is correct, the way chess rating systems work generally means that narrowing the range should not have a big impact on success rate.  However one benefit may be that absolute beginners might have a better chance of understanding why they get problems wrong if those problems are closer to their rating. I know I can get confused without a chess engine to explain the essence of the higher rated problems.

Another possibility is to consider allowing users to nudge the selection algorithm down even for rated problems.  I'll have to think through the maths on this but I think in theory the rating system should not allow users to exploit this in order to achieve inflated ratings, exploiters would get very low rewards for easy problems and get punished greatly for any mistakes. Even if this is feasible I'd have to think carefully about whether to offer this as a non-subscription feature as the free feature set is pretty full already and I need a certain level of differentiation to encourage signups.

Regards,
Richard.

[pollock]

I want easier problems to solve quicker . I like struggling vs hard problems but i think i gain more from master more basic stuff. Offcourse CTS is good for that and dont want to turn this into a clone But easier = better. On the upside can now fail a bit more often and not lose points so heavily .

[richard]

Hi pollock,

The premium user defined problem sets will allow you to do this in an unrated mode by creating your own problem set with problems between a rating range you define.  I'll almost certainly also offer a mode which allows you to shift the selection algorithm around a bit in rated modes. Having thought through it and looking at how stable ratings were in the face of drastic problem selection changes I believe this can be done without compromising the rating system, i.e. your rating will still be a true reflection of your current level even if you choose to do slightly harder or easier problems.

Regards,
Richard.

[tama]

For some reason i fined problem under 1500 useless and ive bin plaing chess for 1 year now, I think the best way to learn is let them play alot of games to get the feel of it, then make them analyse positions and fined the best move. Also tell them to Dl ChessMaster 10th edition with Josh Waytskin, he thought me everything i needed to no in a flash, and my first rating that i ever had was over 1500 on different Internet sites.

And i think after they reach over 1500 they should play slow games, like turnament stile, 5 min internet chess sucks and i fined it to be absolutaly useless.