Funny problem ratings

[tmr]

I suppose it is just a matter of problem ratings settling in after the upgrade but it I've seen a fair number of easy problems with higher ratings and harder problems with lower ratings. 

Take problem 21016 for example ( http://chesstempo.com/chess-problems/21016 ).  This is a simple one move mate and yet it has a rating over 1600 (at least when I did it).  I'd expect this problem to score more in the 1200 range or less.

[richard]

Hi tmr,

There is certainly still some settling down to do, but that problem has been in the system for a while (all the new problems are in the 30K+ range).  Lots of people seem to be missing the mate in 1 and trying to win the Queen with Rg7 so this problem has a surprisingly low % success rate. Hence its large rating compared to number of moves required.  There are definitely examples though of problems that are too easy or too hard for their rating due to not having had enough attempts yet to allow them to settle at a more appropriate rating.

Incidentally a change I made a few days ago that some might not have noticed is that there is now a provisional system for new problems where problems that have been done only 0, 1 , 2 or 3 times before don't have their full impact on users ratings. Each of these respectively will only give 20%,40%,60% and 80% of the full rating adjustment until they have been done 4 times when they will start giving 100% of the rating adjustment.

Regards,
Richard.

[tmr]

Interestingly I did notice yesterday when researching this that problems that hadn't settled down (i.e., not done very many times) didn't give proportionally as much rating increase as a comparably rated problem.  For example I was looking at the statistics of a 1200 rated player who only got a 0.5 rating increase for doing problem 40668 which at the time was rated 1428 while other comparable 1400 problems gave the player about 4 points.  I attributed the difference to the RD portion of the calculation, but it was probably in part your recent change also.  Interestingly the problem is now 1100 after 2 attempts.

[richard]

I'm hoping to get to the point soon where I can stop doing verification runs (which causes a lot of rating turmoil) and focus solely on generating new problems and introduce them in small enough batches so that they blend in a bit more easily.  I have one more verification run planned in the next few weeks and I have my fingers crossed that this will be the last one for a while (although I thought the same thing about the last 2 verification runs :-P ).

Regards,
Richard.