Problem 21400

[drahacikfm]

This problem should be disabled.

This problem is similar to many of the highest-rated standard problems.  There are several winning choices on move 2 or 3, and people keep getting marked wrong for playing winning moves.  That's why these problems have such high ratings.  Not because they are difficult.  20 people tried this problem, and they all got it "wrong", but I bet most of them got a completely winning position.

1.Rxc6 Ne3 and the problem requires you to play 2.Bd5+.  But the simple 2.RxQ NxQ 3.BxN leaves me with an endgame a bishop up and with a dangerous passed c-pawn, which I am going to queen by force soon.  I could win this against Kasparov using only 5 seconds per move!

The generator needs to consider more positions completely winning than it does now, and remove these ambiguous high-rated problems.

[drahacikfm]

I put this on Rybka, and on move 2, White has 10 moves that are completely winning.  Needs to be disabled.

[richard]

This is a good example of what is going wrong with the winning alternative test for non-mates.  I'll give some more details of what I think I've done wrong here. Essentially I'm comparing the evaluation to the wrong thing.  I look at the current evaluation and compare it to the pre-tactic evaluation, so for example if the position was already +2 before the opponents blunder and the best move is +6 but the second best move is +4 then the second best move looks only 2pawns better than the pre-tactics position so is not considered a "good" alternative move.  Obviously this is a mistake on my part and I should have just been looking at the absolute value of +4 and saying, "hey this is pretty good". So basically my code only worked well in this regard when the position was even to start with.  There is also an issue that I'm setting the cutoff too high anyway (just over +4) so even in positions that were level before the opponents blunder I can allow the position. This is especially bad as things become more simplified in the endgame and a bishop and a pawn advantage become much more decisive.  So the first issue is a straight out bug I can fix fairly easily, the second is a matter of picking the right value, I can probably afford to be much more strict here, possibly even going under +3 as the cutoff for "good alternative".

It is also possible there are bugs I don't know about yet, but the above issues seem to cover most of the problematic non-mates.

I'll be getting on to the next generator in the next couple of days.  I'll concentrate on the most obvious flaws from the current generator and hopefully I can get  improvements done ASAP and the verification run started shortly.  I'll probably also purchase another machine to churn through the problems a bit more quickly.

Regards,
Richard.

[drahacikfm]

I think it's not relevant at all what the evaluation was before the opponent made the bad move that allows the tactic.  For example, maybe the opponent had a mate-in-one that he didn't play!   The only relevant thing is the position that we see on the board when we are trying to solve the problem.

[richard]

I agree, this is just a straight out bug really.  Looking at pre-move to post-move evaluations does have a place in allowing me to know where to focus  but I'm certainly doing the wrong thing here in terms of alternative line analysis. I think I hadn't noticed this earlier due to my test cases being mostly even to start with.  In any case it shouldn't be too hard to fix for all these types of positions, I'm just annoyed I didn't catch it earlier (although perhaps not as annoyed as you are for losing rating points on these :-( ).

Regards,
Richard.