On problem selection

[wald]

Hi there,

I gather from reading some previous messages that currently problems are found from real games by looking at large changes of the computer's position evaluation. While this is likely associated with tactics, it typically misses defensive tactics.
Another big problem, to my mind, is that many of the problems are semi-unique. What I mean is that there is more than one winning move, though usually one is best. In the real world, i.e. over-the-board (OTB), finding a convincing win is often sufficient: it doesn't matter whether you mate in 2 or 3; if I see a mate in 3 I give it.
Yet another problem is that some positions are already completely winning and finding the quickest mate is sort of pointless, at least when exercising for OTB situations.

While it is easy to complain about weaknesses, I actually want to make a suggestion to help fix these issues. The suggestion is to alter the problem selection mechanism. What I have in mind is to look for situations in real games which satisfy the following criteria:

1. the initial position appears not too far from equal, i.e. is not completely winning;

2. there is only one move, or only one sequence of moves, the ultimate outcome of which is significantly (say > 2 pawn units) better than any other (the solution to contain only moves significantly better than any alternative).

This may throw up a number of trivial re-takes, but those can be easily eliminated (and they are already in the present database).

cheers.

[richard]

What I have in mind is to look for situations in real games which satisfy the following criteria:

1. the initial position appears not too far from equal, i.e. is not completely winning;

2. there is only one move, or only one sequence of moves, the ultimate outcome of which is significantly (say > 2 pawn units) better than any other (the solution to contain only moves significantly better than any alternative).

Hi Wald,

The generator already uses heuristics very similar to both of those ideas (some of the early discussions in the forum pre-date the  introduction of the first heuristic and I've tightened up the second heuristic a bit since the first version of the generator).   Even with those heuristics in place you still get situations where say you have options of winning the queen or a rook. This meets the > 2 pawn criteria but technically a rook take with no compensation to the opponent is probably also winning.  Whilst these problems can be frustrating , the largest problem at the moment is with mating sequences where slightly longer mating sequences (or non-mating material winning takes) are not given credit.  I have an improved generator which helps with the latter but it still needs further tweaking before it is ready for prime time.

Thanks for the suggestions, problem quality has improved greatly since the first version of the problem set and hopefully subsequent versions will continue to remove some of the less satisfactory problems.

Regards,
Richard.

[wald]

What I have in mind is to look for situations in real games which satisfy the following criteria:

1. the initial position appears not too far from equal, i.e. is not completely winning;

2. there is only one move, or only one sequence of moves, the ultimate outcome of which is significantly (say > 2 pawn units) better than any other (the solution to contain only moves significantly better than any alternative).

The generator already uses heuristics very similar to both of those ideas

If this is so, why then are there still plenty of problems where the initial position is completely winning, e.g. #19656? Did you not re-run the old problems through the new versions of the generator?

Perhaps my original heuristic 2 was not strong enough. One could tighten it by demanding that the move (or sequence of moves) is the only one which either leads to a clear advantage or mate, or avoids a clear disadvantage. In this sense, the difference between winning the queen or rook (your example) is marginal, as both lead to clear advantage. 

Perhaps one can learn from other sites: I found the problems at http://chess.emrald.net/ quite useful (though your web interface is superior ).

Regards, wald.

[richard]

If this is so, why then are there still plenty of problems where the initial position is completely winning, e.g. #19656? Did you not re-run the old problems through the new versions of the generator?

I'm not sure I understand your point with #19656. This position is completely winning because there is a mate opportunity , this is the whole point of the position.  If you want to remove positions like this then there will be no mates in the problem set.  Based on material balance white was actually behind at the start of the tactic which is why this position is allowed, if white was already say a queen up then this position would not have been allowed.
[Minor edit: The above statement was not quite true, currently the "already won" heuristic only applies to non-mate positions, there are technical issues in applying it to mating positions which I'm currently testing a workaround for (but #19656 still looks ok to me)]

Perhaps my original heuristic 2 was not strong enough. One could tighten it by demanding that the move (or sequence of moves) is the only one which either leads to a clear advantage or mate, or avoids a clear disadvantage. In this sense, the difference between winning the queen or rook (your example) is marginal, as both lead to clear advantage. 

This idea is also already in place :-) if the second best move is also very good, then even if the best move is winning the generator will reject the tactic. Of course there is always tweaking around exactly what "very good" means  and perhaps this parameter could be stricter than its current setting (and this heuristic only applies for non-mating tactics).

Perhaps one can learn from other sites: I found the problems at http://chess.emrald.net/ quite useful (though your web interface is superior ).

CTS certainly has very good problems.  As it seems to use tactics that were actually played in real games it has less of the "only a computer would see that" positions than Chess Tempo.  However CTS also has a lot of positions which finish before the main point of the tactic has played out. Chess Tempo also has some of these , but subjectively I feel CTS has more.

As mentioned previously,  problem generation is under ongoing improvement. Hopefully as the generator involves the Chess Tempo problem set can have all of the CTS strengths and less of its weaknesses :-)

Regards,
Richard.

[wald]

Hi Richard,

sorry, I must have been confused about the problem number with 19656. I agree that this is not completely winning.

But take for example 25274. Black is completely winning: a full queen up and White's king is as exposed as can be. Yet this problem is rated rather high at 2027. This is because the solution is the quickest way to mate and many are tempted to go for any winning move rather than for the quickest one. The solution is thus irrelevant, as it is not the only winning or viable move: we do not want to practice how to win a completely winning position in the fastest possible way, but how to reach such a position in the first place.

Cheers.

[richard]

Yes, you are right, this type of position is less than ideal.  As mentioned above the "position is already won" heuristic has only been applied to non-mating positions in the past.  So positions that end in mate have not had this check applied.  In the generator I'm testing at the moment this has been fixed and mating positions are now also subject to this test. It will be a few weeks however before I finish testing and running the new generator.  I still think there is some value in finding mates in these positions as the tactical patterns and motifs still remain the same. However I do agree that the problems should reflect over-the-board reality as much as possible, and looking for mates in won positions does not really match what the player would be doing in a real game.

Regards,
Richard.