Possible New Treatment for Alternatives

[richard]

I've split this topic out from a thread where it was hiding to try and get some more feedback:

When fixing a bug found by Bubak recently (in some situations you could get alternatives given to moves that led to YOU being mated), I decided to play around with some different treatments for alternatives.

I have some code (not yet uploaded to live site) which
a) Removes the 'mate in 1' has no alternatives rule.
b) Allows all alternative mates that are 8 or less in length, so if previously you found a mate in 8 and there was a mate in 2 for example you would not be given an alternative, well now you would.

This is all done on the UI so the downside is that there will be some moves where the 3 best 'runner up' moves are all alternatives, this means that there may have been even more alternatives hiding under the top 4 moves that the engine didn't see.  I can fix this during the next generator run by applying these rules at the generator phase inside of in the UI (at the expense of losing some problems, but I already have over 3k more new problems to add to the set).

So the question is, should I upload these changes? What do the forum readers thinks? (I'm probably going to upload them, but someone might be able to come up with a stunning reason why I shouldn't, [maybe there is a massive silent majority who actually loves getting hit with the 'you missed a mate in 1' stick :-) ]).

Regards,
Richard.

[drahacikfm]

This should perhaps be a new topic, but I'll post it here anyway.  When fixing a bug found by Bubak recently (in some situations you could get alternatives given to moves that led to YOU being mated), I decided to play around with some different treatments for alternatives.

Hmm, I was gone for the weekend, I'll have to find out what Bubak (my son) was doing!

I have some code (not yet uploaded to live site) which
a) Removes the 'mate in 1' has no alternatives rule.
b) Allows all alternative mates that are 8 or less in length, so if previously you found a mate in 8 and there was a mate in 2 for example you would not be given an alternative, well now you would.

So the question is, should I upload these changes? What do the forum readers thinks?

I see the argument in favor:  All winning moves should be allowed as alternates.  Mate in 8 is certainly winning.

But I see the argument against:  If you have a mate in one, and you don't see it, and instead you play a move that leads to mate in 8, and maybe you didn't even know it was mate in 8 but you just thought it was a good move, should you really get credit?  After all, you missed a mate in one!  That's pretty sloppy.  In a real game it only matters that you win.  But on a tactics training site you are supposed to be playing the best move.

My opinion is I don't see much wrong with the way things are now: fail if you don't play mate in 1, fail if you have mate in 2 and play a mate in 5, fail if you have mate in 3 and play a mate in 7, etc).  But I guess it wouldn't be horrible if all Mates in 8 are allowed.

It will cause the rating of some problems to go down, as they will now have more alternates.  I don't think you should assign new ratings to those problems though.  Just let the rating naturally change as a result of the extra alternates.

[revenant]

A mate-in-9 may only be 9 because the opponent happens to have a string of spite-checks that chase our king behind some pawns or something.  So in some cases it may be no harder (or not much harder) for a human player to find than a sequence in another position that even the computer agrees is rock-solid mate-in-3.

I'm not sure what bearing this consideration may have on either the current policy or the projected policy or the question of which is better.  Just wanted to point out that it might be a bit of a wrench in the works when conceptualizing either way, as it's not an aspect of the position that a computer can automatically perceive.

On the face of it, I prefer the current scheme but on reflection, that's just because I've gotten used to it.  The new idea has some merit and I could get used to doing things that way too.     The only really important thing IMO is to describe the policy in detail for users so we know the terrain.  For example, the site's time penalty on the second move feels a bit arbitrary and idiosyncratic, but the excellent documentation on it is crystal clear so I think we all have learned to adjust our strategies in response.

[oded ross]

On the face of it, I prefer the current scheme but on reflection, that's just because I've gotten used to it.  The new idea has some merit and I could get used to doing things that way too.     The only really important thing IMO is to describe the policy in detail for users so we know the terrain.  For example, the site's time penalty on the second move feels a bit arbitrary and idiosyncratic, but the excellent documentation on it is crystal clear so I think we all have learned to adjust our strategies in response.

I agree, and prefer the current method to the suggested changes.

[richard]

I'm bumping this thread as everyone who replied seemed to prefer the way things are, but there are some recent posts (and lots of problem comments) that suggest others may think differently.

Personally I also prefer the current situation, but there are enough opposing views being expressed that I hope this thread can see a bit more discussion from those who'd prefer to see the 'new treatment' see the light of day.

Richard.

[dougy]

My first reaction to this was "yes please!"  Then after thinking about this for a while it sounds like another band-aid solution.  I really wish there was something more elegant (although I can't think of anything right now).

[dhingzpe]

I have some code (not yet uploaded to live site) which
a) Removes the 'mate in 1' has no alternatives rule.
b) Allows all alternative mates that are 8 or less in length, so if previously you found a mate in 8 and there was a mate in 2 for example you would not be given an alternative, well now you would.

I think that for a, the alternate must be mate in 2.
But for b, I think the current way is better.

[revenant]

Well, if I follow the discussion in the tablebase thread properly (not sure I do, as the logic both for and against becomes somewhat convoluted), it seems more and more a case of "Both sides are right" and what is really needed is to improve the system in other, more general ways that will take care of the difficulty in question here as a beneficial side effect.

What I mean is, imagine if you will a Chess Tempo some time in the future (say, a year from now?) that has acquired the following three desirable enhancements:

1.  Endgame tablebases are incorporated into the generator's evaluations.

2.  The database expands to include a whole tree of moves for each problem rather than one linear sequence (with alts).  The user is led down a different branch of the tree depending on the move played at each position.

3.  A rating recalculation is performed not only at the end of the problem but after every move the user plays  (anyone who would like to read my arguments in favor of this feature, please see previous threads "Rate the position, not the problem" and "Pruning algorithm and one-move problems" from July & August 2008).

Evolution #1 will bring a good degree of valuable "chess sense" into many problems that didn't quite have it before (subjectively speaking, as part of the "user experience").  A win is a win, and a user who finds a winning move should be encouraged somehow, not necessarily by a proportional rating increase (as some have proposed) but at least by the "alt" mechanism (which is a *huge* improvement that I don't doubt has made the site attractive to many users who would have otherwise stayed away).  Tablebases will help the generator catch the best alts and accurately sort them, ergo fewer complaints will arise on this issue.

Evolution #2 will increase the training value of the site by enabling a "kind, but firm" teaching paradigm.  (Oh, no.  Did I just say paradigm?!  Pretentious blighter I am.  Just shoot me.)  If a mate-in-4 is there and the user doesn't see it but does find a move that's mate-in-8, the system can lead them all the way down that branch of the tree and make them *prove* they know what they're doing and they can reach the culmination of their idea.

Meanwhile, evolution #3 will reduce their "tactics rage" when they *don't* find the right followups...  and so the comments areas and the Forum will receive fewer gripes!    How?  Because the more granular rating system will show them exactly which move was wrong.  In other words, if currently they would receive -4 points, in this evolution they will receive perhaps +2 points on their (correct) 1st move and then -6 points on their (mistaken) 2nd move.  The overall adjustment is the same but it will be more palatable because there has at least been some form of reward for the good decision at move 1.

In that light, my recommendation is to stick with the current policy on mates and alts, not because it is any better (or worse) than the change proposed to that one area of the system, but rather because the design effort & coding time might be better spent on evolutions #1, #2 & #3 which are going to take some serious forethought if they are going to be viable.

[uri blass]

The main problem for me is considering better moves as wrong and worse moves as right.

I think that it is even better not to allow +10 alternatives to mate in 2
relative to the possibility of allowing +10 but not allowing better moves of mate in 5
when both may need deep search.

Alternatively you can use small search of the computer to decide if you allow alternatives(if the evaluation of the computer after the alternative is good for the solver at small depth then you can decide to allow it because it means that there is a good reason for the alternative that you can see easily and if the computer needs big depth to see that the alternative is good then you can decide not to allow it.

Uri

[drahacikfm]

I think I weighed in before on this thread with a vote to keep things the way they are now.  I don't see any huge problems now.  Once every 20 problems or so I wish the system was different.  But if the system changed, I might be irritated more often than one in 20.

The one change that might be good is to allow alternate moves of Mate in 2 and Mate in 3 when there is a Mate in 1.  But not mate in 8, that seems ridiculous.  If you play a Mate in 8 when there is a Mate in 1, you are obviously missing something simple and big in the position.

Another argument against the Mate in 8 thing when there is a Mate in 2 or 3 is that I doubt very many people playing that move know that it is mate in 8.  They just think it's a good move, and maybe for the wrong reasons.

[revenant]

But if the system changed, I might be irritated more often than one in 20.

Yes...   Once when I was a kid I saw a "Frankenstein" parody sketch on TV in which the creature, who has yet to be reanimated, is lying stitched together but unconscious on the operating table, and Doktor F. is busying himself about the lab in preparation for supplying the initial lightning-bolt of electric power.

All of a sudden for no apparent reason the monster's left leg rises to a 45-degree angle and stays there with one foot in the air.

Doktor F. notices this, comes to the table, presses the monster's left leg back down to the table and goes away.

The monster's right leg then rises to a 45-degree angle and stays.  Doktor F. returns and presses the right leg back down to the table...  Whereupon the left leg rises into the air again. 

I don't remember if the good doctor eventually discovered the cause of the problem, but it does seem an apt metaphor for the willy-nilly hither-thither effects that can come from well-intentioned changes to a system.

[charlesallen]

Has the idea of allowing "threefold repetition" of "good but not the best" moves to be a draw come up?  Sometimes, I just can't find a better move, and having to settle on one that I think is worse, on the off-chance it's actually better, just seems to teach the wrong thing.

[revenant]

See the Forum thread Allow user to repeat position? (e.g. Problem 52069) (October 2008).

[uri blass]

I think that the mate rule is simply wrong and in some cases it allows too many alternatives.

Winning material is not better than long mate.

For example problem 48926
People admit that they played 2...Qxb2+ for the wrong reason and it did not cause them to be marked wrong.


I think that it is better to cancel part of the alternatives not based on computer score after deep search but based on computer score after a search
to small depth.

A possible rule can be that a winning alternative is considered to be correct only if after the alternative toga can see at depth 4 that the opponent is losing.
 
It may mean that part of the long mates alternatives and the moves that win a lot of material  are not going to be considered as correct if they do not lead to fast win of material and part of the long mate and the winning moves that win material are going to be considered as correct in case that you need short search to see that they win significant material.

Uri

[drahacikfm]

It is very dangerous to use evaluations at low depths, such as depth 4.  The computer can think a move is winning, when in fact it is losing, at low depths.  Or the computer can think that a move does not win, when in fact it is completely winning.

The best method is to have the computer get as close as possible to the "truth", using the greatest depth possible, and then make the best possible rules for alternates.  There is no point in using evaluations that are not close to the "truth" and might even be completely wrong.