Problem 40402!

[tama]

The Problem is generaly easy but 0 people have solved it correctly because the end solution is a choice of 2 pieces capturing a hanging piece.

Rxf4 is Winning
Bxf4 is Winning

They both win but Bxf4 seems to be the winning move and there is no alternatives for Rxf4.

I think Rxf4 should have a (Good move, keep looking).

[richard]

Unfortunately Rxf4 is rated at about +2.3 which wasn't enough to pass the alternative threshold.  The new generator has a much lower alternative threshold and would have allowed that as an alternative, although it would have ended up rejecting this position altogether as the new generator also enforces a minimum of gap between best and second best move which was not met here. I need to make a few more changes before I ran the new generator of the current problem set.

As stormcrow points out in the comments Bxf4 was probably preferred due to the discovered attack threat on the white rook on Bxh2+ which white has to waste a tempo countering.

Richard.

[slacker00]

Being expected to choose BxN rather than RxN in this situation seems too close to  mark RxN wrong.  Or else I need an explanation. 

This goes beyond tactics and becomes more of an endgame theory problem, at best, I expect.  Nothing wrong with that, but I still need an explanation why RxN is meaningfully worse than BxN.  In this position, I don't see one tempo being at an extreme premium, especially seeing the continuation of the game line.

As it is, I feel like this kind of problem needs to be tossed out.   I can't learn anything meaningful from it, from a tactics problem standpoint.  Like I said, maybe I'm missing the point of the problem.

[richard]

slacker: No I don't think you are missing anything, it is simply not a good problem. At the moment there are quite a few problems where the second best move is so close to the best move that it is hard to justify why one is better than the other. This is the case here. Normally this isn't that bad as the second best move is usually an allowed alternative in these situations. In this particular problem the second best move was quite close to the best move but was also below the alternative threshold which led the second best move to be marked wrong which makes this problem particularly annoying.  The new generator does toss this problem out.  I'll be running the new generator over the problem set ASAP but before I do that I want to add the ability for the generator to keep all the 4 best moves, their move sequences and their evaluations for presentation to the user.

Regards,
Richard.

[slacker00]

Thanks Richard,  for the answer.
 
I'm definitely excited about the prospect of being able to examine different continuation lines in a problem.  I appreciate any little tool or trick to allow the user the do a "post mortem" examination  of a problem to help comprehend why this answer is correct and the other answer is wrong or inferior or whatever.  I know it's tough to build in this type of functionality, but any little bit of explanation of the various continuations helps  me a lot to help me learn!

I know I could set up the problem with a chess engine and use the chess engine to answer some of my questions about possible continuations.  But, it takes a bit of time and effort for me to set up a position with the chess  engine, not to mention the fact that I'm not that talented with dealing with chess engines for  analysis purposes.  I suppose learning how to use chess engines for analysis is something I need to learn to take the next step in my chess development.  I guess I'm just not that  committed to taking that step at this point in time.

[richard]

Hi Slacker00,

One of the premium features is the ability to grab a PGN description of the start position of a problem, this makes setting up the position in the engine a simple cut and paste operation.

I'm nearly done with the generator modifications required to keep more of the computer analysis around for presentation to the user.  I need to decide to what extent this will be  available to free versus premium users.  On the one hand I'd like to make it free as at the moment there are a lot of problem comments and ratings based on inaccurate analysis of the problem.  This is especially true for "my move was better than the 'best move' but was marked wrong" comments.  This is almost always not true (there are a few cases where this is true due to toga not seeing far enough into the position, but these seem to be rare [despite a legitimate one of these being posted in the forum today :-) ]). Some common problem criticisms are valid such as those caused by the issue mentioned in my previous post where alternatives can be very close to each other in evaluation, it is not uncommon to have a longer analysis (or different engine) give one of the alternatives as better than the 'best move' (at least in this situation you don't lose rating points).  The comments that a position requires more moves are also almost always valid criticisms. I think providing more details of the computer analysis might cut down on the first type of comment where users are often missing important opponent moves and therefore incorrectly blaming the problem instead of their own faulty analysis for failing a problem.

On the other hand I also need to try and encourage signups and the site already offers a reasonable amount of functionality for free, so apart from problem set quality improvements, most of the new enhancements will likely be going in as premium features. The analysis presentation however has the potential to keep more users around and therefore possibly increase total premium user signup. I'm sure some users make a few mistakes, blame the problems (often incorrectly) and never come back. This is probably especially true for CTS users who are used to getting much easier problems on average so when they start getting a 50% success rate where they previously were getting a 80%+ success rate (assuming they are ignoring the clock) they automatically assume there is a problem quality issue [I've done some sampling of CTS problems recently and it appears the problem set quality on here compared to CTS is fairly even now]. So the bottom line is that I haven't decided how much of the analysis presentation I'll be providing for free users yet (if any).

Regards,
Richard.

[tmr]

I don't think you need to make the detailed analysis of the best lines a free feature.  As you mention, the free content is already quite robust and you need more distinction with the premium features to encourage sign up.  I think you also need more distinction between the silver and gold levels.  The current difference (length of problem history) can be easily over come with a simple copy/paste to a spreadsheet (which works at the free level also especially if you do fewer problems).

I don't think incorrect comments is a problem and when you get the better link between the problem set and forum will probably increase participation as folks add comments to correct the incorrect ones.  People like to do this.  Folks just need an easier way to find and respond to these.  As for folks not coming back because of perceived problem set quality, I think you've got this well in hand and lack of the analysis or ability to copy a PGN isn't much of a problem.  I routinely evaluate all of my missed problems on a handheld without much effort at all.  On the occasion I want to put it into an engine on my computer, a PGN would help, but it isn't required.  I don't think any serious player would be detered by this and if they can't take the seconds required to enter the position in a engine probably aren't going to be swayed much by analysis of alternate lines.

Added features aside, I think you're working on the most important features now; problem set quality and size (especially at the high rating end).  To be a really useful site the problem set selection routine has to be locked down so users can rely on their ratings changes to evaluate their performance increase.  Currently only rank is a good indicator but even that depends on a static group of players and that is hard to confirm.  For me this is the most important issue right now (I'm waiting for the next update).  While I'd like to think my several hundred rating point increase over the last few months is attributed to a performance increase I'm pretty sure its really been due to the changing problem set and selection methodologies.

Problem set size at the high end is also important.  As it is, ratings at the high end aren't as comparable as lower ratings because the highest rated players will tend to have increasing ratings over time even if their skill level stays the same because there aren't enough higher rated problems to give them a normal distribution in problem selection.  This could cause disgruntled users on the lower end as there would be no way (except inactivity by higher rated users) for the lower rated uses to catch up as the rating is more directly linked to the number of problems solved rather than skill (though obviously skill got them to that point to begin with).

[richard]

Hi Tmr,

Differentiation of gold and silver is certainly an issue at the moment, with the relatively small differences in features between silver and gold membership, gold is really more of a donation than anything at the moment (although for those who have done >10K problems it does make sure their problem history stays around - at the moment I'm keeping everyone's history (even if the user can't see it all at their current membership level), but it may get to the point where I have to cull old problem attempts, gold members will be certain to have at least 50K problem attempts kept, silver 10K and free users only 1K).

Constant flux in the problem set has also been an issue , as you point out rank is the only reasonable measurement of progress at the moment due to problem set changes impacting rating stability.  The next update looks like it will impact quite a number of problems, the new more conservative thresholds will knock out quite a number of established problems. I have about 4K new problems to replace the culled problems which will take a while for the rating system to digest. I expect this to be the last of these major updates (barring any serious bugs) and that after the next update I can concentrate more on producing new problems.

Inflation at the top end doesn't appear to be as much of a problem now as it once was.  Blitz in particular tended to drift up due to too much reward for solving problems quickly. I've tweaked the blitz rating system a bit over the last month and it seems to have mostly dealt with high end ratings drift. While there is a shortage of problems at the high end Glicko tends to deal with this quite well.  The % success rate of the highest rated players is quite high due to not having enough problems at their current level, however they also get less points for the lower rated problems and when they get a lower rated problem wrong (which does happen) they lose a lot of points. Looking at the high rated players who have done say over 1000 problems, it appears their ratings have become reasonably stable (edit: actually I just checked again and things haven't quite settled since the last update, but for the users that have done a high number of problems since the update it appears they have reached a new ceiling), although users do tend to have quite a long "learning period" where they start to learn some of the quirks of the generator, for example, if you see a tactic that wins a pawn that is probably not the answer etc.  There is another concern however with relatively small numbers of problems at the top end and that is that eventually I run out of problems even remotely close to the high end users rating and they will start to get duplicates (although the selection process makes sure they are not recent duplicates), this is the main reason I'm keen to get more problems into the set.

I'm still ironing out some bugs with the newest generator but hope to start running the new generator over the current set within the next week, so that would put the next problem set update about 3 weeks away.

Regards,
Richard.

[tmr]

Thanks.

I had reviewed a fair number of high rated player's statistics and as you mentioned was probably seeing the fact that the ratings hadn't settled down totally after the last update.  I think you'll get there when you have the generator locked down.  Then it can just start churning out new problems which you can slowly add, probably selectively to fill in the holes in the current problem set.

[richard]

tmr: I've just started the new verification run, this took about 2-3 weeks last time. This one may take a bit longer as I may have to generate more replacement problems as I don't want to drop below about 15K problems.  I've also almost finished the UI for showing the alternatives which I'm quite excited about.  CTS already displays crafty analysis for their problems, but I think being able to interact with the variations on the board is a lot more useful than just having the text output of the engine analysis.  The display will be similar to the move list on the pgn viewer page, with slightly different formatting, and hopefully by the time I've finished, a scrollable move list so you can see all moves and the board at the same time. For now,  variations are only shown for the user's 4 best moves, not the opponents moves.  I'll be adding variations for the opponents moves in the future as well engine analysis for the common mistakes made for each problem (which might not be in the "top 4" variations list).

Regards,
Richard.

[tmr]

This sounds like a great improvement.  It will be interesting to see how you decide on including the feature within the member levels.  I think this really would differentiate between the silver and gold levels.

I didn't know CTS offered an analysis of the lines.  I really don't frequent that site much as I find its blitz only mode not suited for my age slowed brain.  Unfortunately for me my real chess days are long over.  Now its just encouraging my kids and trying to stay better than them.  One of my daughters almost beat me the other day when I blundered while playing both at the same time.  I was proud when she snatched up my queen without hesitation.

[richard]

tmr: I still have a bit more thinking to do on what member level to aim the variations view at.

I've been beaten by my 6 year old in blitz games, he is still a bit off being competitive in longer games as I make less blunders, which is the main difference between us at the moment. He often sees more tactically than I do, but has trouble seeing his opponents possibilities.  I was helping him with tactics here for a bit and we'd look at the problems together, a combination of my patience and his tactical vision together achieved a rating on his account a couple of hundred points above what I can get here on my own :-) (he's dropped down since then, without being encouraged to think through problems he tends to play reflexively most times).

Richard.