Grid computing for CT problem set

[revenant]

Distributed computing for the Chess Tempo problem set (alluded to in today's "Working out the alternates" thread) is a cool long-term idea with intriguing possibilities.  There are probably many users (myself included) who wouldn't be able to afford the membership fee if it were high enough to truly reflect the huge amount of work you've put into the site, yet we would be glad to have an opportunity to "give back" to CT in some other way such as by contributing CPU cycles or coding.

The first question that comes to mind is, would we all have to be running the same chess engine?  When I paste Crafty 22.0 variations into my comments, I notice that its numerical evals are different from Toga's in absolute terms, though usually not in relative terms at a high enough depth, where Toga and Crafty usually at least agree that move A is a lot better than move B which is a little better than move C.  In middlegames depth 16-17 seems to be enough and in endgames it gets stretched out to depth 20-25.  Or (very roughly) about 5-10 minutes of CPU time in any case.

It's doubtful that there's even an approximate linear conversion factor between evals for any two different engines.  They're all talking in terms of "pawns' worth" but they have different ideas of what it means to have an advantage or a weakness in one's position.  Since CT's problem generator algorithms depend on (somewhat arbitrary) constants relating to ambiguities and material thresholds, it's hard to see how absolute evals from Crafty or any other engine besides Toga would be of any use to CT.

Then there is the whole rat's nest of troubleshooting issues associated with any grid computing project, such as building in checks & redundancy so one overheated node on the grid doesn't throw off the results for a bunch of chess problems.

[dlester]

Richard and I have briefly discussed this option in the past and I pointed him to some resources that would make it easier than coding the entire thing from scratch.  It is doable, but still a significant undertaking.

If he was to do it, I would expect he will deploy a package a CPU contributor will have to use in order to participate.  It will include an engine of his choosing, along with the interface to retrieve information from his server and feed back analyzed positions. 

If you think about it, this pretty much has to be the way it works.  He needs to get predictable results.  Allowing people to use any engine they desire would have the potential to cause problems he simply would not have to worry about if all contributors were using a consistent platform.  Because of the way problems are selected, and how there needs to be a definitive 'best line' in the position, there really isn't any benefit in using multiple engines as they should all ultimately arrive at the same conclusion anyway.

[revenant]

True, although different engines could play an auxiliary role by combing the problem set in a second pass after each generator run to provide a consistency check.  If the 4 best moves and their relative evals turn up different from Toga's on a given problem after completing a search to the same ply depth, the procedure could help root out obscure bugs in the generator algorithms or in Toga itself.

(This is as in mathematics, where it bolster's one's confidence to have independent confirmation of any result produced by a computer; e.g. after the Four Color Theorem was proved with a computer in 1976, independent teams reproduced the result by coding the idea from scratch on different platforms.)

[richard]

I'm certainly fairly keen to head in this direction, having a distributed generator is useful not only for tactics generation but dovetails into other non-tactics features I'd like to provide further down the track.  Dlester has pointed out some useful "toolkits" that allow creating distributed applications with relatively little effort.  I also agree with him that it would likely require a single engine at least for the tactics generation. Cross validation might also be useful but that would be a later goal. I'd have to decide at what level to distribute the tasks, the easiest would be to have each node in the network essentially running a toga instance (or an toga instance per CPU core available), however distributing at a more abstract level where each node gives evals for sub-branches of the search tree allows more flexibility and lends the idea to a wider array of applications.

Regards,
Richard.