Message boards : Number crunching : Cross Project Credit Equality
Author | Message |
---|---|
Biggles Send message Joined: 22 Sep 05 Posts: 49 Credit: 102,114 RAC: 0 |
Jose, the sole point of mentioning Leiden Classical was to point out that it is a project that A) has a quorum and B) has never had optimised science apps. With those conditions met, I believe XS wouldn't have as high an RAC on that project as they do on Rosetta even if all members were able to join and get work. It was one project mentioned amongst several. I'll put my main point again in another way - on BOINC, one hour of CPU time on one project should give around the same credit as that same machine spending an hour on another project. 1 SETI hour ~= 1 Einstein hour ~= 1 Climate Prediction hour ~= 1 Predictor hour ~= 1 Leiden Classical hour ~= 1 Rectilinear Crossing Numbers hour and so on. Imagine there is a machine that gets around 10 credits per hour on SETI. It should get approximately the same on Einstein. But it isn't perfect, so maybe it gets 11 credits per hour on Einstein. And maybe it only gets 9 credits per hour on Malaria Control. But that same machine on Rosetta running an optimised client would be getting perhaps 30 credits per hour. That's not fair to those who use standard clients, or to those who have processors that don't produce such staggeringly outrageous benchmark scores. Nor is it fair to those who try and compete over BOINC as a whole. I want to see that past unfairness corrected. Not only that, but I expect to use a new stats system that awards credit based on work done, not on what is claimed by dodgy benchmarks. Yet if Rosetta keeps on exporting stats based on the current claimed and granted credit then it keeps on being inaccurate. EDIT - Another thing Jose, the DC Vault is managed by Team Ninja. They dictate what goes in it. The DC Vault also covers a lot more than just BOINC. It has no bearing on BOINC and is not what is the matter of discussion. |
Angus Send message Joined: 17 Sep 05 Posts: 412 Credit: 321,053 RAC: 0 |
I'll put my main point again in another way - on BOINC, one hour of CPU time on one project should give around the same credit as that same machine spending an hour on another project. 1 SETI hour ~= 1 Einstein hour ~= 1 Climate Prediction hour ~= 1 Predictor hour ~= 1 Leiden Classical hour ~= 1 Rectilinear Crossing Numbers hour and so on. Why is this so darned important? There is NO valid reason to compare results on DC projects under BOINC than it is to say distributed.net needs to have parity with SoB, or D2OL, or DPAD, or F@H, or any other of the myriad of DC projects. Each needs to stand on it's own, and recruit people and CPUs based on the merits of the project and how it's run. BOINC is simply a piece of software that is only one part of what allows a DC project to do work. It has no relevance to credit parity across projects. This is a myth that was started by David Anderson - something out of a bad dream. Maybe a few project principles need to stand up to Anderson, and say "Screw you - we're doing our own thing with credits. Too bad." Since BOINC is open-source, what can he do? Nothing. and maybe it will empower others to see the fallacy, and quit wasting precious man-hours on trying to achive the impossible. Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :) "You can't fix stupid" (Ron White) |
John McLeod VII Send message Joined: 17 Sep 05 Posts: 108 Credit: 195,137 RAC: 0 |
Leiden Classical another of the great Boinc Projects that have not allowed mew members in months and whose files get corrupted while its manager is enoying the sun at Curacao. OK, I just read through the Leiden thread at DC vault, and the moderator is aware that Leiden is limiting users, and (s)he believes this is a temporary problem, and is not going to remove Leiden for now just to add it back in a few weeks. I am assuming that if Leiden does not open up membership again in a timely manner, it will be removed until it does. My statement was that if the rules of the DC Vault were being broken, they should be enforced. The problem with Rosetta is that the expectation that all BOINC projects would produce about the same # of credits / hour of crunching on a given host was violated, thus leading to unfairness, allegations of cheating, and a general flame fest. In both cases I am advocating adhering to the rules. Just in one case a more general rule than the one you are looking at. This is not a double standard. BOINC WIKI |
John McLeod VII Send message Joined: 17 Sep 05 Posts: 108 Credit: 195,137 RAC: 0 |
I'll put my main point again in another way - on BOINC, one hour of CPU time on one project should give around the same credit as that same machine spending an hour on another project. 1 SETI hour ~= 1 Einstein hour ~= 1 Climate Prediction hour ~= 1 Predictor hour ~= 1 Leiden Classical hour ~= 1 Rectilinear Crossing Numbers hour and so on. Ok, reworded by request of the moderator. Cross project credit equality has been part of the design of BOINC since day 1 (before there were any projects on BOINC other than an alpha test of S@H). It is important to the developers of BOINC that this be so. It is important to others that this be so. It has no effect on those that wish to compare a single projects output to itself. BOINC WIKI |
Biggles Send message Joined: 22 Sep 05 Posts: 49 Credit: 102,114 RAC: 0 |
Why is this so darned important? There is NO valid reason to compare results on DC projects under BOINC than it is to say distributed.net needs to have parity with SoB, or D2OL, or DPAD, or F@H, or any other of the myriad of DC projects. Each needs to stand on it's own, and recruit people and CPUs based on the merits of the project and how it's run. It was a feature of BOINC that was always part of its original design. David Anderson was the guy behind BOINC, that makes it a lot more than a myth. Also, remember that the combined RAC is supposed to give an indication of the total throughput of the project in terms of TeraFLOPS. That might not matter to the individual users, but it does matter to the admins when the amount of computing power they have at their disposal affects their success in getting research grants or other resources. Lastly on the topic of cross BOINC crediting, proper parity gives rise to healthy competition. And we all know how powerful a motivator competition can be. I will concede that cross-project parity is of relatively little importance however. I'd certainly like to see it fixed, but the whole credit system needs fixing here first. The thing is, even if we isolate Rosetta from BOINC and don't worry about parity with other projects, the credit system still doesn't work even for just this project. The reason for that is that Athlon 64s on Windows (general statement, mostly true) are getting 3 times the points of any other system for the same amount of actual work. I already explained elsewhere that clock for clock a Barton core Athlon XP will do approximately the same amount of work as a Newcastle/Winchester/Venice core Athlon 64, but that it gets far less points. And that is killing healthy competition. It removes a lot of incentive for many people to crunch for this project if they can't get stats that measure up to the work they are actually doing. There's a helluva lot of P4s out there, and even with optimised clients they get far less credit than an Athlon 64 would, despite doing more work than the stats would indicate. That'll drive people away. The project needs to sort out the credit issue before people leave en mass. |
John McLeod VII Send message Joined: 17 Sep 05 Posts: 108 Credit: 195,137 RAC: 0 |
Please split the cross project equality into another topic, it's a big enough topic to warrant one. I understand that it is demeaning, we have discussed this on other threads in different projects for several months now with the same result. I and others state that it was part of the design, and is important to the developers and others, and he says that he doesn't care that it is part of the design. BOINC WIKI |
Angus Send message Joined: 17 Sep 05 Posts: 412 Credit: 321,053 RAC: 0 |
Please split the cross project equality into another topic, it's a big enough topic to warrant one. I understand fully that it was part of the design. The myth is that it will ever happen. It's impossible, and is not worth the effort being put into it. It needs to be DROPPED from the BOINC agenda so the science projects can proceed with their real work. There has probably been thousands of wasted programmer hours and way too many CPU cycles wasted talking about it. I fully agree that **WITHIN** a project that parity is needed for all operating systems on all platforms. Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :) "You can't fix stupid" (Ron White) |
Ethan Volunteer moderator Send message Joined: 22 Aug 05 Posts: 286 Credit: 9,304,700 RAC: 0 |
Moving some posts here to continue this discussion. |
John McLeod VII Send message Joined: 17 Sep 05 Posts: 108 Credit: 195,137 RAC: 0 |
The thing is, even if we isolate Rosetta from BOINC and don't worry about parity with other projects, the credit system still doesn't work even for just this project. The reason for that is that Athlon 64s on Windows (general statement, mostly true) are getting 3 times the points of any other system for the same amount of actual work. I already explained elsewhere that clock for clock a Barton core Athlon XP will do approximately the same amount of work as a Newcastle/Winchester/Venice core Athlon 64, but that it gets far less points. It will be much less of an issue. The remaining part is that if enough of those hosts are running core clients that inflate credit requests, cross project comparison will not work as well. Yes, I know that cross project comparison is not perfect as it stands, but it should be worked on so that it gets better. BOINC WIKI |
Angus Send message Joined: 17 Sep 05 Posts: 412 Credit: 321,053 RAC: 0 |
Cross project credit equality has been part of the design of BOINC since day 1 (before there were any projects on BOINC other than an alpha test of S@H). It is important to the developers of BOINC that this be so. It is important to others that this be so. It has no effect on those that wish to compare a single projects output to itself. It does have a profound effect on a project in that it has to waste endless time trying to achieve an impossible goal. Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :) "You can't fix stupid" (Ron White) |
John McLeod VII Send message Joined: 17 Sep 05 Posts: 108 Credit: 195,137 RAC: 0 |
Please split the cross project equality into another topic, it's a big enough topic to warrant one. The attempt should be made to keep parity across projects. I admit it will be impossible for it to be perfect, but it should be possible to get fairly close. The place to start would be a clear statement banning the optimized BOINC clients from participating in RALPH. If this is done and enforced, then the new method should come fairly close. BOINC WIKI |
Angus Send message Joined: 17 Sep 05 Posts: 412 Credit: 321,053 RAC: 0 |
Please split the cross project equality into another topic, it's a big enough topic to warrant one. And how do you propose to do that with an open-source core client? It can be made to report itself as any legitimate (non-optimized, although optimizing by itself is perfectly legal) client. Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :) "You can't fix stupid" (Ron White) |
Biggles Send message Joined: 22 Sep 05 Posts: 49 Credit: 102,114 RAC: 0 |
The attempt should be made to keep parity across projects. I admit it will be impossible for it to be perfect, but it should be possible to get fairly close. The place to start would be a clear statement banning the optimized BOINC clients from participating in RALPH. If this is done and enforced, then the new method should come fairly close. I do agree with you, but with BOINC being open source there is nothing to prevent someone from putting subtle changes into the source. A couple of percent here and there isn't really noticable to begin with, but it adds up. Really, in using Ralph to calculate averages for the granted work credit, the benchmark and some form of timer need to be built into Rosetta itself so that it can't be easily modified. Even then it would still be possible to hack the binaries, but it would at least be something that was challenging instead of something that any kid with a bit of programming knowledge could change. |
Angus Send message Joined: 17 Sep 05 Posts: 412 Credit: 321,053 RAC: 0 |
The attempt should be made to keep parity across projects. I admit it will be impossible for it to be perfect, but it should be possible to get fairly close. The place to start would be a clear statement banning the optimized BOINC clients from participating in RALPH. If this is done and enforced, then the new method should come fairly close. Doesn't that effectively remove RALPH as the beta (alpha?) test site for Rosetta? It sounds like you're talking about one type of Ralph client application to determine the credit/decoy for a given WU type, and then using a different application on Rosetta to actually crunch the majority of the work. Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :) "You can't fix stupid" (Ron White) |
John McLeod VII Send message Joined: 17 Sep 05 Posts: 108 Credit: 195,137 RAC: 0 |
The attempt should be made to keep parity across projects. I admit it will be impossible for it to be perfect, but it should be possible to get fairly close. The place to start would be a clear statement banning the optimized BOINC clients from participating in RALPH. If this is done and enforced, then the new method should come fairly close. Build a timer into Rosetta? That sounds a great deal like FLOPS counting. Not a bad idea, but substantially more difficult to program than using fixed credits per task, which works if tasks within a series have the same difficulty. BOINC WIKI |
Biggles Send message Joined: 22 Sep 05 Posts: 49 Credit: 102,114 RAC: 0 |
Doesn't that effectively remove RALPH as the beta (alpha?) test site for Rosetta? It sounds like you're talking about one type of Ralph client application to determine the credit/decoy for a given WU type, and then using a different application on Rosetta to actually crunch the majority of the work. What I mean is the Ralph version of Rosetta should have a timer and a benchmark. Benchmark within the science app itself to give a fair, non-optimised benchmark score. A timer also to prevent the modifying of the reported work unit run time. For instance, if I was running Ralph, and my goal was to skew the averages of work units so that more credit was received for them, I need to use a bigger BOINC benchmark by using an optimised client, or if the benchmark is accurate, by making the work unit appear longer than it actually is. Hence the need for the actual science app - on Ralph if they use it to generate the average credits for a series of units, or Rosetta if they decide to make it more robust and a better average - to have a benchmark and a timer in it. John MacLeod VII wrote: Build a timer into Rosetta? That sounds a great deal like FLOPS counting. Not a bad idea, but substantially more difficult to program than using fixed credits per task, which works if tasks within a series have the same difficulty. I don't see the difficulty in adding a few system calls like GetProcessTimes() or GetThreadTimes(). It doesn't need to be in the main body of the science code itself, rather after it. That way we have a measure of how much CPU time was actually used, as opposed to just what a BOINC client reports. This nullifies one of the ways of possibly cheating. |
John McLeod VII Send message Joined: 17 Sep 05 Posts: 108 Credit: 195,137 RAC: 0 |
OK, this is the wrong thread - it needs to be in the discussion about BOINC credit being compatible across projects. Please move it to the correct thread as I cannot answer a post in a different thread and get the back link. Credits per hour does matter when you are trying to do cross project compatibility, and I believe that the new scheme uses some hosts (RALPH) as a bed to fix the credits per result for each series. In particular, I was interested in disallowing the "optimized" core clients from being used to determine that score. BOINC WIKI |
Message boards :
Number crunching :
Cross Project Credit Equality
©2024 University of Washington
https://www.bakerlab.org