Just a heads up for you folks.

Message boards : Number crunching : Just a heads up for you folks.

To post messages, you must log in.

Previous · 1 · 2 · 3

AuthorMessage
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1831
Credit: 119,653,907
RAC: 11,163
Message 17241 - Posted: 27 May 2006, 12:48:20 UTC - in response to Message 17229.  

A question for members of the teams vying for dominance--Rom is currently working on an a benchmark internal to the rosetta code that will ultimately used in calculating credits. would you like to see this on rosetta@home sooner or later?


This will solve many problems and level the playing field and will be welcomed by most participants. The only problem is how many credits are you going to reward? The current optimized BOINC clients claim about 3 times the credit the regular BOINC client does. All competing teams use only the optimized client.If you adjust the new credit system according to the regular BOINC client all users who run the optimized client will see there average credit dropping to one third of past numbers. That causes a lot of frustration. Seti changed recently to an internal benchmark and there is much frustration in the boards from the credit hunters since it caused their RAC to drop.

Climateprediction issues credit per WU but they reward more credit per WU than a regular BOINC client would claim. I think you need to do the same for Rosetta as well.


I agree absoultely - and the sooner the better. There's a psychological advantage to bringing the standard client's claims up, rather than bringing the 'optimised' client's credits down.

Regarding the cross-project issue of people running the project that gives them the most credit, from a project-selfish point of view it'll be beneficial for Rosetta to give more points I guess. I think this needs to be standardised between the BOINC projects quite quickly though, and the flops counting seems a good way to go on that, but as BOINC is open source I guess it will always be open to exploit... I suppose ideally the science app should contain the benchmark routine, and if the science app is compiled to use the extensions such as SSE then the benchmark should too.

I'm sure there will be some discontent which ever way this goes, but there's discontent as it stands anyway! At least this would give a level playing field from now on ;)
ID: 17241 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
doc :)

Send message
Joined: 4 Oct 05
Posts: 47
Credit: 1,106,102
RAC: 0
Message 17256 - Posted: 27 May 2006, 21:53:02 UTC

bring the level playing field, yes!
bringing the ppl using optimized clients back to normal levels will ofcourse bring a lot of complaints, but thats in my oppinion the only fair way to go, you cant give X times more credit than seti or whatever other project gives.

(i dont want to know how much of the estimated TFLOPS on the frontpage is from wrong benchmarks ;))
ID: 17256 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
XS_STEvil

Send message
Joined: 30 Dec 05
Posts: 9
Credit: 189,013
RAC: 0
Message 17264 - Posted: 28 May 2006, 4:39:01 UTC - in response to Message 17217.  

I am all for a new way to index machine performance as long as it does it right.

This most likely means that you get 1 credit for every work unit completed and the only factor that may ultimately affect this will be WU length (ie: 2 hours vs 24 hours).. the same way seti@home operated before BOINC.

People like me on dial-up would feel the pain.. a long and slow burning pain ;)


But what about other issues? Like being "over comitted"? Why dont we assign work units as needed rather than overfilling people so that when they run out of time they dont get credit for completing assigned WU's?

The user is in control of how many work units are loaded on the client machine. If you use your time parameter, and connection interval correctly, you could work the same work unit for 24 hours. Or you could run 24 work units in 24 hours. This assumes that the protein is small enough to create a model in under an hour.

People get over committed by downloading work and then raising the time setting a significant amount. To avoid that, raise the time limit when your queue is low on work units. Then the system will load less work, and will not over commit based on the work it already has.

This time setting was put in specifically for modem users. There is a FAQ explaining its use. As for credit, it is my understanding that all work will get credit. Rosetta is not like other projects in that regard. There is a FAQ for that too.


But due to the size of the work units (1mb-3mb+) you cannot spend all day downloading and uploading work units on dial-up, especially when you have multiple machines so it is a necessity to download a lot since you can not always have set upload/download times.

For example this laptop was running 48hr WU's to begin with, but that option is no longer selectable so it can only run 24's now (which doubles the required bandwidth). Even with the laptop set to only get enough WU's for 5 days work I still sometimes run out of work, and even more so my brother's desktop does because of the "communication deferred xx.xx.xx" feature (not much of a feature, it just accumulates over time if no connection is present which eventually makes it attempt to connect once a week or so...).

In reguard to the credits, I was pointing out that if the system went to 1 credit for 1 work unit (the most fair and easiest way to do this that I can think of at the moment) then it wouldnt matter if you did a 12 hour WU in 6 hours or a 24 hour one in 12 hours, you would recieve one credit.... but then that isnt fair to people like me who are on modems and selectively take the 24hr WU's because of the dial-up connection.

Reguarding credits for overdue WU's, I remember reading somewhere that users did not recieve credit for them. Apparently this has changed so its no longer a point to be made.. thanks for clearing that up ;)
This signature was annoying.
ID: 17264 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Robert Everly

Send message
Joined: 8 Oct 05
Posts: 27
Credit: 665,094
RAC: 0
Message 17279 - Posted: 28 May 2006, 17:38:10 UTC - in response to Message 17110.  

A question for members of the teams vying for dominance--Rom is currently working on an a benchmark internal to the rosetta code that will ultimately used in calculating credits. would you like to see this on rosetta@home sooner or later?


Sooner. Absolutely without question. Just make it fair.

ID: 17279 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Carlos_Pfitzner
Avatar

Send message
Joined: 22 Dec 05
Posts: 71
Credit: 138,867
RAC: 0
Message 17284 - Posted: 29 May 2006, 0:35:04 UTC - in response to Message 17135.  
Last modified: 29 May 2006, 1:15:05 UTC

33 TFLOPS go baby go!!



I have 20 TFLOPS on one computer !

Read here technical specifications

U can buy a dozen of them, and go to 240 TFLOPS , easily !
Cray Brazil
Cray Computadores do Brasil Ltda
Rua Dom Pedro II, 1240 - cj 405
Porto Alegre, RS, CEP 90550-141
Brasil
Phone: 55-51-3337-8960
Fax: 55-51-3337-8721

ps: CEP = postal code

BTW: Im starting to make a low cost, 2 TFLOPS desktop, for sale.
*Verify u can pay the electric bill before buying my desktop(s) -> (1KVA)
Click signature for global team stats
ID: 17284 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Delk

Send message
Joined: 20 Feb 06
Posts: 25
Credit: 995,624
RAC: 0
Message 17285 - Posted: 29 May 2006, 0:37:43 UTC - in response to Message 17279.  

A question for members of the teams vying for dominance--Rom is currently working on an a benchmark internal to the rosetta code that will ultimately used in calculating credits. would you like to see this on rosetta@home sooner or later?


Sooner. Absolutely without question. Just make it fair.


Please make the benchmark balanced between windows & linux.
ID: 17285 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Carlos_Pfitzner
Avatar

Send message
Joined: 22 Dec 05
Posts: 71
Credit: 138,867
RAC: 0
Message 17286 - Posted: 29 May 2006, 1:01:23 UTC - in response to Message 17285.  
Last modified: 29 May 2006, 1:22:17 UTC

A question for members of the teams vying for dominance--Rom is currently working on an a benchmark internal to the rosetta code that will ultimately used in calculating credits. would you like to see this on rosetta@home sooner or later?


Sooner. Absolutely without question. Just make it fair.


Please make the benchmark balanced between windows & linux.


*May be implement a quorum of 2 instead.

Has two advantages
1) A diff can be run to verify that both results are equal !
Greater for science Integrity
**(some high overclocked computers may produce wrong results)

2) Grant the average claimed. granted = (high_claim + low_claim)/2
*Prime grid actually grants credit this way

May be then, askof produce a optimzed bynaries to speed up crunching
*On Einsten
their optimized binary reduced the crunching time
*from ( 6 hours / WU ) to ( 1 hour / WU )

May be he can do the same here ?

Click signature for global team stats
ID: 17286 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Ethan
Volunteer moderator

Send message
Joined: 22 Aug 05
Posts: 286
Credit: 9,304,700
RAC: 0
Message 17290 - Posted: 29 May 2006, 4:30:13 UTC - in response to Message 17286.  

Thanks for the suggestion, Carlos.

The idea of a quorum has been mentioned before. For this project, the scientific results are not compromised by the lack of duplicate results. This means that if a quorum of 2 was implemented, the computing power of R@H would be cut in half without any scientific benefit. Hopefully everyone agrees this would be a bad thing.

I'm aware of the benefit of your 2nd point for those who track credits. . I believe there are other threads which talk about how 'liberal' computers are being handled.

Cheers,
Ethan
ID: 17290 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Moderator9
Volunteer moderator

Send message
Joined: 22 Jan 06
Posts: 1014
Credit: 0
RAC: 0
Message 17308 - Posted: 29 May 2006, 15:12:57 UTC - in response to Message 17286.  

...
1) A diff can be run to verify that both results are equal !
Greater for science Integrity
**(some high overclocked computers may produce wrong results)

In most cases no two Rosetta results are equal. Redundancy would serve no science purpose of any kind, and in fact would simply slow the progress of the science.
Moderator9
ROSETTA@home FAQ
Moderator Contact
ID: 17308 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile anders n

Send message
Joined: 19 Sep 05
Posts: 403
Credit: 537,991
RAC: 0
Message 17319 - Posted: 29 May 2006, 19:14:07 UTC - in response to Message 17185.  

Did anyone bother to tell FreeDC that they were in a competition? It does not look like they have ramped up any yet. The team RAC has not gone up very much. It looks like they will be overtaken in a few weeks if they don't move to defend.


It looks like days right now.

Anders n

ID: 17319 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jose

Send message
Joined: 28 Mar 06
Posts: 820
Credit: 48,297
RAC: 0
Message 17324 - Posted: 29 May 2006, 22:43:53 UTC - in response to Message 17319.  

Did anyone bother to tell FreeDC that they were in a competition? It does not look like they have ramped up any yet. The team RAC has not gone up very much. It looks like they will be overtaken in a few weeks if they don't move to defend.


It looks like days right now.

Anders n



They know
This and no other is the root from which a Tyrant springs; when he first appears he is a protector.”
Plato
ID: 17324 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Bok

Send message
Joined: 17 Sep 05
Posts: 54
Credit: 3,514,973
RAC: 0
Message 17442 - Posted: 31 May 2006, 17:00:49 UTC

Of course we know, there is a limit to how much one team can produce though. With XtremeSystems having way more members, it's difficult to match their numbers. Maybe if we managed to get everyone running Rosetta we could, but that's not how we do things...

As for the whole benchmark and optimized clients thing. Well what started as a way to level the playing field between windows and linux has got out of hand somewhat. I'm all for having Rosetta implement there own benchmark to level the playing field, though in my mind the playing field is already level. The clients are readily available to all those who wish to use them.

That being said, I see no way you could benchmark a xeon at the levels some of those anonymous clients were showing. Fastest I've seen yet, that I believe, is the Conroe at 3.1 which was giving 7K/26K. I have a few myself in the 5K/15K range (all dual core opterons). My DX 3.6 (800FSB) is pretty close to that with the appropriate client.

Bok
Free-DC

Stats for all projects

Custom Stats
ID: 17442 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jose

Send message
Joined: 28 Mar 06
Posts: 820
Credit: 48,297
RAC: 0
Message 17444 - Posted: 31 May 2006, 17:42:38 UTC - in response to Message 17442.  

Of course we know, there is a limit to how much one team can produce though. With XtremeSystems having way more members, it's difficult to match their numbers. Maybe if we managed to get everyone running Rosetta we could, but that's not how we do things...

As for the whole benchmark and optimized clients thing. Well what started as a way to level the playing field between windows and linux has got out of hand somewhat. I'm all for having Rosetta implement there own benchmark to level the playing field, though in my mind the playing field is already level. The clients are readily available to all those who wish to use them.

That being said, I see no way you could benchmark a xeon at the levels some of those anonymous clients were showing. Fastest I've seen yet, that I believe, is the Conroe at 3.1 which was giving 7K/26K. I have a few myself in the 5K/15K range (all dual core opterons). My DX 3.6 (800FSB) is pretty close to that with the appropriate client.

Bok



First of all, let me mention the fact that you are one of the most respected legends of the DC Community. Your work ethic and your friendliness to all (teammate or not) is know and appreciated.

Ty for your comments re the credit issue and the benchmarks in question.
This and no other is the root from which a Tyrant springs; when he first appears he is a protector.”
Plato
ID: 17444 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2 · 3

Message boards : Number crunching : Just a heads up for you folks.



©2024 University of Washington
https://www.bakerlab.org