Another solution for the credit issue that hasn't been mentioned.

Message boards : Number crunching : Another solution for the credit issue that hasn't been mentioned.

To post messages, you must log in.

1 · 2 · 3 · 4 . . . 7 · Next

AuthorMessage
Tallbill

Send message
Joined: 23 Jul 06
Posts: 12
Credit: 101,854
RAC: 0
Message 23755 - Posted: 20 Aug 2006, 16:25:49 UTC
Last modified: 20 Aug 2006, 16:36:01 UTC

I dont have time to read every message everywhere, but I havn't seen this recommended yet. Why not go with the system of measuring credit done based on cpu and performance as planned, but instead of lowering scores, set the benchmarks up to be equal to what the optimized clients do now.

As far as I know, it was never against the rules to be using an optimized client, so instead of punishing the people who have, instead use those levels to set the standard of what a WU is worth, and take the data back to february to RAISE everyone up to an equal level, instead of lowering everyone to an equal level.

The only problem left here is that credits will be worth more then other boinc projects, but this is an independant project that can measure its work however it wants. Statistic sites will just have to skew results as they see fit.

I hope this makes sense, and really wouldn't piss people off or have issues with a second data column or separate scoring.


Edit - I've read that credits wont be backdated whatsoever, but this idea could still apply towards future credits. Multiproject parity isn't as important as equality within a project.
ID: 23755 · Rating: 3 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jose

Send message
Joined: 28 Mar 06
Posts: 820
Credit: 48,297
RAC: 0
Message 23767 - Posted: 20 Aug 2006, 16:50:51 UTC - in response to Message 23755.  

I dont have time to read every message everywhere, but I havn't seen this recommended yet. Why not go with the system of measuring credit done based on cpu and performance as planned, but instead of lowering scores, set the benchmarks up to be equal to what the optimized clients do now.

As far as I know, it was never against the rules to be using an optimized client, so instead of punishing the people who have, instead use those levels to set the standard of what a WU is worth, and take the data back to february to RAISE everyone up to an equal level, instead of lowering everyone to an equal level.

The only problem left here is that credits will be worth more then other boinc projects, but this is an independant project that can measure its work however it wants. Statistic sites will just have to skew results as they see fit.

I hope this makes sense, and really wouldn't piss people off or have issues with a second data column or separate scoring.


Edit - I've read that credits wont be backdated whatsoever, but this idea could still apply towards future credits. Multiproject parity isn't as important as equality within a project.



At last a rational person.

:)
ID: 23767 · Rating: -9.9920072216264E-15 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Saenger
Avatar

Send message
Joined: 19 Sep 05
Posts: 271
Credit: 824,883
RAC: 0
Message 23768 - Posted: 20 Aug 2006, 16:57:36 UTC - in response to Message 23767.  

At last a rational person.

:)

But that's just what I said several times, suddenly, with a different sender, you agree. Strange.
ID: 23768 · Rating: 2 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Vester
Avatar

Send message
Joined: 2 Nov 05
Posts: 258
Credit: 3,651,260
RAC: 1,152
Message 23769 - Posted: 20 Aug 2006, 17:01:46 UTC

With four optimized computers, I found that a 300% benchmark improvement due to optimizaton gave only a 60% RAC increase. Reducing or increasing by three-fold won't be fair to anyone, so adjustments need to be carefully considered.
ID: 23769 · Rating: -1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B^S] thierry@home
Avatar

Send message
Joined: 17 Sep 05
Posts: 182
Credit: 281,902
RAC: 0
Message 23770 - Posted: 20 Aug 2006, 17:03:29 UTC - in response to Message 23755.  

I dont have time to read every message everywhere, but I havn't seen this recommended yet. Why not go with the system of measuring credit done based on cpu and performance as planned, but instead of lowering scores, set the benchmarks up to be equal to what the optimized clients do now.

As far as I know, it was never against the rules to be using an optimized client, so instead of punishing the people who have, instead use those levels to set the standard of what a WU is worth, and take the data back to february to RAISE everyone up to an equal level, instead of lowering everyone to an equal level.

The only problem left here is that credits will be worth more then other boinc projects, but this is an independant project that can measure its work however it wants. Statistic sites will just have to skew results as they see fit.

I hope this makes sense, and really wouldn't piss people off or have issues with a second data column or separate scoring.


Edit - I've read that credits wont be backdated whatsoever, but this idea could still apply towards future credits. Multiproject parity isn't as important as equality within a project.


I don't see realy what is the difference by leveling things to the top instead of the bottom .....
ID: 23770 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jose

Send message
Joined: 28 Mar 06
Posts: 820
Credit: 48,297
RAC: 0
Message 23772 - Posted: 20 Aug 2006, 17:07:07 UTC - in response to Message 23768.  

At last a rational person.

:)

But that's just what I said several times, suddenly, with a different sender, you agree. Strange.


Saenger: you included the "backtracking" word.. He is not. Also he is not interested in crossproject equality.

This is where your position and his differ:

Edit - I've read that credits wont be backdated whatsoever, but this idea could still apply towards future credits. Multiproject parity isn't as important as equality within a project.


There is a huge difference in what he is recomending and what you want.

ID: 23772 · Rating: -2 · rate: Rate + / Rate - Report as offensive    Reply Quote
Ethan
Volunteer moderator

Send message
Joined: 22 Aug 05
Posts: 286
Credit: 9,304,700
RAC: 0
Message 23773 - Posted: 20 Aug 2006, 17:07:40 UTC

The backdating idea was to apply the new credit system to past results. The idea of calculating credits based on what the optimized clients reported, then applying those values to standard clients, hasn't been talked about (that I've seen).

It's a slipperly slope, but I think comments on this idea would be useful.
ID: 23773 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B^S] thierry@home
Avatar

Send message
Joined: 17 Sep 05
Posts: 182
Credit: 281,902
RAC: 0
Message 23774 - Posted: 20 Aug 2006, 17:08:41 UTC
Last modified: 20 Aug 2006, 17:12:06 UTC

Jose, explain please.
ID: 23774 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jose

Send message
Joined: 28 Mar 06
Posts: 820
Credit: 48,297
RAC: 0
Message 23775 - Posted: 20 Aug 2006, 17:09:23 UTC - in response to Message 23769.  

With four optimized computers, I found that a 300% benchmark improvement due to optimizaton gave only a 60% RAC increase. Reducing or increasing by three-fold won't be fair to anyone, so adjustments need to be carefully considered.


The issue becomes developing a good, solid, and fair to all set of correction factors.. It can be done.
ID: 23775 · Rating: -1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile carl.h
Avatar

Send message
Joined: 28 Dec 05
Posts: 555
Credit: 183,449
RAC: 0
Message 23776 - Posted: 20 Aug 2006, 17:12:41 UTC
Last modified: 20 Aug 2006, 17:14:40 UTC

I did WU 3565878 so did Thierry I got 90 credits using 5.5 he got 60....give Thierry 30 more....

That type of thing Thierry back to February.

The problem here is X Boincers will not agree or shouldn`t !
Not all Czech`s bounce but I`d like to try with Barbar ;-)

Make no mistake This IS the TEDDIES TEAM.
ID: 23776 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B^S] thierry@home
Avatar

Send message
Joined: 17 Sep 05
Posts: 182
Credit: 281,902
RAC: 0
Message 23777 - Posted: 20 Aug 2006, 17:18:38 UTC - in response to Message 23776.  

I did WU 3565878 so did Thierry I got 90 credits using 5.5 he got 60....give Thierry 30 more....

That type of thing Thierry back to February.

The problem here is X Boincers will not agree or shouldn`t !


OK, but getting 30 more or remove you 30 is exactly the same, isn't it?
ID: 23777 · Rating: 2 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jose

Send message
Joined: 28 Mar 06
Posts: 820
Credit: 48,297
RAC: 0
Message 23778 - Posted: 20 Aug 2006, 17:20:05 UTC - in response to Message 23774.  

Jose, explain please.


Why dont we use this that was in one of the threads that was deleted and was recieving some fair comments as a starting point

... may I suggest the following procedure:

For every old protein
For every procedure used on this protein (RelaxAll, IgnoreAll, whatever ...)
For every WU returned
calculate the Credit claimed Per Decoy generated (CPD)
Next WU returned

sort CPD in a descending order
ignore the top 3% values
Take the highest CPD value after that as being fixed (CPDf)

For every WU returned
Calculate corrected credit = decoys * CPDf
Adjust the awarded credit in the data base to be the new calculated credit.
Next WU returned
Next procedure
Next Protein

Please note:

The reason to ignore the top 3% values (and no more, or will affect employee A) is simple. This clears out (takes care of) all the:
potential rare PC flukes
a unit run that was too lucky
user "edited" the actual CPU time or benchmark
etc..
By applying the above procedure on each protein/procedure pair independently, we remove the effect of the protein properties (length, etc..) and procedure complexity from the equation.

The end result of this is the equivalent of a quorum of thousands of WUs of the same protein/procedure. No participant will lose credit, and the few questionable cases will be automatically adjusted in line downwards (thus saving the huge ongoing task of identification/verification/clarification). The total credit will end up being consistent with the actual work done (the widgets) while the unit price (dollar per widget) is fixed. Equality, and hopefully peace restored.



ID: 23778 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jose

Send message
Joined: 28 Mar 06
Posts: 820
Credit: 48,297
RAC: 0
Message 23779 - Posted: 20 Aug 2006, 17:20:50 UTC - in response to Message 23776.  

I did WU 3565878 so did Thierry I got 90 credits using 5.5 he got 60....give Thierry 30 more....

That type of thing Thierry back to February.

The problem here is X Boincers will not agree or shouldn`t !


It is not that simple Carl. :)
ID: 23779 · Rating: -1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Saenger
Avatar

Send message
Joined: 19 Sep 05
Posts: 271
Credit: 824,883
RAC: 0
Message 23780 - Posted: 20 Aug 2006, 17:21:58 UTC - in response to Message 23772.  

Saenger: you included the "backtracking" word.. He is not. Also he is not interested in crossproject equality.

So did he in most of his post, only in the edit he said: "OK, if not, then at least for the future."
That's what I said several times. I can live without real stats for the past, it would only be nicer to have them. And the values are a decision, the project has to take, but that will have an effect on the participants.
If it doesn't fit in the BOINC scheme, I think quite a lot users will be lost. I don't know how hard it would be for Willy, Zain, Neil e.a. to put a correction factor in place for their stats pages if the values don't fit, or if they simply have to ditch Rosetta from the stats because of incompability.
ID: 23780 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B^S] thierry@home
Avatar

Send message
Joined: 17 Sep 05
Posts: 182
Credit: 281,902
RAC: 0
Message 23781 - Posted: 20 Aug 2006, 17:22:34 UTC - in response to Message 23778.  

Jose, explain please.


Why dont we use this that was in one of the threads that was deleted and was recieving some fair comments as a starting point

... may I suggest the following procedure:

For every old protein
For every procedure used on this protein (RelaxAll, IgnoreAll, whatever ...)
For every WU returned
calculate the Credit claimed Per Decoy generated (CPD)
Next WU returned

sort CPD in a descending order
ignore the top 3% values
Take the highest CPD value after that as being fixed (CPDf)

For every WU returned
Calculate corrected credit = decoys * CPDf
Adjust the awarded credit in the data base to be the new calculated credit.
Next WU returned
Next procedure
Next Protein

Please note:

The reason to ignore the top 3% values (and no more, or will affect employee A) is simple. This clears out (takes care of) all the:
potential rare PC flukes
a unit run that was too lucky
user "edited" the actual CPU time or benchmark
etc..
By applying the above procedure on each protein/procedure pair independently, we remove the effect of the protein properties (length, etc..) and procedure complexity from the equation.

The end result of this is the equivalent of a quorum of thousands of WUs of the same protein/procedure. No participant will lose credit, and the few questionable cases will be automatically adjusted in line downwards (thus saving the huge ongoing task of identification/verification/clarification). The total credit will end up being consistent with the actual work done (the widgets) while the unit price (dollar per widget) is fixed. Equality, and hopefully peace restored.





Jose,
I asked you about the Tallbill proposal. You never answer to a direct question, do you.
ID: 23781 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jose

Send message
Joined: 28 Mar 06
Posts: 820
Credit: 48,297
RAC: 0
Message 23782 - Posted: 20 Aug 2006, 17:22:59 UTC - in response to Message 23777.  

I did WU 3565878 so did Thierry I got 90 credits using 5.5 he got 60....give Thierry 30 more....

That type of thing Thierry back to February.

The problem here is X Boincers will not agree or shouldn`t !


OK, but getting 30 more or remove you 30 is exactly the same, isn't it?



Thierry look at what I am proposing.
Focus on that.
ID: 23782 · Rating: -2 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B^S] thierry@home
Avatar

Send message
Joined: 17 Sep 05
Posts: 182
Credit: 281,902
RAC: 0
Message 23783 - Posted: 20 Aug 2006, 17:25:48 UTC - in response to Message 23782.  

I did WU 3565878 so did Thierry I got 90 credits using 5.5 he got 60....give Thierry 30 more....

That type of thing Thierry back to February.

The problem here is X Boincers will not agree or shouldn`t !


OK, but getting 30 more or remove you 30 is exactly the same, isn't it?



Thierry look at what I am proposing.
Focus on that.


You always answer by something else, I give up for today (if I can).
ID: 23783 · Rating: 0.99999999999999 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jose

Send message
Joined: 28 Mar 06
Posts: 820
Credit: 48,297
RAC: 0
Message 23784 - Posted: 20 Aug 2006, 17:27:05 UTC - in response to Message 23781.  

Jose, explain please.


Why dont we use this that was in one of the threads that was deleted and was recieving some fair comments as a starting point

... may I suggest the following procedure:

For every old protein
For every procedure used on this protein (RelaxAll, IgnoreAll, whatever ...)
For every WU returned
calculate the Credit claimed Per Decoy generated (CPD)
Next WU returned

sort CPD in a descending order
ignore the top 3% values
Take the highest CPD value after that as being fixed (CPDf)

For every WU returned
Calculate corrected credit = decoys * CPDf
Adjust the awarded credit in the data base to be the new calculated credit.
Next WU returned
Next procedure
Next Protein

Please note:

The reason to ignore the top 3% values (and no more, or will affect employee A) is simple. This clears out (takes care of) all the:
potential rare PC flukes
a unit run that was too lucky
user "edited" the actual CPU time or benchmark
etc..
By applying the above procedure on each protein/procedure pair independently, we remove the effect of the protein properties (length, etc..) and procedure complexity from the equation.

The end result of this is the equivalent of a quorum of thousands of WUs of the same protein/procedure. No participant will lose credit, and the few questionable cases will be automatically adjusted in line downwards (thus saving the huge ongoing task of identification/verification/clarification). The total credit will end up being consistent with the actual work done (the widgets) while the unit price (dollar per widget) is fixed. Equality, and hopefully peace restored.





Jose,
I asked you about the Tallbill proposal. You never answer to a direct question, do you.


I am answering why I think the proposal can be workable. But if you and your friends only came to this thread to continue banging your heads and pick up a fight with me . Then , it takes two to fight.

I wil ignore you as a person who only wants to fight and not find a solution to the issues. But , warning, I will react to your attempts to flame here with the same passion that I have.


The time has come to look for solutions. Some reasonable, working and fair compromises have to be reached .

If you want to stay in your purist high horse ..you are welcome but then thon comkplaim where the only reaction you get is scorn.
ID: 23784 · Rating: -4 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Saenger
Avatar

Send message
Joined: 19 Sep 05
Posts: 271
Credit: 824,883
RAC: 0
Message 23785 - Posted: 20 Aug 2006, 17:28:49 UTC - in response to Message 23782.  

Thierry look at what I am proposing.
Focus on that.

It's about retroactive levelling of the playing field.
That's fine, and that's just what I always wanted.

The values are at first (at least from a project-intern point of view) irrelevant. You can norm the credits per decoy at whatever you want, as long as it's consistent. I don't see any difference with norming them to the bottom, norming them to the top, or even norming them to twice the top value in project-intern effects.

The only difference the value makes is the compability to BOINC.
ID: 23785 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile carl.h
Avatar

Send message
Joined: 28 Dec 05
Posts: 555
Credit: 183,449
RAC: 0
Message 23786 - Posted: 20 Aug 2006, 17:30:59 UTC
Last modified: 20 Aug 2006, 17:31:16 UTC

Hold up let me get a picture !

Saenger and Jose agree on a possible way to backdate the credit system !;-)
Not all Czech`s bounce but I`d like to try with Barbar ;-)

Make no mistake This IS the TEDDIES TEAM.
ID: 23786 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
1 · 2 · 3 · 4 . . . 7 · Next

Message boards : Number crunching : Another solution for the credit issue that hasn't been mentioned.



©2024 University of Washington
https://www.bakerlab.org