BIG credit problem! User has a zillion credits!

Message boards : Number crunching : BIG credit problem! User has a zillion credits!

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
[BOINCstats] Willy

Send message
Joined: 24 Sep 05
Posts: 11
Credit: 3,761,636
RAC: 0
Message 7600 - Posted: 25 Dec 2005, 13:23:26 UTC
Last modified: 25 Dec 2005, 13:24:43 UTC

Take a look at the top participants (https://boinc.bakerlab.org/rosetta/top_users.php) and check out number 1. This can't be right. It seems that granting credit is really screwed.

Can this be fixed a.s.a.p. (I suggest deleting the user) because this is messing up both Rosetta and BOINC combined stats.
ID: 7600 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Scribe
Avatar

Send message
Joined: 2 Nov 05
Posts: 284
Credit: 157,359
RAC: 0
Message 7602 - Posted: 25 Dec 2005, 13:46:09 UTC

He could be overclocking! ;-))
ID: 7602 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
LP
Avatar

Send message
Joined: 4 Nov 05
Posts: 16
Credit: 177,147
RAC: 0
Message 7603 - Posted: 25 Dec 2005, 13:49:06 UTC

Or he could have "optimized". ;)
hahaha
ID: 7603 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
[BOINCstats] Willy

Send message
Joined: 24 Sep 05
Posts: 11
Credit: 3,761,636
RAC: 0
Message 7604 - Posted: 25 Dec 2005, 13:55:12 UTC

I have changed some update code, and this specific user gets 0 (zero) credits when imported into the BOINCstats DB until his credit is fixed.
ID: 7604 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile River~~
Avatar

Send message
Joined: 15 Dec 05
Posts: 761
Credit: 285,578
RAC: 0
Message 7607 - Posted: 25 Dec 2005, 14:48:44 UTC - in response to Message 7600.  
Last modified: 25 Dec 2005, 14:52:19 UTC

Take a look at the top participants (https://boinc.bakerlab.org/rosetta/top_users.php) and check out number 1. This can't be right. It seems that granting credit is really screwed.

Can this be fixed a.s.a.p. (I suggest deleting the user) because this is messing up both Rosetta and BOINC combined stats.


There is a trivially easy exploit that can be done on a project like Rosetta where there is no redundancy that can be used to check credit claims.

No, I am not going to publish it here. If any of the BOINC or Rosetta admins email me I can explain further but if you are in contact with any ex-Pirates they may well remember it being discovered on that project.

Whether it is an exploit or a bug, a half-fix is to impose a max credit for WU based on the max runtime, this would ideally be a change to the BOINC server code.

Users who exceed this limit should get logged so that in extreme cases admins can easily do a manual check for bug or malice.

In the absence of a manual check the user should get zero for that run. After a manual check fair credit can be added manually if it was a bug. If it's fraud all past credit for that user should be cancelled in my opinion.

In fairness this does not look like deliberate fraud - with some of the funny WU that have been released here recently it could be a machine induced glitch. The admins will be able to tell quickly if it is just one WU, or several WU from one user, or several wu from several users.

edit:
I would have thought that a deliberate fraud would have been more subtle, like at the very least asking for a numerical amount of credit!

The exploit does need working round, however, unless this has already been done.
ID: 7607 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile River~~
Avatar

Send message
Joined: 15 Dec 05
Posts: 761
Credit: 285,578
RAC: 0
Message 7608 - Posted: 25 Dec 2005, 14:57:40 UTC - in response to Message 7604.  
Last modified: 25 Dec 2005, 15:18:16 UTC

I have changed some update code, and this specific user gets 0 (zero) credits when imported into the BOINCstats DB until his credit is fixed.


A better long term thing would be to install some sanity checking to give zero credit for any credit that is misformatted or exceeds some (generous) max plausible figure. That way future problems of a similar nature will not make you have to hack your code on Christmas day!

Maybe you'd still want to log such things so that you can give similar alerts to the relevant project team(s)

River~~

PS- as a BOINCstats user, thanks for the service in general and for providing Christmas maintenance cover :-)
ID: 7608 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
[BOINCstats] Willy

Send message
Joined: 24 Sep 05
Posts: 11
Credit: 3,761,636
RAC: 0
Message 7610 - Posted: 25 Dec 2005, 15:14:46 UTC - in response to Message 7608.  
Last modified: 25 Dec 2005, 15:15:20 UTC

A better long term thing would be to install some sanity checking to give zero credit for any credit that is misformatted or exceeds some (generous) max plausible figure. That way future problems of a similar nature will not make you have to hack your code on Christmas day!


But what is a plausible figure? This would be different for every user. Of course, this one went a bit over the top. Just a bit.

PS- as a BOINCstats user, thanks for the service in general and for providing Christmas maintenance


They call me 'no-life-Willy'. Well, sometimes they do. Mostly at work.

ID: 7610 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile River~~
Avatar

Send message
Joined: 15 Dec 05
Posts: 761
Credit: 285,578
RAC: 0
Message 7612 - Posted: 25 Dec 2005, 15:36:29 UTC - in response to Message 7610.  


But what is a plausible figure?


myself, i'd select the total credits for each project, then use the highest of those figures as a cut off for the next days credits for any single user on any single project. That makes a safely large and obviously implausible cut-off point.

There are other ways to generate a figure from yesterday's data - any such data-generated limit would be safe for any normal future development because it will scale up as Moore's law pushes the credits ever higher.

It won't catch all errors, no sanity check can catch a subtle cheat, but a value in the zillions should not get through, in my opinion. Some errors that get through would still distort the stats, but not in such an extreme way as this one did.

R~~



ID: 7612 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Paul D. Buck

Send message
Joined: 17 Sep 05
Posts: 815
Credit: 1,812,737
RAC: 0
Message 7620 - Posted: 25 Dec 2005, 17:17:39 UTC
Last modified: 25 Dec 2005, 17:18:18 UTC

But what is a plausible figure?

Average CS per second of recent work returned * CPU seconds * 3

THat gives a generous margin of error, trimming 3 to 2.25 or what ever if you want a tighter test than 50%.

==== edit

Or just bite the bullet and FLOP count ... :)


ID: 7620 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
alo_dk

Send message
Joined: 11 Dec 05
Posts: 19
Credit: 30,425
RAC: 0
Message 7621 - Posted: 25 Dec 2005, 17:31:51 UTC
Last modified: 25 Dec 2005, 17:48:22 UTC

ID: 7621 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Tern
Avatar

Send message
Joined: 25 Oct 05
Posts: 576
Credit: 4,695,362
RAC: 9
Message 7622 - Posted: 25 Dec 2005, 17:43:41 UTC - in response to Message 7621.  


ID: 7622 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile River~~
Avatar

Send message
Joined: 15 Dec 05
Posts: 761
Credit: 285,578
RAC: 0
Message 7627 - Posted: 25 Dec 2005, 20:22:11 UTC - in response to Message 7622.  

Any "check", no matter how good, other than flops-counting or redundancy, is just a patch.


more like a fuse than a patch I'd say

Certainly a check is not a solution; but it should be installed before a solution comes along, and should be kept after a solution comes along as a safety device.

And you are right, we need a solution as well as the fuse.

River~~
ID: 7627 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile stephan_t
Avatar

Send message
Joined: 20 Oct 05
Posts: 129
Credit: 35,464
RAC: 0
Message 7811 - Posted: 28 Dec 2005, 12:09:31 UTC - in response to Message 7600.  

Can this be fixed a.s.a.p. (I suggest deleting the user) because this is messing up both Rosetta and BOINC combined stats.


LOL get ready to be flamed! Some people here really take the mickey if you start claiming this or that person is cheating. See the 'cheating' thread I started a while back.

Personally I quit the project 2 weeks ago and will only come back once the stats are fair. It's been proven the stats can be altered even without modifying the client at all. If people are cheating 'intelligently' (ie, not claiming a billion credit but 20 or 30% more instead), then it's undetectable.

Edit: I was about to say 'next thing you know they'll tell you it's just a glitch because of a bad WU' - and they did. :-D
Team CFVault.com
http://www.cfvault.com

ID: 7811 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Nothing But Idle Time

Send message
Joined: 28 Sep 05
Posts: 209
Credit: 139,545
RAC: 0
Message 7829 - Posted: 28 Dec 2005, 16:53:46 UTC - in response to Message 7811.  

Can this be fixed a.s.a.p. (I suggest deleting the user) because this is messing up both Rosetta and BOINC combined stats.


LOL get ready to be flamed! Some people here really take the mickey if you start claiming this or that person is cheating. See the 'cheating' thread I started a while back.

Personally I quit the project 2 weeks ago and will only come back once the stats are fair. It's been proven the stats can be altered even without modifying the client at all. If people are cheating 'intelligently' (ie, not claiming a billion credit but 20 or 30% more instead), then it's undetectable.

Edit: I was about to say 'next thing you know they'll tell you it's just a glitch because of a bad WU' - and they did. :-D


<aside>
Wondered where you've been; haven't seen you around the boards for awhile. Miss your contributions here in the discussions. Come back soon. If not 100% Rosetta what are you doing?
</aside>
ID: 7829 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Tern
Avatar

Send message
Joined: 25 Oct 05
Posts: 576
Credit: 4,695,362
RAC: 9
Message 7834 - Posted: 28 Dec 2005, 18:14:02 UTC - in response to Message 7811.  

LOL get ready to be flamed! Some people here really take the mickey if you start claiming this or that person is cheating. See the 'cheating' thread I started a while back.


Some people have now been pointed out that actually appear to be cheating; _this_ particular one I don't think is, or if he is, he's incredibly bad at it.

Personally I quit the project 2 weeks ago and will only come back once the stats are fair. It's been proven the stats can be altered even without modifying the client at all. If people are cheating 'intelligently' (ie, not claiming a billion credit but 20 or 30% more instead), then it's undetectable.


That is of course your choice; discussions on the issue are continuing behind the scenes, and I'm pretty sure you'll be able to return fairly soon... :-)

Edit: I was about to say 'next thing you know they'll tell you it's just a glitch because of a bad WU' - and they did. :-D


If you're referring to _my_ comment, I said it was likely a bad run of the benchmarks - there is no way a WU could cause this... the only mention of WUs in this thread is on how to check one to see if the credit claimed is reasonable.

ID: 7834 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1018
Credit: 4,334,829
RAC: 0
Message 7838 - Posted: 28 Dec 2005, 19:44:27 UTC

I removed the host and alerted the user about the problem and suggested not to run the project on that particular computer until the cause of the problem was determined. I do not think it was cheating but a bug (possibly in the client that was used on that particular computer) that caused the ridiculously high credit granting.
ID: 7838 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
The Pirate
Avatar

Send message
Joined: 22 Sep 05
Posts: 20
Credit: 7,090,933
RAC: 0
Message 7853 - Posted: 28 Dec 2005, 23:56:34 UTC

I'm in this project for the science, so it doesn't bother me one way or the other is someone is cheating on the credits. As long as they are not cheating the science.

ID: 7853 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile stephan_t
Avatar

Send message
Joined: 20 Oct 05
Posts: 129
Credit: 35,464
RAC: 0
Message 7889 - Posted: 29 Dec 2005, 13:49:50 UTC - in response to Message 7834.  

discussions on the issue are continuing behind the scenes, and I'm pretty sure you'll be able to return fairly soon... :-)


I like the sound of that :-D
Team CFVault.com
http://www.cfvault.com

ID: 7889 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Lee Carre

Send message
Joined: 6 Oct 05
Posts: 96
Credit: 79,331
RAC: 0
Message 8032 - Posted: 31 Dec 2005, 3:00:51 UTC - in response to Message 7853.  

I'm in this project for the science, so it doesn't bother me one way or the other is someone is cheating on the credits. As long as they are not cheating the science.

the only problem with that is with those who crunch for (fair) credits, because they then (rightly) claim that the credits are meaningless, and aren't as willing to crunch anymore, a lot of CPU power comes from these "uber-crunchers" so the credit system has it's uses

It's also good to compare your production rate to others, for example without RAC (i know it's not accurate, but it gives a rough idea) how would you compare a user with 5 hosts, and a user with 7 hosts
ID: 8032 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile River~~
Avatar

Send message
Joined: 15 Dec 05
Posts: 761
Credit: 285,578
RAC: 0
Message 8048 - Posted: 31 Dec 2005, 15:59:20 UTC - in response to Message 8032.  

I'm in this project for the science, so it doesn't bother me one way or the other is someone is cheating on the credits. As long as they are not cheating the science.

the only problem with that is with those who crunch for (fair) credits, because they then (rightly) claim that the credits are meaningless, and aren't as willing to crunch anymore, a lot of CPU power comes from these "uber-crunchers" so the credit system has it's uses

It's also good to compare your production rate to others, for example without RAC (i know it's not accurate, but it gives a rough idea) how would you compare a user with 5 hosts, and a user with 7 hosts


For me it is not either/or.

I am not in it for the credits but for the science.

But I do like my contribution to be recognised and with thousands of hosts the credit system is one practical way to do it.

I might not have done the washing up for the thanks; but if kid bro got thanked when I did it, *then* I got upset.

R~~
ID: 8048 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
1 · 2 · Next

Message boards : Number crunching : BIG credit problem! User has a zillion credits!



©2024 University of Washington
https://www.bakerlab.org