New credit system now being tested at RALPH@home

Message boards : Number crunching : New credit system now being tested at RALPH@home

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · Next

AuthorMessage
Whl.

Send message
Joined: 29 Dec 05
Posts: 203
Credit: 275,802
RAC: 0
Message 22195 - Posted: 10 Aug 2006, 12:06:11 UTC - in response to Message 22192.  
Last modified: 10 Aug 2006, 12:29:47 UTC

The only info needed is the average run time for each target on a particular spec machine.......

I would think that everyone would be happy with that as a system???


Hi Danny,

That sounds very good, but what happens when an AMD64 @ over 2.6Ghz is reported as a Sempron 3100+ as happened to me, until I changed the OS ? Or some of the massively overclocked systems on some teams that are reported as standard speed ?

Would probably cause another huge row, at a cost to the project.

I dont mean to have a go at you though Danny by saying that.

[Edit] BTW, even after I changed the OS, Rosetta/BOINC still did'nt report the true speed the 64 was running at. It just reported the model, which would'nt really give a true reflection of the work it has been doing.[/Edit]
ID: 22195 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Jack Shaftoe
Avatar

Send message
Joined: 30 Apr 06
Posts: 115
Credit: 1,307,916
RAC: 0
Message 22196 - Posted: 10 Aug 2006, 12:06:47 UTC - in response to Message 22193.  

Going back to the zeroing out credit discussion and after hearing both positives and negatives to it wouldn't it be easier to just have 2 databases?

One database where the current stats could continue with the credits from the new credit system added on but also have a new database of credit from when the new system is put in place starting form zero.


That's a great idea in my opinion. I'm sure we can agree that everyone would get what they wanted if they could see both sides of the coin - just the new system on one set of graphs and combined new/old on another set.

Not sure if it's easy/difficult to implement though.
ID: 22196 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jose

Send message
Joined: 28 Mar 06
Posts: 820
Credit: 48,297
RAC: 0
Message 22197 - Posted: 10 Aug 2006, 12:38:44 UTC
Last modified: 10 Aug 2006, 12:39:25 UTC

I am reaching a very high level of frustration with this type of thread.

I have been working with a friend on a statistical analysis tool that could help pointing out the most egergious deviations in credit claims across the type of client being used, the actual work per credit claimed and the ammount of time crunched per credit and per model. Given the situation here , I have decided it is not worth continuing the testing. To put it mildly , I am having serious doubts about the undersatnding of statistical methods by some people here and I am worried at what is being passed as statistical analysis. To be even more blunt: I am having doubts about the intelectual honesty and the motivation of some of the main posters in this thread.

I am tired of the erroneous comparison of the nature aof the work units in the different projects. THEY ARE NOT COMPARABLE as the projects themselves are not comparable. That is the fallacy that Boinc perpetuates: that differnt in nature projects can be measured by the same standards.

I am tired of the whole optimized vs non optimized clients thing. specially the continuous use of the word cheating. This is the last time you are going to hear me say this: AS LONG AS THE USE OF OPTIMIZED CLIENTS IS NOT EXPLICETLY PROHIBITED IN A PROJECT , THEIR USE IS NOT CHEATING. Specially since the open nature of the BOINC code allows for changes and modifications in the code.

I am tired on the editorializing done in some of the statistics to back up claims that some teams, what the hell, a specific team is cheating. It is a known fact that I am a member of that team. I KNOW that the cheater tag that some of the people here are trying to tag on us is unfair and borders on the slanderous. I am giving fair warning: Attempt to flame my team and my team members at your own peril... Flame and you will be roasted. If you flame us, do not expect us to take your slanders lying down. I AM NOT THE MOST GENTLE OF PEOPLE , AND I AM NOT THE MOST SUAVE OF PEOPLE... this BS against my team has to STOP. Fair warning again: Flame at your own risk.

If you dont like the fact that in my team we crunch for Rosetta as if Rosetta were our wife, lover or best friend and that we only know one speed: full blast and that level dedication (including the investment in state of the art cpus, mobos and memory ) yields the numbers we are getting and THAT WE WILL GET REGARDLESS OF THE CREDIT SYSTEM used; I have news for you: tough noogies.... our team is commited to Rosetta because we believe in the project worth so if you thing we are going to fade out because your baiting and jealousy driven harping ..think again...We will not go away.

As to the tracking of cheats..the real ones... we are still doing it...

So if you are offended by what I have said... I really dont care. I am going back to work in what it ismportant: the crunching of models. But I have given fair warning...dont act surprised if you ontinue to question the integrity of my team (or for what it is worth the top teams in this project) if you dont get a cool reaction. We have tried reason: obviously your agenda in attacking the top teams doesnt allow for reason.

Okies ...flame back at will :P



ID: 22197 · Rating: 0.99999999999999 · rate: Rate + / Rate - Report as offensive    Reply Quote
suguruhirahara

Send message
Joined: 7 Mar 06
Posts: 27
Credit: 162,644
RAC: 677
Message 22199 - Posted: 10 Aug 2006, 13:41:01 UTC
Last modified: 10 Aug 2006, 13:41:44 UTC

I am tired of the whole optimized vs non optimized clients thing. specially the continuous use of the word cheating. This is the last time you are going to hear me say this: AS LONG AS THE USE OF OPTIMIZED CLIENTS IS NOT EXPLICETLY PROHIBITED IN A PROJECT , THEIR USE IS NOT CHEATING. Specially since the open nature of the BOINC code allows for changes and modifications in the code.

So why don't project leaders prohibit the use of the optimised client?
According to posts on seti@home forum, at least the source code of the 5.5.0 was not available, which means that no one except the code writer cannot know what the application does. Even more without publishing source code distributing the software violates GPL. I'm not sure whether even using the software violate it or not, though.
ID: 22199 · Rating: -0.99999999999999 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mats Petersson

Send message
Joined: 29 Sep 05
Posts: 225
Credit: 951,788
RAC: 0
Message 22200 - Posted: 10 Aug 2006, 13:48:21 UTC - in response to Message 22199.  

I am tired of the whole optimized vs non optimized clients thing. specially the continuous use of the word cheating. This is the last time you are going to hear me say this: AS LONG AS THE USE OF OPTIMIZED CLIENTS IS NOT EXPLICETLY PROHIBITED IN A PROJECT , THEIR USE IS NOT CHEATING. Specially since the open nature of the BOINC code allows for changes and modifications in the code.

So why don't project leaders prohibit the use of the optimised client?
According to posts on seti@home forum, at least the source code of the 5.5.0 was not available, which means that no one except the code writer cannot know what the application does. Even more without publishing source code distributing the software violates GPL. I'm not sure whether even using the software violate it or not, though.


Using software produced by GPL-breakages is not illegal - but probably immoral. Producing code without supplying source-code is definitely a breach of contract, the producer of such code is breaking the law, even if he/she/it/they are not gaining anything from doing so, since part of the GPL is that the source code must be made available to anyone, and it must also be available for another three years ...

--
Mats
ID: 22200 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
kevint

Send message
Joined: 8 Oct 05
Posts: 84
Credit: 2,530,451
RAC: 0
Message 22203 - Posted: 10 Aug 2006, 14:01:23 UTC - in response to Message 22171.  
Last modified: 10 Aug 2006, 14:18:08 UTC

Ok first of all I really dont like seeing SETI and Rosetta in the same sentance. They are 2 different projects and point systems so leave the sepprate.

I honestly dont understand why starting from scratch is a problem. We know you did the points, we know we can do them again. This is not costing you anymore money in any way. I myself love the competition.. Go ahead zero me and keep your points cuz you know what... I'll pass you again and again. I am getting really tired of all the compaining from you guys just crunch the project and get on with your life. Stop looking at the top computers and what client they are running and worry about yourself. Xtremesystems is a very dedicated team and we spend more money than you can imagine to get where we are today. I am not saying non of you guys are spending any money... But my biggest issue here is all the bitching and complaining. Did you see us complain when we had to remove 3.5mil points cuz we knew it was the right thing. NO. We just crunched harder and got more members. Do you SETI guys go on their official website and complain that you dont like this or that or someone's computer is faster so they must be cheating.... Just face the fact that we are faster and always will be optimized or not!



EXCACLY my point - you appear to have started Rosetta a few months ago, have a total RAC of 120, and what - a total of 80 credits -
Yes, you would not care - - you are what we call a ZERO RACer.
You have not spent not even a single dollar, euro, etc.. on the project. you have not investment in time nor money. Therefore you would not care if you are zeroed out.
And as for doing the credits again.. I don't WANT to do them again- that is the most stupid thing I have heard in a long time.
On the other hand - there are those that have invested time and money into the project and to just toss out those accomplishments are not only absurd but could very well be project damning. As I know that I would not be back, and there are others, many others that would just leave and go crunch for SIMAP or QMC or something else.

As far as SETI and Rosetta being different projects I agree - but there MUST be a cross project equalization of credit systems. That is one of BOINC's features, the abilty to crunch several projects and be granted similar credit for these project.

Don't be negative towards other project team, if SETI.USA were to turn focus to Rosetta - it would stomp and in no time be the number 1 team here. People crunch what interests them. I am interested in many things so I crunch many projects. And a fair cross project equalized credit system is important.

Are you new to DC ? I have been crunching DC since May of 1999 -
SETI.USA


ID: 22203 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Jack Shaftoe
Avatar

Send message
Joined: 30 Apr 06
Posts: 115
Credit: 1,307,916
RAC: 0
Message 22204 - Posted: 10 Aug 2006, 14:11:38 UTC - in response to Message 22197.  
Last modified: 10 Aug 2006, 14:14:43 UTC

So if you are offended by what I have said... I really dont care.


Not offended, just annoyed to see interesting ideas buried in more long angry posts full of noise. Argh...

To get back to point, resetting everyone to zero is not an option. Too many people don't like that idea and threaten to leave. I don't support that anymore.

What about the 2 graphs idea? One for new system only, one for total in both systems? If there are objections to this - what are they and why? I would like this. Can it be done?
ID: 22204 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
suguruhirahara

Send message
Joined: 7 Mar 06
Posts: 27
Credit: 162,644
RAC: 677
Message 22205 - Posted: 10 Aug 2006, 14:23:06 UTC - in response to Message 22203.  

there MUST be a cross project equalization of credit systems. That is one of BOINC's features, the abilty to crunch several projects and be granted similar credit for these project.

Same here. A few adjustments can go, but the completely different system cannot as far as this project applys BOINC framework at least, since the new system could change the relationship between crunching and 1 credit granted.

just annoyed to see interesting ideas buried in more long angry posts full of noise. Argh...

Me too. Moderators should do something here...
ID: 22205 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B^S] thierry@home
Avatar

Send message
Joined: 17 Sep 05
Posts: 182
Credit: 281,902
RAC: 0
Message 22206 - Posted: 10 Aug 2006, 17:52:14 UTC
Last modified: 10 Aug 2006, 18:27:18 UTC

The word 'cheaters' was used in a lot of threads and most of the time misused. I think many people want to point out the fact that if some use an optimized client and others not, they cannot compare their score to others (especialy when they are low in credits ;-)).

To summarize:

- The cheaters is the guy how do something somewhere to multiply his credits by a few zillions.

- The optimized clients are there, available, free and the projects don't care if you use it or not, so everybody is free to use them (or not).

- Projects like CPDN, Seti, Seti Beta, Einstein, Rieselsieve have a fixed and standard credits system, so no problem with optimized client

- Projects like Predictor, Sztaki, LHC, Simap, Primegrid, ... have a quorum of three results (sometime two), so no problem either.

- Rosetta project team is working on a new credit system.

- Finally QMC is the only project where the optimized client use can be discussed.

So it's maybe the time to stop all those silly discussions/lists/...
Or we can start new flame threads in the QMC forum for those who are interested by flaming (I'm sure there are some).



ID: 22206 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
suguruhirahara

Send message
Joined: 7 Mar 06
Posts: 27
Credit: 162,644
RAC: 677
Message 22208 - Posted: 10 Aug 2006, 18:06:15 UTC
Last modified: 10 Aug 2006, 18:49:36 UTC

Cudgel your brains about the new credit system, visit RALPH's forum, and write your ideas down on that thread. Rosetta project team offers you all to post suggestions here.

edit: not only thierry@home but others should go there
ID: 22208 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B^S] thierry@home
Avatar

Send message
Joined: 17 Sep 05
Posts: 182
Credit: 281,902
RAC: 0
Message 22209 - Posted: 10 Aug 2006, 18:16:34 UTC

Thank you. I already knew it and I have already made it.
ID: 22209 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
njkid32

Send message
Joined: 27 Mar 06
Posts: 15
Credit: 80
RAC: 0
Message 22210 - Posted: 10 Aug 2006, 18:19:50 UTC - in response to Message 22203.  

Ok first of all I really dont like seeing SETI and Rosetta in the same sentance. They are 2 different projects and point systems so leave the sepprate.

I honestly dont understand why starting from scratch is a problem. We know you did the points, we know we can do them again. This is not costing you anymore money in any way. I myself love the competition.. Go ahead zero me and keep your points cuz you know what... I'll pass you again and again. I am getting really tired of all the compaining from you guys just crunch the project and get on with your life. Stop looking at the top computers and what client they are running and worry about yourself. Xtremesystems is a very dedicated team and we spend more money than you can imagine to get where we are today. I am not saying non of you guys are spending any money... But my biggest issue here is all the bitching and complaining. Did you see us complain when we had to remove 3.5mil points cuz we knew it was the right thing. NO. We just crunched harder and got more members. Do you SETI guys go on their official website and complain that you dont like this or that or someone's computer is faster so they must be cheating.... Just face the fact that we are faster and always will be optimized or not!



EXCACLY my point - you appear to have started Rosetta a few months ago, have a total RAC of 120, and what - a total of 80 credits -
Yes, you would not care - - you are what we call a ZERO RACer.
You have not spent not even a single dollar, euro, etc.. on the project. you have not investment in time nor money. Therefore you would not care if you are zeroed out.
And as for doing the credits again.. I don't WANT to do them again- that is the most stupid thing I have heard in a long time.
On the other hand - there are those that have invested time and money into the project and to just toss out those accomplishments are not only absurd but could very well be project damning. As I know that I would not be back, and there are others, many others that would just leave and go crunch for SIMAP or QMC or something else.

As far as SETI and Rosetta being different projects I agree - but there MUST be a cross project equalization of credit systems. That is one of BOINC's features, the abilty to crunch several projects and be granted similar credit for these project.

Don't be negative towards other project team, if SETI.USA were to turn focus to Rosetta - it would stomp and in no time be the number 1 team here. People crunch what interests them. I am interested in many things so I crunch many projects. And a fair cross project equalized credit system is important.

Are you new to DC ? I have been crunching DC since May of 1999 -


Ok I asked the rest of my team to stop coming here and fighting with you children and I am going against my own word. But, if your going to talk to me this way I just cant resist....

Let me just tell you that what RAC that you see there was a long time ago and I crunch for VNS. I am the one with the Kentsfield at 3.5ghz and Conroe at 4ghz. https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=270793
I am not new to DC and if you want to show whos toys are better lets go right ahead cuz YOU WONT WIN!!

If you think that SETI guys want to come to Rosetta and stomp all over XS man, your crazy... But, if you think you can do it bring it kid!

So dont sit here and tell me that I am nobody cuz from the 30th page where your comps are keep clicking the back button till you get to page one then look at the top. Thats why my team call him Clark Kent.

ID: 22210 · Rating: 0.99999999999999 · rate: Rate + / Rate - Report as offensive    Reply Quote
suguruhirahara

Send message
Joined: 7 Mar 06
Posts: 27
Credit: 162,644
RAC: 677
Message 22211 - Posted: 10 Aug 2006, 18:48:59 UTC - in response to Message 22208.  
Last modified: 10 Aug 2006, 18:50:41 UTC

@njkid32
calmdown.

ID: 22211 · Rating: -2 · rate: Rate + / Rate - Report as offensive    Reply Quote
Scribe
Avatar

Send message
Joined: 2 Nov 05
Posts: 284
Credit: 157,359
RAC: 0
Message 22213 - Posted: 10 Aug 2006, 19:03:20 UTC

Gawd here we go again, back to the old pissin contest! Why don't you all just wait and see what Dr Baker's choice is, instead of trying to outguess what they will eventually choose to do.
ID: 22213 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
senatoralex85

Send message
Joined: 27 Sep 05
Posts: 66
Credit: 169,644
RAC: 0
Message 22214 - Posted: 10 Aug 2006, 19:17:03 UTC
Last modified: 10 Aug 2006, 19:18:25 UTC

Hmm, this thread is getting way off topic. May I ask that we focus on the new credit system as best we can so that when the Baker lab reads this post they can better address our concerns.

Thanks mmciastro for your information. Keep us posted on what gooes on RALPH as I do not have time to keep up with it and like periodic updates!

Jose, I would suggest talking with the Rosetta @ home staff about your statistical tool. I would agree with you that most of the users on the boards here (including myself) do not have the necessary tools to perform some of the statistical analysis that you seem to be eluding to. I have a hard enough time applying standard deviation. Its a great idea and if the staff is willing to work with you on it, I think it would be an asset to this project.

What does everyone else think about using statistics to catch dishonest crunchers? If so, do you have any suggestions on a system so that honest people do not get upset when they are mislabled as dishonest?
ID: 22214 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1831
Credit: 119,537,301
RAC: 5,962
Message 22221 - Posted: 10 Aug 2006, 22:31:43 UTC - in response to Message 22195.  

The only info needed is the average run time for each target on a particular spec machine.......

I would think that everyone would be happy with that as a system???


Hi Danny,

That sounds very good, but what happens when an AMD64 @ over 2.6Ghz is reported as a Sempron 3100+ as happened to me, until I changed the OS ? Or some of the massively overclocked systems on some teams that are reported as standard speed ?

Would probably cause another huge row, at a cost to the project.

I dont mean to have a go at you though Danny by saying that.

[Edit] BTW, even after I changed the OS, Rosetta/BOINC still did'nt report the true speed the 64 was running at. It just reported the model, which would'nt really give a true reflection of the work it has been doing.[/Edit]


Yeah - its a good point, but I don't think we'd need to know specifically what speed the computers run at - just that their config hasn't been changed during the time they've been running rosetta. It might be possible to calibrate the old jobs from when they were run on the inhouse computers, although I'm sure a fairly large sample of computers could be put together that have been running from very early on which would give a fairly accurate measure. I'd have a go myself, but i've not got the spare time at the moment.

I think it'd just be a case of taking a pool of (say 20) computers, and then for each job type that's been released we just need to know the number of models produced and the time taken so the average time per model can be calculated for each computer. The new jobs can then be run on these computers and the average time taken per model calculated again.

Then it's just a case of working the following out for each computer:
A new job
New job total time = 13900
New job number of decoys = 32
Average CPU time per decoy (Xnew) = 434.4
Credits awarded per decoy (CA) = 2.00

calculating the credit for an old job
Job ABC123 total time = 14400s
Job ABC123 number of decoys = 32
Average CPU time per decoy (Xold) = 450
Credits awarded per decoy = (CA / Xnew * Xold) = 2.07

Obviously you need to take an average over, say, 20 computers, but from this we can (even now) award credits fairly accurately right back to the start of rosetta.

ID: 22221 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
BennyRop

Send message
Joined: 17 Dec 05
Posts: 555
Credit: 140,800
RAC: 0
Message 22222 - Posted: 10 Aug 2006, 22:32:59 UTC

If those that are claiming 10 times or more the RAC for that cpu - are being flagged by Jose, and their models/hour is equal to the standard for that cpu - then I don't see any problem with presenting that quietly to Dr. Baker and letting the Rosetta team deal with it. They've got the records for each of the major WUs and can tell what the "average" credit/model for each of the major WUs is; and see how out of line potential problem systems are reporting. But it should be kept quiet and not the public lynching that has been going on for the past few months.

Regardless, it's a short term problem - since it'll disappear when we move to the new credit system.

Since the Rosetta labs has mentioned several times that the various optimized boinc clients were okay to use - it's a little late to normalize their scores for running the client with Rosetta. Once you've handed out a bowl of crisps/chips and given the crowd permission to eat them - once they've eaten the bowl of crisps/chips you can't ask for them back. (Although it's been pointed out that some here might actually attempt that..) It's time to forget about the war of the benchmarks and move on to the upcoming War of Model Production.

The most interesting bit will be finding which cpus produce better than they benchmarked with the old system.. in which case, if we have the benchmarks being done by the best benchmarking cpus under the old system, some cpus will get a really nice boost in scores if they can outproduce the benchmarking cpus.

Take a look at the credit system thread, and post your suggestions for possible improvements.
ID: 22222 · Rating: 2 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile gomeyer

Send message
Joined: 11 Jun 06
Posts: 5
Credit: 181,323
RAC: 0
Message 22224 - Posted: 10 Aug 2006, 23:29:09 UTC

Just a couple of general thoughts about a new credit system from someone new to R@H, if I may. My theme will be Equity.

Although no credit system is likely to make everyone happy achieving some level of Credit Equity within the Rosetta community would be very beneficial. Judging by some of the views expressed in this thread I’d say such an attempt is appropriate.

However the most important potential gain to be made by a new credit system would be with one that attempted to create better Cross-Project Equity.

BoincStats and similar sites make competition across projects inevitable. People are less likely to join any project that is not at least on a par with others and we all know how important more participation is here. Also, just a guess but I’m thinking some current Rosetta participants might consider increasing their share if points were awarded with something approaching equity. I for one would.

My point is that I hope that any new credit system would specifically address these two issues.
ID: 22224 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
XS_Vietnam_Soldiers

Send message
Joined: 11 Jan 06
Posts: 240
Credit: 2,880,653
RAC: 0
Message 22229 - Posted: 11 Aug 2006, 1:09:27 UTC

After reading through all the comments and ideas, I'd be interested in knowing exactly how many WU are actually getting "crunched" on the average day/week,etc..
Not points, but actual work units..
Is there a moderator or someone who can find this info for us?
Might be interesting to see the breakdown.
Thanks for your time,
Movieman

ID: 22229 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Hoelder1in
Avatar

Send message
Joined: 30 Sep 05
Posts: 169
Credit: 3,915,947
RAC: 0
Message 22231 - Posted: 11 Aug 2006, 1:38:55 UTC - in response to Message 22229.  

After reading through all the comments and ideas, I'd be interested in knowing exactly how many WU are actually getting "crunched" on the average day/week,etc..
Not points, but actual work units..
Is there a moderator or someone who can find this info for us?
Might be interesting to see the breakdown.
Thanks for your time,
Movieman

I think the number you are looking for is actually listed on the homepage under server status: Successes last 24h: 132,647. If I am not mistaken these would be the number of WUs crunched in the last 24 hours, but please note that a WU takes anywhere from 3 to 24 hours to complete, depending on the user's preferences. Perhaps structures per day would be a more meaningful number. If David Kim's estimate of 2 credits/decoy that he uses for his Ralph tests is approximately right than the structures per day would be roughly 'credits last 24 hours' divided by 2 or about 2,000,000 structures per day. I hope this helps, -H.

Team betterhumans.com - discuss and celebrate the future - hoelder1in.org
ID: 22231 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · Next

Message boards : Number crunching : New credit system now being tested at RALPH@home



©2024 University of Washington
https://www.bakerlab.org