I never understood how the RaH project determines output per WU/result and the relative value of it. We can select to run a WU for 6, 12 or 24 hours (for example) and a certain number of decoys are produced. Are all decoys equivalent, that is, from a science point of view is it better to run:
1) a single 24-hour WU?
2) 2, 12-hour WUs?
3) 4, 6-hour WUs?
4) 24, 1-hour WUs?

Which scenario is likely to produce the most valuable output? If the project could dictate the runtime per WU, which would it choose?
Thanks

I never understood how the RaH project determines output per WU/result and the relative value of it. We can select to run a WU for 6, 12 or 24 hours (for example) and a certain number of decoys are produced. Are all decoys equivalent, that is, from a science point of view is it better to run:
1) a single 24-hour WU?
2) 2, 12-hour WUs?
3) 4, 6-hour WUs?
4) 24, 1-hour WUs?

Which scenario is likely to produce the most valuable output? If the project could dictate the runtime per WU, which would it choose?
Thanks

All are the same with respect to the number of possible decoys done. (decoys vary in length)
All the longer hours decoys do is keep running (restarting) after a decoys have finished with the 3hr default time (I think it is that long) with a different random seed. (you'll see it in the result info).

1hr tasks could be different as the watchdog timer may kick in, and you'll get no results ? I'm not sure here. (or it could just keep going until one decoy has been done.

When selecting above 3 or 4 hours you start to reduce bandwidth for yourself and the project, but of course task take longer to return (though does the long wait to report a result negate this effect ?)

---

Team mauisun.org

They basically are going for x number of random models crunched. Say they've decided for a given protein they want 100,000 models crunched. To the science, it is all the same as to how those 100,000 models are done. 100,000 WUs, or 100.

Each WU has a different random number "seed". In computer programmer speak, a "seed" is a starting point for generating random numbers. If two machines had the same seed, they would crunch through the same exact models in sequence. So, from that single seed, you generate a new random number at the start of each model. This "random" number is predictable if you know the seed, which model from that starting point you want, and the formula being used for random number generation.

So, it's not that you are crunching the same models over and over. You are basically adding one to the seed each time and generating a new random number which is used as the basis for the model.

So, with a longer WU runtime, you crunch more models on the same WU. They are all equally valuable. And there's really no difference between crunching 50 models in a 24hr WU, and crunching 25 models each in two 12hr WUs. No difference to the science, which was your question. By only downloading one WU and running 24hrs, your bandwidth is reduced dramatically. And you hit the server half as many times.

If the project had to chose, they'd probably say something like "it depends". And try and align you with a WU runtime which lets your machine crunch for about a day and then report back the results. If you are crunching 24hrs, then this would mean 24hr WU runtime preference, regardless of the speed or model of your machine. But if your machine is only on about 8hrs a day, then it would mean using something closer to an 8hr preference. But the reason they give YOU the choice of WU preference, and a 1 week deadline is that they want you to do what works for you. The science is really the same regardless.

---

Add this signature to your EMail:
Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might!
https://boinc.bakerlab.org/rosetta/

Each WU has a different random number "seed". In computer programmer speak, a "seed" is a starting point for generating random numbers. If two machines had the same seed, they would crunch through the same exact models in sequence.

So when is a random number not a random number? To be technical it is a pseudo-random number generator ;-)

Just to be a bit of a wise a** :-)

---

!!Stupidity should be PAINFUL!!

I never understood how the RaH project determines output per WU/result and the relative value of it. We can select to run a WU for 6, 12 or 24 hours (for example) and a certain number of decoys are produced. Are all decoys equivalent, that is, from a science point of view is it better to run:
1) a single 24-hour WU?
2) 2, 12-hour WUs?
3) 4, 6-hour WUs?
4) 24, 1-hour WUs?

Which scenario is likely to produce the most valuable output? If the project could dictate the runtime per WU, which would it choose?
Thanks

As already decribed, the scientific value is proportional to the number of decoys for a given decoy, and so all your opptions would have about the same science value. Providing, that is, the WU all ran exactly to the preferred time. You will have noticed the times jitter around the target value.

As you move from option 1 to 4, the variation in length becomes more pronounced, as the random variation in run length is pretty much independent of the overall run time. A ten minute underrun counts more in a 1-hr WU than a 24hr one. Note that the rough amount of overrun/underrun varies on the type of decoy, ie the type of WU being run.

The amount of data downloaded per WU is fairly independent of the run length. This means that if download bandwidth is the main issue for you, option 4 costs of 24x option 1 for the same science value, and indeed the same credit.

The amount of data returned on upload at the end of the run is proportional to the number of decoys plus a small overhead. So option 4 would mean slightly more data uploaded than option 1, but nowhere near the ratio seen on download.

History

The idea to limit each WU to an overall runtime rather than to a fixed number of decoys was first suggested because variation in run-length between decoys was causing annoying side-effects to the BOINC code, which tend to assume that a project knows roughly how long a WU will take.

The idea to give users a choice came because there were complaints from dial-up users and those with metered connections that WU were costing them too much in download and needed to run longer to be worth having, together with simultaneous complaints from multi-project users that the WU were already too long and they wanted nice short WU so that they could finish work from several projects in the same day. By limiting the run time to a user-selected value rather than to a hard coded one, all these users could be provided for.

As Feet1st says, they wanted to give us the choice.

Technical

How it actually does it is that the server has no idea how many decoys you are going to run, and it doesn't even try to guess. You get sent a WU with an initial random number seed. The server code doesn't even know how long you want to run the work for, all WU can run for all time lengths.

At the end of every decoy, the app checks the time taken so far to do N decoys, divides by N to get the average so far, and then decides if it has time to do one more. A nice touch is that the app looks at the preferences each time, so you can even stretch or shrink the WU while it is running!

River~~

---

Sorry, I must be a little dense today, I'm still not getting it from a "science" point of view.

Crazy-off-the-wall-hypothetical-example (assume for the sake of the hypo the following is illustrative and not figurative): I win some sort of contest that will allow me 24 hours of time on the Blue Gene supercomputer, and I donate that time to Rosetta crunching.

I can run one WU for 24 hours and complete 24,000,000 decoys.

I can run two WU's @ 12 hours / 12,000,000 decoys each.

...

I can run twenty-four WU's @ 1 hour / 1,000,000 decoys each.

For the "science", there has to be a point of diminishing returns.

At what point are decoys on a new WU more valuable than additional decoys on an original WU?

Or perhaps to be asked in a different way, why ARE we crunching different work units? Why aren't we all running the same work unit to generate a gazillion decoys?

At what point are decoys on a new WU more valuable than additional decoys on an original WU?

Or perhaps to be asked in a different way, why ARE we crunching different work units? Why aren't we all running the same work unit to generate a gazillion decoys?

Realistically, only ONE structure really matters -- that global minimum energy structure. To say the rest don't matter is perhaps a little too strong a statement, but its often not too far from the truth.

Think of it as a sort of a lottery. There are typically millions of different combinations out there, but only one winning combination of numbers. Sometimes, the ones that get some of the numbers right are even worth something, but not as much as that winning ticket. So, how many lottery tickets have to be bought to "guarantee" a winner? (all possible combinations) Can there be a winner if there is only one ticket sold? (yes -- not likely, but still possible)

How does one decide "how many tickets to buy"? Well, you start with a "seat of the pants" type of guess. You look at the number of variables involved. In the Rosetta case it would be related to the size and flexibility of the protein in question.

Also, remember that a lot of the Rosetta work has been done on proteins of known structure (they know the "right answer"). Doing work like this will help them to refine the number of structures to request for proteins of unknown structure. As with all scientific models, the more data that you incoroporate from diverse areas, the better your model, and subsequent predictions, become. I also think that different people in the group are working on different proteins, and there is value in keeping all of them busy.

Prior to CASP, we were producing 10k decoys/models per WU. It didn't matter if 10 people produced 1000, 100 people produced 100, or 1000 people produced 10, or a mixture of amounts. The goal was 10k per WU.

For CASP, we were producing 100k decoys/models per WU; although for at least 1 WU we produced up to 1.5 million decoys/models.

They know the likelyhood that producing 10k decoys will result in a decoy of a certain RMSD range from the natural fold of the protein. They know the likelyhood of producing 100k decoys will result in an even better decoy (and give a larger pool of decoys with low RMSD and low energy).

Since we're working on improving the science app, only producing 10k decoys per WU is sufficient to see if changes in the algorithms result in better low RMSD, low energy decoys.

For the HIV WUs, we might perform 100k decoys/models per WU to build a sufficiently large description of how the "landscape" of the protein behaves. And then have them the scientists send us out a new WU that incorporates what was learned during the first 100k decoys to allow us to zip through certain calculations in a minute that took 30 minutes in the first set of 100k decoys.

It was an approach that being played with/tested prior to CASP to allow us to produce the same result from 1 batch of 10k decoys.. followed by a created 2nd WU that we'd produce another 10k decoys - as producing 100k decoys of the original WU from scratch.

The time duration each of us choose doesn't matter to the science. I use 24 hour run times to help cut down bandwidth usage on the Rosetta servers; and make sure my machine isn't uploading/downloading Rosetta data when I'm trying to view something linked from YouTube.

If you're on a slower system, and you've chosen 1 hour run times, you probably won't have 24 WUs to upload every day. If it takes your system 4 hours to produce a single decoy.. it'll continue on with that first decoy for 4 hours until it finishes.

At least this is what I've got out of the various discussions we've had about these issues.

---

Ok, understand a little better now.

So, I guess the jist of it is, so long as I don't have a crazy crunching farm that can churn out more than 10k/100k decoys/model per day, it doesn't make any difference as to my individual run time. My results will be aggregated with others for the same WU to reach the 10k/100k point, to decide what changes are needed in the algorithm.

Boy, it'd be nice to one day reach the point cruncher-wise where I WOULD have to worry about generating more than 10k/100k per day!

Since we're working on improving the science app, only producing 10k decoys per WU is sufficient to see if changes in the algorithms result in better low RMSD, low energy decoys.

For the HIV WUs, we might perform 100k decoys/models per WU to build a sufficiently large description of how the "landscape" of the protein behaves. And then have them the scientists send us out a new WU that incorporates what was learned during the first 100k decoys to allow us to zip through certain calculations in a minute that took 30 minutes in the first set of 100k decoys.

It was an approach that being played with/tested prior to CASP to allow us to produce the same result from 1 batch of 10k decoys.. followed by a created 2nd WU that we'd produce another 10k decoys - as producing 100k decoys of the original WU from scratch.

Penguin.
You're asking the right question. The answers so far to that have been right on. I just wanted to add that to some extent it depends upon what the project team's purpose for the WUs is. I mean if they are studying a protein with a known structure, they may be testing a new approach to see if they can find the true structure with less random hunting around, and have now learned how to sniff out the best structures ahead of time, or perhaps the opposite to weed out the worse structures sooner in the calculations.

If they are studying an unknown protein (as with CASP), it's size will be significant. Also, how much they already know about the amino acid sequence will come in to play. If they've already "bar coded" portions of the protein, or if it is comprised of a small number of amino acids, then perhaps 10,000 models is sufficient. But if it is a large protein, or largely unknown sequences, they may create one WU, run it for 10,000 models just to get a feel for the landscape, and then create another WU to focus on the most promising areas of that landscape.

When the true structure is known, you can compute your RMSD to that know structure and see if you think you are approaching as close as you are going get, and the question is, "did you arrive at that solution any more efficiently then your prior version of the program?"

When the true structure is not known... well, I gotta believe they have some idea what energy level they're shooting for. But I've never seen this discussed.

[edit] Just saw your post Penguin. Yes, the massive numbers involved is why your individual WU runtime preference doesn't really matter to the science. If they're not getting the projected number of models on a WU, they just send out a few more of them. And the precise number of models is not a precise number, so if we come in within 10% of the target number of models, that's close enough. (I have no idea how close they actually come to their targets).

It ALSO points out WHY there is a need for distributed computing to solve the problem.

CRUNCH MORE ROSETTA! (it's been a while since I posted that)

---

Add this signature to your EMail:
Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might!
https://boinc.bakerlab.org/rosetta/

A few months ago one of the project team posted in response to this type of question. The only actual number he gave was for an average desktop crunching Rosetta 24/7. He said "8 hours would be fine" or something like that if bandwidth wasn't a problem. He wouldn't give any other suggestions to those crunching several projects or if your machine was off part of the day.

A few months ago one of the project team posted in response to this type of question. The only actual number he gave was for an average desktop crunching Rosetta 24/7. He said "8 hours would be fine" or something like that if bandwidth wasn't a problem. He wouldn't give any other suggestions to those crunching several projects or if your machine was off part of the day.

My recollection of what David Kim said -- when the time parameter was implemented -- was that he preferred 8 hours or more to minimize the number of results that had to be stored in the data base. For people with modems and upload/download issues then shorter runtimes were a necessity though not preferred from the point of view of the project servers.

...snipped...
For the "science", there has to be a point of diminishing returns. At what point are decoys on a new WU more valuable than additional decoys on an original WU? Or perhaps to be asked in a different way, why ARE we crunching different work units? Why aren't we all running the same work unit to generate a gazillion decoys?

Thankyou, you stated more precisely what I was attempting to infer.
So is a result and a WU synonymous or are "results" a sub-division of a WU?
Are we aggregating "results" from different hosts on a single protein?

In the end I don't really care I guess; I have no personal need to know the inner workings of the science. I just want to contribute to the project in the most efficient way. I have a P4/HT 3GHz and cable modem so I have no issues other than to run the tasks efficiently. If the team wants 8-hour run times or 12-hour run times it doesn't matter to me; I just need to know which.

Sorry 'bout that, didn't mean to hi-jack your thread...

My understanding is that Work-Units are a supra-set, with the generated Decoys as the sub-set. The same Work-Unit is sent out to many crunchers so that in aggregate they generate about 10,000 Decoys.

With a cable modem, you don't have the same bandwidth issues that dial-up users would have.

Personally, I used to run 24h WU's, but now I'm down to 12h WU's only because of the fear that if I lose power or something I'd rather not lose 23 hours of work as opposed to 11 hours of work.

I know that this is probably an unreasonable fear, due to checkpointing, but I know there have been whole threads on how frequently check points should happen.

Hope this helps more than it confuses...

...snipped...
For the "science", there has to be a point of diminishing returns. At what point are decoys on a new WU more valuable than additional decoys on an original WU? Or perhaps to be asked in a different way, why ARE we crunching different work units? Why aren't we all running the same work unit to generate a gazillion decoys?

Thankyou, you stated more precisely what I was attempting to infer.
So is a result and a WU synonymous or are "results" a sub-division of a WU?
Are we aggregating "results" from different hosts on a single protein?

In the end I don't really care I guess; I have no personal need to know the inner workings of the science. I just want to contribute to the project in the most efficient way. I have a P4/HT 3GHz and cable modem so I have no issues other than to run the tasks efficiently. If the team wants 8-hour run times or 12-hour run times it doesn't matter to me; I just need to know which.

Or perhaps to be asked in a different way, why ARE we crunching different work units? Why aren't we all running the same work unit to generate a gazillion decoys?

this is a good question. the answer is that we are trying to solve many different problems, and experimenting with many new approaches. by having different work units running simultaneously, we get feedback on many different fronts which speeds the improvement of our methods. but you are correct--if we were just trying to predict the structure of a single protein, we would run the same work unit on all of your computers (we did this to some extent during casp, but there were so many prediction problems that each one only had a fraction of a day to itself)

---

Just to add to this, (for clarity)

the default run time, if you just leave rosetta to do its thing set to 'default' is 3hrs.

---

Team mauisun.org

So is a result and a WU synonymous or are "results" a sub-division of a WU?

A task is sent out to the user, and a result sent back. Loosely speaking we often speak of the task being a "result" even before it is returned (eg it has a resultid, not a taskid).

The term WU is used in two ways on this project, one is the same way it is used on other BOINC projects, the other is Rosetta-specific.

On this project, many results with the same protein, the same processing method, and different starting seeds are often referred to loosely as being "the same WU". That is how the term has been used in this thread.

On other projects, and on the pages generated by BOINC code, a WU means a cohort of different results that all have absolutely identical parameters all the way, and which are given to different boxes to crunch in the hope that the answers will be either the same, or at least "close enough" to each other.

On Rosetta this only happens when a result is returned with an error, and then exactly the same thing gets sent out again a limited number of times. This helps David and friends work out if it is an instabilty in their code or a user-generated blip. This workunit for example was first sent to me, but I needed to take the box offline urgently before it even started to run, so I aborted it. It was then re-allocated to someone else coplete with the same starting seed.

Almost all the time, then, each WU (in the BOINC sense) has only one result on Rosetta.

Are we aggregating "results" from different hosts on a single protein?

Kind of. But sometimes David et al like to use several methods on the same protein, in which case only those from method 1 will be aggregated together, and those from method 2 aggragated together separately, and so on. Each of these aggregates can be compared for accuracy and how much computing they took to achieve.

River~~

---