Hi all,
I have been wrangling with dividing & defining my disk space amongst 4 projects (Rosetta, Seti, WCG, Climateprediction).

I have read a few threads on the topic but nothing definitive/that is straightforward.

My first question is, where are the web preferences read from? I have set the disk settings to be the same for all projects, but I suspect WCGs settings are used for the BOINC client - is this so? if so, why?! is there a global setting elsewhere separate from projects?

Noinc disk setting (web prefs, no local prefs used):
https://imgur.com/Xh2bDQA


Secondly, I am struggling to understand how much disk space projects need, especially Rosetta.

Rosetta (along with the other projects) is currently set to use 1GB max (I have read that more is "needed" but I'd rather be precise and have it scale its requirements to use what is provided above its minimum requirement, as all applications should) but it either uses tons of space if I make more available, or as it is now, it needs 953MB but says less is available. I can only guess that the 1GB I have specified is not being allocated and probably divided amongst all 4 projects. Can someone confirm what is happening vs the settings I have, as below?

Projects and share:
https://imgur.com/BPSFaX0

Current boinc project disk allocation:
https://imgur.com/oTckSPg

Rosetta space request:
https://imgur.com/gAGVKxQ

Every project is set to use 1GB disk max:
https://imgur.com/CPIeUMz


Cheers[/img]

---

hi,

web preferences are in your profil. you can change everything there. WU-runtime, RAM usage, etc.... almost everything.
maybe if you change the settings locally, the the client may have some problems with diskspace, i dont know.
I use the webpreferences, so rosetta is running the same on all of my computers.
Check it in WGC, if there are options to do so also.

My boinc is using 5GB of diskspace, but is allowed to use almost the whole disk.
needed diskspace depends on your CPU.
e.g. Ryzen 5 3600 is able to work on 12 WUs at the same time.
each WU is set to 4 hours.
so in 24 Hours i work on 48 WUs.
I allowed boinc and rosetta to save 1 working day and have two days additional.
so i have to store 3 days which is 144 WUs on disk.
i can´t check it right now, so please check filesize on your system for each WU.
some are bigger, some are less. just for having an idea of diskspace.


hopefully i could answer your questions.

Thanks for your response Meek - apologies int he delay: I didnt receive a notification for this post!

re: web preferences - yes I use and understand that, but I have specific questions about this as per the OP.

re: work units per cpu - where is this defined exactly?

As per the OP - the settings speak of maximum disk space etc and I need to know how to have those settings followed by BOINC - in addition to my other questions.
As it stands disk usage demands do not match what I have specified in settings - maybe the settings aren't written to reflect what they actually do or something else needs to be set (or I don't understand them!)


Any help with specifics on how to achieve what I aim would be appreciated - simply allowing BOINC to 'use as much as it wants' is not a strategy, nor an approach I take with any program - systems are to be managed, not controlled by the applications needs!

---

Thanks for your response Meek - apologies in the delay: I didnt receive a notification for this post!

re: web preferences - yes I use and understand that, but I have specific questions about this as per the OP.

re: work units per cpu - where is this defined exactly?

As per the OP - the settings speak of maximum disk space etc and I need to know how to have those settings followed by BOINC - in addition to my other questions.
As it stands disk usage demands do not match what I have specified in settings - maybe the settings aren't written to reflect what they actually do or something else needs to be set (or I don't understand them!)

Any help with specifics on how to achieve what I aim would be appreciated - simply allowing BOINC to 'use as much as it wants' is not a strategy, nor an approach I take with any program - systems are to be managed, not controlled by the applications needs!

For work units per cpu core, this is a function of runtime and the number of days of work you store.

Target CPU run time is held at the project: select Rosetta in the Boinc Manager and select Your Account in the Projects tab, then go to Rosetta Preferences. The default is 8 hours
Back in the Boinc Manager, under the Options menu, select Computing Preferences and on the first tab - Computing - first ensure 100% is selected for both %age of CPUs (cores of your CPU) and %age of CPU time. Then chose
"Store at least [0.1] days of work"
and "Store up to an additional [1.4] days of work" (you may prefer different settings, but I wouldn't go too much more in case you can't meet deadlines)

This is a manageable compromise between having a buffer of tasks and meeting their deadlines, which vary between 3 and 8 days. So if you have 8 cores, you'll have 4.5 8hr (=1.5 days buffer) x 8 (cores) = 36 tasks in your buffer

I use a different mix of priorities between Rosetta and WCG compared to you, so on the Disk Manager tab of the Boinc Manager currently shows 4.47Gb needed for Rosetta and 342Mb for WCG, totalling just short of 5Gb disk space used and I allocated 10Gb to Boinc in total (reduced from a default I think was 20Gb, which seemed to be too much)

If I were you, I'd temporarily allocate 10Gb to Boinc to ensure you don't continue to receive the error messages you currently are, fill your buffer for each project, see what the disk tab is telling you, then cut back disk allocation to match what it's telling you (plus a margin for variation)

---

Thanks Much Mr. Celery! That certainly nails part of my question - I have 50/60% cpu/cputime and 0.5/0.5 for work units/additional - I'll look into your settings later and try to calculate the outcome cs my current ones.

I'm guessing from what you say I could set parameters to ensure I use no more than 1GB of disk for Rosetta without missing deadlines?
I wish we could just set the settings more explicitly instead of all these calcs (or at least have the results of our settings displayed as we set them!)

Oh and re: "in the Boinc Manager, under the Options menu, select Computing Preferences ... " these settings are read from the web, and as per my OP I dont see a non-project setting so these must be from one of the projects - do you know the source of the settings which populate the managers computing preferences tab?

Cheers.

DD

---

Sorry for the late reply - I've been very tied up recently.
I'm going to rearrange your post to reply to it in bits

Thanks Much Mr. Celery! That certainly nails part of my question
I'm guessing from what you say I could set parameters to ensure I use no more than 1GB of disk for Rosetta without missing deadlines?

I'm not sure I'm claiming you can do that. If you run 4 different projects, you'll have to run a very tight ship to cram it into anything near 1Gb

I have 50/60% cpu/cputime and 0.5/0.5 for work units/additional - I'll look into your settings later and try to calculate the outcome cs my current ones.

50% CPU means that if you have, say, a 4-core CPU, only 2 will be used to run tasks and the other 2 not run any tasks.

60% CPU time is a bit confusing. You'd think it would mean it only utilises 60% (of each of the 2 running cores), leaving 40% for other tasks. Unfortunately it doesn't.
I found out a long time ago it means it runs each of the 2 cores at 100% for 60% of the time, then 0% for 40% of the time, which is a very different thing. If that's good enough for you, fine, but I'd really recommend switching up to 100%. Your tasks will finish quicker and be just as productive, only in less time (I think) and use no extra disk space.

The 0.5.5 is also a bit odd. In total you'll have 1 day's worth of tasks, which is fine in itself (though the less you ask for, the less disk space you'll use, if that's your priority). You certainly won't miss any deadlines.
The first 0.5 means that more tasks will come down when you've still got 0.5 days (12hrs) worth of tasks still left to run. The second 0.5 means it'll grab another 0.5 days worth of tasks to make a total of 1 day.
I chose to use 0.1 for the first field, meaning 2.4hrs of tasks, then 1.4 (days) of buffer to make a total of 1.5 days. You could also reduce your first field and it would reduce the disk space you require (eg 0.1.5 = 0.6 days worth of tasks - 40% less than you currently store)

I wish we could just set the settings more explicitly instead of all these calcs (or at least have the results of our settings displayed as we set them!)

It is quite detailed as it is - it just doesn't help you with the totals. And as I said, none of them quite mean what you'd initially think they mean.

Oh and re: "in the Boinc Manager, under the Options menu, select Computing Preferences ... " these settings are read from the web, and as per my OP I dont see a non-project setting so these must be from one of the projects - do you know the source of the settings which populate the managers computing preferences tab?

I can't really advise here. I only use 2 projects and vastly prioritise Rosetta (96.67%) over WCG (3.33%) so I've never had to think about it before.

---

Thanks again Sir Celery (and no thanks again Boinc for the non-notification!)

Now that I am forced to stay home I have some time to fix all my niggling issues (and I still need to virtualise my pc which is why I want storage space to minumum to reduce the overall base image size).

I have squished your suggestions into one set of settings (50% cpu and 100% of cpu time with 01./1.0 for work units) and I will tweak towards your suggestions over time.

I also realise that the settings hint toward the relationship:
Boinc --> project
thus there is a kind of master/macro setting for resource allocation (e.g. "2GB for Boinc == no more than 2GB in total across all projects (although the totals don't add up!)) so at least I can (potentially) cap the disk-space usage from the macro level.
project level is still a mess to control but I've taken your advice and will see how it goes. it doesn't help that Climateprediction.net has a default 1 hour delay for communication updates..

Thanks again

---

Every day I am getting requests from Rosetta to increase space - FFS I already have 10GB allocated to BOINC - how much does Rosetta need???

---

Every day I am getting requests from Rosetta to increase space - FFS I already have 10GB allocated to BOINC - how much does Rosetta need???

It will tell you in those messages.
RAM requirements are roughly 1.3GB of RAM for each Task running. With HDD space allow roughly 310MB of HDD space for each Task you have downloaded- so the more Tasks you process at a time, and the larger your cache then the more HDD space you will need.
So set a small cache, and you'll reduce (a lot) the amount of HDD space needed.
eg store at least 1 days of work (or less), Store up to an additional 0.02 days of work.


If you're processing with all 6 CPUs and have a 1 Day cache, you'll need around 6GB of HDD space just for Rosetta. Add extra for other projects.
If one of your other projects uses a lot, then you'll need to add a lot more- check the Disk tab on the Manager to see what is using how much.

---

Grant
Darwin NT

Every day I am getting requests from Rosetta to increase space - FFS I already have 10GB allocated to BOINC - how much does Rosetta need???

It will tell you in those messages.

That was definetly rhetorical! The question is WHY does it need so much (and WHY is the required amount not indicated anywhere within this software?)

RAM requirements are roughly 1.3GB of RAM for each Task running. With HDD space allow roughly 310MB of HDD space for each Task you have downloaded- so the more Tasks you process at a time, and the larger your cache then the more HDD space you will need.

Great! I want to use 1GB maximum, so 3 tasks should do it. So how do I specify the amount of tasks I download then?

So set a small cache, and you'll reduce (a lot) the amount of HDD space needed.
eg store at least 1 days of work (or less), Store up to an additional 0.02 days of work.

If you're processing with all 6 CPUs and have a 1 Day cache, you'll need around 6GB of HDD space just for Rosetta. Add extra for other projects.
If one of your other projects uses a lot, then you'll need to add a lot more- check the Disk tab on the Manager to see what is using how much.[/quote]


Thankyou Grant - apologies for the late reply (I just found out the email notifications were never enabled!)

Your detail is very helpful as it seems Rosetta/Boinc expose very little about how to manage resources - it is beyond vague!

The annoying thing is that even in my frustration, I have reduced Rosetta to:
10 resource share (or 6.99%, of 4 projects)
50% CPu's (of 16 cores)
50% cpu time
Store at least 0.1 days of work
Store up to an additional 0.1 days of work
Use at most 1GB HD (this is not respected at all)

and after a project reset it downloaded 1 task (at the time I looked) and now has zero running tasks and occupying 1.67GB of storage (~80% of all projects usage), asking for more ("Rosetta needs 427.35MB more disk space. You currently have 1289.27 MB available and it needs 1716.61 MB. 02/05/2020 20:08:22") - ultiumately, the requested storage has not really reduced, and it is storing 1.67GB even when no Rosetta tasks are running)

So going by what you have written Rosetta should not be demanding so much storage, but it is!

Any ideas of how to DIRECTLY dictate to Rosetta what to use? even with number of tasks as above?

Do you know what the exact relationship/calculation between cpus --> tasks--> HDD space usage? (especially based upon my reported settings and number of cores)
It doesn't make sense to me to have to consider technical specifics like cores vs cache when all that is needed is the software takes the target ("use at most n GB storage") and then adjusts its variables to suit. This project is a major step backward in regards to user interface design and usability! (this comment is not for you, of course!)

I'm not sure if I posted this but I currently have a 16 core Ryzen 7 (although this has been an issue for my last few PCs, maybe prior to my first dual core)

I'm getting to the point that deleting the project is the only way I can avoid this as it impacts my other project (e.g. climate predictions never runs and WCG and Seti often lack space due to what Rosetta takes).

Please help!
and thanks again

---

Great! I want to use 1GB maximum, so 3 tasks should do it. So how do I specify the amount of tasks I download then?

Sorry, but i didn't see your desire to use 1GB disk space maximum.
Can't be done*, and if you really insist on that then it's best you detach from Rosetta, it processes a lot of data compared to many of the other projects. If you can live with 2.5GB for Rosetta, plus what's needed for your other projects, then you can continue to to do work for Rosetta.
*(it might be possible, but given what would be involved and the capabilities of your hardware it just seems silly to even try IMHO.)

The annoying thing is that even in my frustration, I have reduced Rosetta to:
10 resource share (or 6.99%, of 4 projects)
50% CPu's (of 16 cores)
50% cpu time
Store at least 0.1 days of work
Store up to an additional 0.1 days of work
Use at most 1GB HD (this is not respected at all)

50% CPU time should be 100%, Having it set at 50% means it will take all your projects twice as long to process a Task than it would with it set to 100%.
Store up to an additional 0.1 days of work would be better still as 0.01.
As for the resource share- Any new project you start will always result in the new project getting more time than the other projects -even if it has lower Resource share setting- because it has no past processing history. It has no Credit at all. And the you juggle things, the longer it takes to settle down. And if it considers a projects is owed a certain amount, fiddling to try to get more to the other project just takes it even longer for it to settle down. The smaller the cache, the sooner the projects will sort themselves out. Things like "50% cpu time" and pausing computation while the system is in use or there is non-BOINC CPU usage just means it takes longer for those things to settle down & balance out.

and after a project reset it downloaded 1 task (at the time I looked) and now has zero running tasks and occupying 1.67GB of storage (~80% of all projects usage), asking for more ("Rosetta needs 427.35MB more disk space. You currently have 1289.27 MB available and it needs 1716.61 MB. 02/05/2020 20:08:22") - ultiumately, the requested storage has not really reduced, and it is storing 1.67GB even when no Rosetta tasks are running)

Rosetta recently released a new application that requires less disk space to run. Rosetta now requires up 2.5GB in total- before it required just over 1GB for each Task being run. That is a massive reduction, particularly for mutli-core/thread systems. My dIsk usage has dropped from 13GB to just over 2GB.
It still needs disk space when no Tasks are running- just like very other single BOINC project. It saves you having to re-download all the necessary files every time you start a new Task, they are already there. You only need the new Data files to process and don't have to re-download the applications & database files over & over & over & over & over again each time it starts processing work for that project again.

Do you know what the exact relationship/calculation between cpus --> tasks--> HDD space usage? (especially based upon my reported settings and number of cores)
It doesn't make sense to me to have to consider technical specifics like cores vs cache when all that is needed is the software takes the target ("use at most n GB storage") and then adjusts its variables to suit. This project is a major step backward in regards to user interface design and usability! (this comment is not for you, of course!)

You are confusing BOINC & Rosetta.
Rosetta & all the other projects make use of BOINC, and each project has different needs for CPU, storage, RAM etc. The default settings for BOINC won't work with all possible combinations of projects, but they do work for most.

I'm getting to the point that deleting the project is the only way I can avoid this as it impacts my other project (e.g. climate predictions never runs and WCG and Seti often lack space due to what Rosetta takes).

Give Rosetta 2.5GB + WCG as much space as it needs, Seti is finished so it's not an issue anymore. So give BOINC that much storage space and things will work.
That's all you need to do.


Set BOINC so it has enough room for both WCG & Rosetta. Make sure it's 100% CPU time (and having 100% CPUs as well would also help things settle down sooner). Set the smallest cache possible (0.1 & 0.001 should be good enough), and then just let it do it's thing for a week or so. Then tweak things if necessary (very slightly), and give it another week or so.

---

Grant
Darwin NT