BOINC seems to not free up used storage space

Message boards : Number crunching : BOINC seems to not free up used storage space

To post messages, you must log in.

AuthorMessage
mobluse

Send message
Joined: 5 Apr 20
Posts: 6
Credit: 72,232
RAC: 18
Message 95946 - Posted: 3 May 2020, 19:43:53 UTC
Last modified: 3 May 2020, 20:09:52 UTC

It seems BOINC doesn't release used storage space when it finishes. I've limited storage to 2 GB, but BOINC holds 1.54 GB even though no tasks are running. How come? How to free? Now it can't download new tasks. I only run RosettaAtHome/Rosetta@home.


I use Raspberry Pi 4 B 4 GB RAM with Raspbian Buster 32-bit and run BOINC in a container with Debian Buster 64-bit.
ID: 95946 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 95949 - Posted: 3 May 2020, 20:25:23 UTC - in response to Message 95946.  

BOINC Manager holds on to project files that will be needed for future work, such as the main executable program, and graphics program. If you intend to do more work with Rosetta, you really want those to stick around, and save all of the download time in the future. If you do not intend to do more R@h work, then detaching from the project is the way to free up all storage used by the project.
Rosetta Moderator: Mod.Sense
ID: 95949 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mobluse

Send message
Joined: 5 Apr 20
Posts: 6
Credit: 72,232
RAC: 18
Message 95953 - Posted: 3 May 2020, 21:11:12 UTC - in response to Message 95949.  

But do those executable programs take up 1.54 GB? That seems a bit much.

Now I succeeded in getting Rosetta to continue by resetting the project. That freed up all storage, and then it downloaded data and continued running tasks. Now it also uses 1.54 GB as before even though it has data files for two tasks.
ID: 95953 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1679
Credit: 17,819,447
RAC: 22,700
Message 95962 - Posted: 3 May 2020, 23:05:23 UTC - in response to Message 95953.  

But do those executable programs take up 1.54 GB? That seems a bit much.
No, but the data files do.
If there isn't enough room reserved by the project, then it can't process the work, and sends out a message saying it needs more work.
Then people complain about being told they need to make more room, why weren't they told that before? So they make mere room. then they might need a bit more room, and complain they weren't told that they might need more room.
Different projects require different amounts of storage space. Give Rosetta 2.5GB & it should be OK with the new v4.20 applications (unless you get a Rosetta Mini task- then it will need another GB or 2 for each RM Task).
Grant
Darwin NT
ID: 95962 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Admin
Project administrator

Send message
Joined: 1 Jul 05
Posts: 4805
Credit: 0
RAC: 0
Message 95964 - Posted: 3 May 2020, 23:15:35 UTC

The rosetta application now saves the extracted database in the project directory and this does not get removed even if there are no jobs to run.
ID: 95964 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
MarkJ

Send message
Joined: 28 Mar 20
Posts: 72
Credit: 25,238,680
RAC: 0
Message 95991 - Posted: 4 May 2020, 6:57:14 UTC - in response to Message 95946.  

I use Raspberry Pi 4 B 4 GB RAM with Raspbian Buster 32-bit and run BOINC in a container with Debian Buster 64-bit.

A bit off topic but you know you can tell Raspbian to run as aarch64 and BOINC to request work for the aarch64 platform? It might save you using a container when you don't need to. See the Running on a 4GB Raspberry Pi 4 thread.
BOINC blog
ID: 95991 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mobluse

Send message
Joined: 5 Apr 20
Posts: 6
Credit: 72,232
RAC: 18
Message 96004 - Posted: 4 May 2020, 11:19:07 UTC - in response to Message 95962.  

After it had finished the last two tasks, it stopped again with no tasks and complained about too little disk space. If I reset or remove the project and add it again the disk space is freed and it works again. Why can't Rosetta@home free the disk space of 1.54 GB automatically if it needs this to continue? To have old data after the tasks have finished seems useless. Now I've increased disk space for BOINC to 3 GB from 2 GB.
ID: 96004 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 96017 - Posted: 4 May 2020, 14:04:19 UTC - in response to Message 96004.  

As I said previously, it is not "old data", it is stuff that is needed to run the next work units that you are assigned.

You have stepped in to an application version change here that dramatically reduces the disk space requirement of the project. So it is hard to say exactly what combination of things you may have observed. Your system probably already had the prior version of the application downloaded, and needed to bring down the new application version. Which, if it had enough disk space, it would have achieved on its own.
Rosetta Moderator: Mod.Sense
ID: 96017 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2123
Credit: 41,204,457
RAC: 10,266
Message 96066 - Posted: 4 May 2020, 19:48:58 UTC - in response to Message 96004.  

After it had finished the last two tasks, it stopped again with no tasks and complained about too little disk space. If I reset or remove the project and add it again the disk space is freed and it works again. Why can't Rosetta@home free the disk space of 1.54 GB automatically if it needs this to continue? To have old data after the tasks have finished seems useless. Now I've increased disk space for BOINC to 3 GB from 2 GB.

When I first started, BOINC defaulted to requiring 20Gb disk space (yes, really). But now it's a lot less.
Try setting disk space to 5Gb if you can spare it.
ID: 96066 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1679
Credit: 17,819,447
RAC: 22,700
Message 96068 - Posted: 4 May 2020, 20:15:59 UTC - in response to Message 96066.  

Why can't Rosetta@home free the disk space of 1.54 GB automatically if it needs this to continue?
Because then people would complain about it using more disk space than they allowed it to use.
Grant
Darwin NT
ID: 96068 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mobluse

Send message
Joined: 5 Apr 20
Posts: 6
Credit: 72,232
RAC: 18
Message 96104 - Posted: 5 May 2020, 11:31:17 UTC - in response to Message 96068.  

No, they wouldn't because more disk space is not needed. When I manually reset I don't use more disk space than the allotted. BOINC should be able to do automatically what I do manually to get it to continue with Rosetta@home.
ID: 96104 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Bryn Mawr

Send message
Joined: 26 Dec 18
Posts: 392
Credit: 12,101,016
RAC: 5,587
Message 96114 - Posted: 5 May 2020, 15:47:34 UTC - in response to Message 96104.  

No, they wouldn't because more disk space is not needed. When I manually reset I don't use more disk space than the allotted. BOINC should be able to do automatically what I do manually to get it to continue with Rosetta@home.


But we are in a non-typical position whilst Rosetta is transitioning from one form of storage to another. Automating a manual system that works now will probably be unproductive in a couple of weeks time.

Also, deleting all of the data files as soon as tasks have finished could also be counter-productive as most of them will probably have to be reloaded when the next task comes down.
ID: 96114 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1679
Credit: 17,819,447
RAC: 22,700
Message 96127 - Posted: 5 May 2020, 18:48:39 UTC - in response to Message 96104.  
Last modified: 5 May 2020, 18:49:28 UTC

No, they wouldn't because more disk space is not needed.
Yes they would, because they have because more disk space is needed again.

If more disk space wasn't needed you wouldn't get the message saying it was.
If you set BOINC so that is has enough disk space for all of your projects, you won't get messages asking for more disk space for a particular project to run. As Mod.Sense posted- there is no point in cleaning out all the files after Rosetta has done some wok, as it will just have to download many of the files all over again to do the Task. It gets rid of the files it doesn't need anymore, but keeps the ones it does need. Why waste bandwidth?
Grant
Darwin NT
ID: 96127 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mobluse

Send message
Joined: 5 Apr 20
Posts: 6
Credit: 72,232
RAC: 18
Message 96235 - Posted: 7 May 2020, 14:44:41 UTC - in response to Message 96114.  

I now have set disk space to 4 GB and then it doesn't stop. BOINC mostly uses only 1.55 GB, but needs much more, perhaps 3.5 GB, when downloading a new task. The only project I run on this machine is Rosetta.
ID: 96235 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : BOINC seems to not free up used storage space



©2024 University of Washington
https://www.bakerlab.org