Tasks reset themselves after BOINC client restarts

Questions and Answers : Preferences : Tasks reset themselves after BOINC client restarts

To post messages, you must log in.

AuthorMessage
KY

Send message
Joined: 23 May 22
Posts: 2
Credit: 442,719
RAC: 3,588
Message 109261 - Posted: 20 May 2024, 10:23:59 UTC

i'm observing that the progress on the Rosetta tasks always resets when i restart the BOINC client. checking if there's a solution for this because i'm losing progress every day since i don't turn on my computer 24/7. thanks for any help/advice!
ID: 109261 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1681
Credit: 17,854,150
RAC: 22,647
Message 109264 - Posted: 21 May 2024, 6:17:56 UTC

In your account, under Computing preferences, under Other, make sure "Request tasks to checkpoint at most every xx seconds" is set to 60.
There are some Tasks where the check pointing doesn't always work, but most of the current Tasks are checkpointing & resuming without issue for me.


But there is something seriously, horrendously wrong with your system (which is probably impacting on the ability to checkpoint).

The very few Tasks you have completed, it took your system 17.5 hours to do only 3 hour's worth of actual work. WTF?????? You've got similar issues with your longer running Einstein Tasks
Run time (sec):140,064.26
CPU time (sec): 21,944.39
39 min to do 6 min worth of work.


Are you running Foldling@home, playing games & transcoding video all at the same time as doing BOINC work???
You need to look in Windows Task Manager and see just what it is that's using up all of your CPU time. Sort that out and not only will you be able to do 5 or more times the amount of work you are presently doing, but the checkpointing will probably also start working as well.
Grant
Darwin NT
ID: 109264 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
KY

Send message
Joined: 23 May 22
Posts: 2
Credit: 442,719
RAC: 3,588
Message 109265 - Posted: 21 May 2024, 6:35:33 UTC - in response to Message 109264.  

hmm. that's curious cos i'm not doing anything on my computer most of the time. most of the CPU time are taken up by the tasks. and i only have problems with rosetta checkpoints, the other projects are working OK, as in they resume where they left off.
ID: 109265 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1681
Credit: 17,854,150
RAC: 22,647
Message 109266 - Posted: 21 May 2024, 9:57:57 UTC - in response to Message 109265.  
Last modified: 21 May 2024, 10:06:55 UTC

hmm. that's curious cos i'm not doing anything on my computer most of the time. most of the CPU time are taken up by the tasks.
That's the thing- most of your CPU time is not being used by BOINC- BOINC is only using about 1/5th (or less).


and i only have problems with rosetta checkpoints, the other projects are working OK, as in they resume where they left off.
Those have much shorter runtimes, so the checkpoint intervals are most likely much more frequent. While some Rosetta Tasks do have checkpointing issues, i've yet to have an issue with any of the 3 hour Tasks.

Either way- i'd suggest fixing up whatever is using most of your CPU, then see if that fixes the checkpointing issue.
There is something on that computer using a massive amount of CPU time. And i mean massive.

This is a Task from my computer that i use every day- mostly just email, web browsing & the occasional video.
Run time 3 hours 3 min 58 sec
CPU time 3 hours 0 min 6 sec
4 min difference.

This is one of your Tasks
Run time 17 hours 26 min 16 sec
CPU time  3 hours 0 min 1 sec
14.5 hrs difference!!!

CPU time is the amount of time the CPU spent processing the Task. Run time is the time it took from when it started, to when it finished in actual time (ie a clock on the wall time).
A few minutes is one thing, a half hour or so difference for a very heavily used system, is another- but 5 times as long as your system is taking, that's just insane.


Do you use web settings or local settings? Either way- Computing preferences should be set to something along these lines.
Usage limits	
Use at most 100 % of the CPUs
Use at most 100 % of CPU time

When to suspend	
Suspend when computer is on battery           leave blank
Suspend when computer is in use               leave blank
Suspend GPU computing when computer is in use leave blank
'In use' means mouse/keyboard input in last   3 minutes
Suspend when no mouse/keyboard input in last   --- minutes
Suspend when non-BOINC CPU usage is above      --- %
Compute only between                           ---

Other	
Store at least                            0.1 days of work
Store up to an additional                 0.01 days of work
Switch between tasks every                60 minutes
Request tasks to checkpoint at most every 60 seconds

Disk
Use no more than 20 GB
Leave at least    2 GB free
Use no more than 60 % of total

Memory
When computer is in use, use at most     95 %
When computer is not in use, use at most 95 %


NB- If "Use at most xxx % of CPU time" is set to anything other than 100%, then you'll get a large difference between CPU time & Run time. The smaller the percentage, the bigger the gap.
You may need to set "Use at most 100 % of the CPUs" to something less than 100% to avoid running out of memory- Some Rosetta Tasks can use as much as 2.5GB of RAM. Get a batch of those & your 20 core/thread system with only 32GB of RAM will run in to trouble very quickly, especially if your "When computer is in use, use at most 95 % is set to less than that.
Also worth making sure "Leave non-GPU tasks in memory while suspended" is not selected.
Grant
Darwin NT
ID: 109266 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1681
Credit: 17,854,150
RAC: 22,647
Message 109267 - Posted: 22 May 2024, 6:51:55 UTC

And your most recently completed Task was even worse than the previous worst.
Run time 19 hours 40 min 22 sec
CPU time  2 hours 59 min 49 sec



I also notice that most of your Einstein Tasks are GPU ones. While the GPU does most of the work, a quick look at the forums shows that for each GPU Task being processed, you need 1 (or more, depending on the Task type being processed) CPU cores/threads to support it.
You'd need to consult with the people on the Einstein forums what app_config file to use (and it goes in the Einstein project data folder, and you have to Options, "Read local preferences file" for it to take effect (or exit BOINC & restart).
It would look something like this (with more entries for each GPU application being run)-
<app_config>
   <app>
      <name>hsgamma_FGRPB1G</name>
      <gpu_versions>
         <gpu_usage>.5</gpu_usage>
         <cpu_usage>1</cpu_usage>
      </gpu_versions>
   </app>
   <app>
      <name>einsteinbinary_BRP4G</name>
      <gpu_versions>
         <gpu_usage>0.5</gpu_usage>
         <cpu_usage>1.0</cpu_usage>
      </gpu_versions>
   </app>
</app_config>



Having said that, while not having one or more CPU cores reserved to support your GPU Tasks will be impacting on both your CPU & GPU output, it alone still doesn't come close to explaining the severity of the issue your system is displaying.
Grant
Darwin NT
ID: 109267 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Questions and Answers : Preferences : Tasks reset themselves after BOINC client restarts



©2024 University of Washington
https://www.bakerlab.org