Rosetta only running 5 tasks on 12 core CPU, VM memory size too large perhaps?

Questions and Answers : Windows : Rosetta only running 5 tasks on 12 core CPU, VM memory size too large perhaps?

To post messages, you must log in.

AuthorMessage
Annihilannic

Send message
Joined: 13 Jun 22
Posts: 5
Credit: 790,949
RAC: 0
Message 106440 - Posted: 28 Jun 2022, 23:10:16 UTC

My system has an AMD Ryzen 9 5900X 12-Core Processor and 32GB RAM, however even with my computing preferences set to 100% CPU and 90% RAM, it only runs 5 tasks simultaneously, using about 12% of the available CPU resources.

I suspect this is a RAM limitation because I notice from Oracle VirtualBox that the "rosetta python projects 1.03" task VMs are configured with 6GB of base memory, and I presume it's no coincidence that 5 x 6 GB fits neatly into 32GB. But in reality the VirtualBox processes running those VMs are only using ~50MB each, so it seems to me the virtual machine image configured for these tasks is grossly overallocated memory-wise, thereby preventing rosetta from reaching the full potential of computing resources available.
ID: 106440 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1681
Credit: 17,854,150
RAC: 22,647
Message 106442 - Posted: 29 Jun 2022, 5:27:56 UTC - in response to Message 106440.  

.. the "rosetta python projects 1.03" task VMs are configured with 6GB of base memory, and I presume it's no coincidence that 5 x 6 GB fits neatly into 32GB. But in reality the VirtualBox processes running those VMs are only using ~50MB each
It has been commented on from when Python tasks were first released here.
Since nothing has been done since ten, the odds of something happening in the future are as good as non existent. So systems will need at least 7.5GB+ of available RAM per Task they want to run, regardless of how little they actually use when running.
Grant
Darwin NT
ID: 106442 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Annihilannic

Send message
Joined: 13 Jun 22
Posts: 5
Credit: 790,949
RAC: 0
Message 106443 - Posted: 29 Jun 2022, 8:23:00 UTC

Thanks... do the people running the project even monitor this forum, or are we shouting in a vacuum here?
ID: 106443 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1681
Credit: 17,854,150
RAC: 22,647
Message 106445 - Posted: 29 Jun 2022, 11:17:01 UTC - in response to Message 106443.  

Thanks... do the people running the project even monitor this forum
So infrequently as to be pretty much not at all.
Grant
Darwin NT
ID: 106445 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2125
Credit: 41,228,659
RAC: 10,982
Message 106453 - Posted: 1 Jul 2022, 1:24:01 UTC - in response to Message 106440.  

My system has an AMD Ryzen 9 5900X 12-Core Processor and 32GB RAM, however even with my computing preferences set to 100% CPU and 90% RAM, it only runs 5 tasks simultaneously, using about 12% of the available CPU resources.

I suspect this is a RAM limitation because I notice from Oracle VirtualBox that the "rosetta python projects 1.03" task VMs are configured with 6GB of base memory, and I presume it's no coincidence that 5 x 6 GB fits neatly into 32GB. But in reality the VirtualBox processes running those VMs are only using ~50MB each, so it seems to me the virtual machine image configured for these tasks is grossly overallocated memory-wise, thereby preventing rosetta from reaching the full potential of computing resources available.

Running a 5800X here with 32Gb RAM
If you increase the amount of memory allocated within Boinc to 90% or higher you can get up to 10 or even 11 tasks running at once if you want to push things to the limit.
I found stability issues if I tried too many at once, but 9 or 10 are much more manageable in my experience.
Experiment a little and see what you're comfortable with
ID: 106453 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Annihilannic

Send message
Joined: 13 Jun 22
Posts: 5
Credit: 790,949
RAC: 0
Message 106456 - Posted: 1 Jul 2022, 3:32:39 UTC - in response to Message 106453.  
Last modified: 1 Jul 2022, 3:32:51 UTC

If you increase the amount of memory allocated within Boinc to 90% or higher you can get up to 10 or even 11 tasks running at once if you want to push things to the limit.
I found stability issues if I tried too many at once, but 9 or 10 are much more manageable in my experience.
Experiment a little and see what you're comfortable with


I was going to reply saying I've already done that, but I had only checked the 'When computer is not in use, use at most' value, assuming that it would use this one when I had BOINC set to 'Run always'. When I increased 'When computer is in use, use at most' to match it, then I get 11 tasks like you. It's using about 35% of CPU capacity and only about 600MB of RAM... let's see how stable it is. I wouldn't really expect stability issues at those levels...
ID: 106456 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2125
Credit: 41,228,659
RAC: 10,982
Message 106458 - Posted: 1 Jul 2022, 7:15:34 UTC - in response to Message 106456.  

If you increase the amount of memory allocated within Boinc to 90% or higher you can get up to 10 or even 11 tasks running at once if you want to push things to the limit.
I found stability issues if I tried too many at once, but 9 or 10 are much more manageable in my experience.
Experiment a little and see what you're comfortable with

I was going to reply saying I've already done that, but I had only checked the 'When computer is not in use, use at most' value, assuming that it would use this one when I had BOINC set to 'Run always'. When I increased 'When computer is in use, use at most' to match it, then I get 11 tasks like you. It's using about 35% of CPU capacity and only about 600MB of RAM... let's see how stable it is. I wouldn't really expect stability issues at those levels...

I overclock and have general heat (and dust) issues, so the slightest thing sends me over the edge. Which is why I deliberately restrict RAM so only 10 tasks run at a time.
It helps me, so if you get any kind of issue over the next 24hrs it's a successful easy solution.

I forgot to mention, you were right to suspect RAM being the limitation - a hurdle that has to be jumped over when tasks start, but not while running.
Someone speculated that if 200% RAM could be allocated to Boinc, more tasks would run, but that doesn't pan out. (Yes, I tried...)
ID: 106458 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Annihilannic

Send message
Joined: 13 Jun 22
Posts: 5
Credit: 790,949
RAC: 0
Message 106459 - Posted: 1 Jul 2022, 7:36:40 UTC - in response to Message 106458.  

Someone speculated that if 200% RAM could be allocated to Boinc, more tasks would run, but that doesn't pan out. (Yes, I tried...)


I had a similar thought, but couldn't set it to a value over 100 through the GUI. Is there another way?
ID: 106459 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2125
Credit: 41,228,659
RAC: 10,982
Message 106463 - Posted: 2 Jul 2022, 2:39:45 UTC - in response to Message 106459.  

Someone speculated that if 200% RAM could be allocated to Boinc, more tasks would run, but that doesn't pan out. (Yes, I tried...)

I had a similar thought, but couldn't set it to a value over 100 through the GUI. Is there another way?

Not that anyone's found or revealed.
It was said partly in jest, but considering the RAM demand at task-launch compared to what's used when the task runs, there's a case for saying the project started it with their own comical suggestions
ID: 106463 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Questions and Answers : Windows : Rosetta only running 5 tasks on 12 core CPU, VM memory size too large perhaps?



©2024 University of Washington
https://www.bakerlab.org