Multi-GPU Support

Message boards : Number crunching : Multi-GPU Support
Message board moderation

To post messages, you must log in.

AuthorMessage
Penguin

Send message
Joined: 24 Jun 20
Posts: 5
Credit: 13,163,758
RAC: 0
Message 67 - Posted: 28 Jun 2020, 0:14:44 UTC

Has multi-GPU support been corrected? Or any ETA on when it will be functional. Really not a fan of using a gpu_exclusion..
ID: 67 · Report as offensive
Profile chip
Project administrator

Send message
Joined: 14 Jun 20
Posts: 78
Credit: 1,321,619
RAC: 0
Message 71 - Posted: 28 Jun 2020, 0:52:01 UTC - in response to Message 67.  

Has multi-GPU support been corrected? Or any ETA on when it will be functional. Really not a fan of using a gpu_exclusion..


Yes, there's a working branch in one of our developer's forks of the kaktwoos-cl app: https://github.com/Hyenadae/kaktwoos-cl/tree/boinc-checkp
It'll likely be merged sometime between this evening and tomorrow afternoon to be tagged as a new release and tested before shipping.
If all goes well, we should have native multi-GPU support by tomorrow evening (UTC)
ID: 71 · Report as offensive
Penguin

Send message
Joined: 24 Jun 20
Posts: 5
Credit: 13,163,758
RAC: 0
Message 72 - Posted: 28 Jun 2020, 0:58:32 UTC - in response to Message 71.  

Has multi-GPU support been corrected? Or any ETA on when it will be functional. Really not a fan of using a gpu_exclusion..


Yes, there's a working branch in one of our developer's forks of the kaktwoos-cl app: https://github.com/Hyenadae/kaktwoos-cl/tree/boinc-checkp
It'll likely be merged sometime between this evening and tomorrow afternoon to be tagged as a new release and tested before shipping.
If all goes well, we should have native multi-GPU support by tomorrow evening (UTC)


Fantastic news! Thank you for your rapid response! I'm sure many others will be equally happy.
ID: 72 · Report as offensive
Profile Hy
Project administrator
Project developer
Avatar

Send message
Joined: 15 Jun 20
Posts: 74
Credit: 19,537,761
RAC: 0
Message 74 - Posted: 28 Jun 2020, 3:04:36 UTC
Last modified: 28 Jun 2020, 3:07:15 UTC

Hi (Hy) here.

Yes, multiGPU is being worked on and we need to verify it working on various vendors + gpus before being merged.

We will be doing some backend (wrapper) changes as well so the "--device $GPU_DEVICE_NUM" argument is passed,
meaning whatever BOINC wants it to run on, it should run on.

I've also worked on adding checkpointing functionality to kaktwoos-cl so jobs can be suspended and resumed, and still return to a near-recent position while keeping all found seeds' details (cacti height). The seeds themselves are properly being found and passed to the end of the program where they would be printed/outputted as normal.

I also need to verify that data being passed to and from the multi-gpu enabled code is valid, and so that data from GPU #2 isn't being outputted to GPU #1 because of improperly # read/write buffers.

We have confirmed in the standalone application that a GTX 780ti (GPU1) and 980ti (GPU2) have been used by Kaktwoos using just toggles on the --device # on execution, separately and by two instances of kaktwoos-cl at once.

I am hoping to test the multi-gpu+checkpoint enabled code tomorrow, but keep in mind that it is likely that checkpointing *and* multi-gpu may not be perfect.
ID: 74 · Report as offensive
Sergey Kovalchuk

Send message
Joined: 24 Jun 20
Posts: 26
Credit: 1,106,925
RAC: 0
Message 75 - Posted: 28 Jun 2020, 3:07:07 UTC

the project has application versions for three types of GPUs - nvidia, amd and intel.
but in preferences you cannot allow GPU intel

the scheduler’s answer does not mention intel_gpu at all
this platform is not fully implemented in project configs
ID: 75 · Report as offensive
Profile Hy
Project administrator
Project developer
Avatar

Send message
Joined: 15 Jun 20
Posts: 74
Credit: 19,537,761
RAC: 0
Message 76 - Posted: 28 Jun 2020, 3:14:16 UTC - in response to Message 75.  
Last modified: 28 Jun 2020, 3:14:44 UTC

Hi Sergei,

I worked on Intel vendor support (actually my first compile test for Linux) and we *do* specify intel_gpu in the scheduler, yet it isn't being recognized or used by BOINC yet
Strangely enough it is very difficult to find documentation on projects that have Intel GPUs. enabled.

I can confirm the executables themselves work on Intel Opencl (Beignet) Linux and Windows. Despite the additional to plan_class_spec for the past week (that enabled AMD support), my intel laptop is not being given jobs- even if it is detected as intel_gpu Intel OpenCL.

See: https://github.com/minecrafthome/minecrafthome/blob/master/images/makeproject/plan_class_spec.xml


opencl_intel
intel_gpu



I'm still looking for solutions and suggestions as well. There's very few places left to add intel_gpu or find the exact code that makes the scheduler hand off work
ID: 76 · Report as offensive
Sergey Kovalchuk

Send message
Joined: 24 Jun 20
Posts: 26
Credit: 1,106,925
RAC: 0
Message 77 - Posted: 28 Jun 2020, 3:27:03 UTC - in response to Message 76.  
Last modified: 28 Jun 2020, 3:27:43 UTC

https://minecraftathome.com/minecrafthome/prefs.php?subset=project

the user has no option to allow the use of GPU intel.
check which configuration variable changes the option selection.
it should be in the definitions of coprocessors

then the client will receive from the scheduler <no_intel_gpu> 0 </no_intel_gpu> and will request jobs.
ID: 77 · Report as offensive
zombie67 [MM]
Avatar

Send message
Joined: 24 Jun 20
Posts: 21
Credit: 181,142,041
RAC: 0
Message 78 - Posted: 28 Jun 2020, 3:29:07 UTC

FWIWm einstein@home supports intel GPUs. Perhaps ask them?

https://einsteinathome.org/apps.php
Reno, NV
Team: SETI.USA
ID: 78 · Report as offensive
Sergey Kovalchuk

Send message
Joined: 24 Jun 20
Posts: 26
Credit: 1,106,925
RAC: 0
Message 80 - Posted: 28 Jun 2020, 3:44:05 UTC - in response to Message 78.  

Einstein greatly redone the interface

but you can ask the admins Collatz and SETI (until these ran away after closing) ;o)
ID: 80 · Report as offensive
Jord
Volunteer moderator
Help desk expert
Avatar

Send message
Joined: 24 Jun 20
Posts: 85
Credit: 207,156
RAC: 0
Message 81 - Posted: 28 Jun 2020, 4:12:45 UTC - in response to Message 76.  

[quote]
    <plan_class>
        <name>opencl_intel</name>
        <gpu_type>intel_gpu</gpu_type>
        <opencl/>
    </plan_class>
<gpu_type> is just the vendor, you don't have to add that it's a GPU as the tag says that already.
So:
    <plan_class>
        <name>opencl_intel</name>
        <gpu_type>intel</gpu_type>
        <opencl/>
    </plan_class>
Ref: https://boinc.berkeley.edu/trac/wiki/AppPlanSpec#GPUapps

Also, your AMD and Nvidia OpenCL plan classes and apps may be named wrong. See https://boinc.berkeley.edu/trac/wiki/AppPlan#Predefinedplanclasses, opencl_nvidia_101 and opencl_amd_101

When in doubt, you can email the BOINC Projects email list, then other project developers will answer: https://groups.google.com/a/ssl.berkeley.edu/forum/#!forum/boinc_projects
ID: 81 · Report as offensive
Profile Hy
Project administrator
Project developer
Avatar

Send message
Joined: 15 Jun 20
Posts: 74
Credit: 19,537,761
RAC: 0
Message 83 - Posted: 28 Jun 2020, 4:44:42 UTC - in response to Message 81.  
Last modified: 28 Jun 2020, 4:52:28 UTC

We originally had intel and that didn't seem to work. I went through the scheduler code and found it only refers to intel_gpu, ati(=amd) and nvidia

https://github.com/minecrafthome/minecrafthome/commit/fa5e96978202c9e9c9d667c584aa80527fd1cad2#diff-86aca54528af0a0a13e54cf6dda227c9 < Our change.

I've found some webserver settings that the scheduler may be looking for, it's not as simple to enable intel_gpus as amd and Nvidia because of the apparent lack of demand and stock configs that support it.

https://gitlab.aei.uni-hannover.de/einsteinathome/boinc/commit/ccb147d4c4d66ad99e4360e8dbad4a499af0690d

I've requested for code from here to be added to our server if possible, and that could be one of the pieces missing because my Intel-only Linux laptop is giving me
intel_gpu when there clearly are. Even if it isn't, when we *do* get the scheduler acknowledging our config, then people can turn off work for their Intel GPUs like every other
ID: 83 · Report as offensive
Jord
Volunteer moderator
Help desk expert
Avatar

Send message
Joined: 24 Jun 20
Posts: 85
Credit: 207,156
RAC: 0
Message 85 - Posted: 28 Jun 2020, 5:40:22 UTC - in response to Message 83.  
Last modified: 28 Jun 2020, 5:43:54 UTC

Well you're right that it's intel_gpu, I see that in places in the source code as well now. Should've checked that previously.
What I do know about the Project preferences page is that the GPU selections are done via the plan classes, so if there's something wrong in the plan class or application name, that GPU will not show up. Which seems to be the case here.

https://github.com/BOINC/boinc/blob/master/html/inc/prefs_project.inc shows
if ($app_types->intel_gpu)
but your applications page shows the name for the Intel app is opencl_intel, not e.g. opencl_intel_gpu

Make sure your applications are named correctly everywhere.

(If you want to compare to a comprehensive list of apps, see Seti's app list)
ID: 85 · Report as offensive
Profile Hy
Project administrator
Project developer
Avatar

Send message
Joined: 15 Jun 20
Posts: 74
Credit: 19,537,761
RAC: 0
Message 101 - Posted: 29 Jun 2020, 4:58:56 UTC

ID: 101 · Report as offensive
Profile chip
Project administrator

Send message
Joined: 14 Jun 20
Posts: 78
Credit: 1,321,619
RAC: 0
Message 117 - Posted: 1 Jul 2020, 12:54:44 UTC

Support for checkpointing and multigpu has landed https://minecraftathome.com/minecrafthome/forum_thread.php?id=14&postid=116#116
ID: 117 · Report as offensive

Message boards : Number crunching : Multi-GPU Support