Message boards :
News :
OneChunk CPU Pre-Announcement (X86_64 & ARM64)
Message board moderation
Author | Message |
---|---|
Send message Joined: 15 Jun 20 Posts: 74 Credit: 19,537,761 RAC: 0 |
Over on the Minecraft@Home discord, Boysanic ("Boincboi"), Matt (Stronghold simulation coding), other developers and I have been discussing and planning a rollout of our first CPU application. This OneChunk project is "a BOINC project in the works. Our goal is to find all world seeds w/ at least an 11 eye portal in MC Java 1.7.10". So, if this all works out, "you'll finally be able to say you've found an 11 eye portal by yourself!" Essentially, we are combining parts of the "SSG" (Seed-Speedrun) and 11-eye portal seed-filtering programs together to go through a large dataset. This input file is 9GB in size, but we plan to split it up into many small manageable tasks to be distributed through BOINC, which is where you come in! Going through a 320kb sample input takes a 4.8GHz Ryzen 5800 (the fastest single-core CPU) around 2 minutes, simulating and checking over 5,220 worldseeds a second. A 2GHz Raspberry Pi 4, with 64bit Raspbian manages to compute 1,370 worldseeds a second, making it around 3.8x slower but much more efficient. (45W vs 5W) Thanks to work by other developers, I have tested and confirmed that both x86_64 ("64bit Windows and Linux") and ARM64 (64bit Linux ARMv8) configurations will work. The OneChunk 'Filter9000' BOINC application is single-threaded, but will likely have multiple tasks running concurrently on your computer to best utilize its resources and scale to your unique hardware configurations. For 'PC' requirements, we suggest Ubuntu 18.04/20.04 64bit or Windows 7 / 10 64bit, running on an CPU such as an AMD FX or Intel 1st Gen Core i5/i7 CPU or newer. If anyone has a Raspberry Pi 3(B/B+) or 4, we would like to invite you to install the latest 64bit Raspbian on a spare SDcard in preparation. We also expect Nvidia Jetson Nano/TX1/Xavier devices with Ubuntu 18.04 64bit or newer to successfully run our Arm64 tasks, once we configure our BOINC server to provide and schedule tasks for this platform. Please follow instructions on the Raspberry Pi website, but we suggest that more experienced Pi users join us for this experiment! OS image below: https://downloads.raspberrypi.org/raspios_arm64/images/raspios_arm64-2021-04-09/2021-03-04-raspios-buster-arm64.zip Also, check out our Forums for upcoming updates, and to see the first 25-Tall Superflat cactus! |
Send message Joined: 24 Jun 20 Posts: 25 Credit: 448,784,541 RAC: 152,294 |
|
Send message Joined: 25 Jun 20 Posts: 6 Credit: 61,664,191 RAC: 18,170 |
I have 5x RP4s attached and ready to go. I have 1x RP4s attached and ready to go. :-) |
Send message Joined: 8 Mar 21 Posts: 53 Credit: 245,502,973 RAC: 975 |
I will be converting over my RPi 3B+ to ARM64 Buster. Card already created and just waiting for the cpu application. |
Send message Joined: 12 Apr 21 Posts: 3 Credit: 22,630,281 RAC: 0 |
I also have a RP4 - what do I need to do? I have installed the OS what do I also need to install Boinc? or what else do I need to do? |
Send message Joined: 15 Jun 20 Posts: 74 Credit: 19,537,761 RAC: 0 |
On Windows 10 (for example), you can use Raspberry Pi Image Writer to write the posted Raspbian 64bit OS to a fresh SD card, and then set up your system as usual (plus connect it to the internet). Make sure you have some sort of cooling on your Pi 4 though, like a heatsink or fan, especially if you are overclocking. Then, you can run " sudo apt-get update && sudo apt-get install boinc boinc-client boinc-manager " to install BOINC, and then you can launch it through either the terminal (boincmgr) or the menu. It is advised though that you have a screen attached to use BOINC via the GUI, but more experienced BOINC/Linux users will be able to use: https://minecraftathome.com/minecrafthome/weak_auth.php boinc --daemon --attach_project "minecraftathome.com/minecrafthome/" "weak_key_here" boinccmd --project minecraftathome.com/minecrafthome/ update boinccmd --get_tasks boinccmd --network_available to connect their screen-free (like SSH) pi to begin crunching when we release the application :) (Also, code updates can be tracked here: https://github.com/minecrafthome/OneChunk but we have like a dozen things on our checklist to do before its running) |
Send message Joined: 12 Apr 21 Posts: 3 Credit: 22,630,281 RAC: 0 |
thanks I was able to get everything installed, BOINC and waiting for new tasks! Thanks everyone |
Send message Joined: 15 Jun 20 Posts: 74 Credit: 19,537,761 RAC: 0 |
Small update, we've decided roughly on the input work and are slowly making pieces of the BOINC template files, and task estimates. Keep in mind, here we are using "BOINC Benchmark Whetstone MIPS = OPS" These break down to: 5,800,000,000 OP/sec per Ryzen 5800X Core Whetstone (what credit is based off of) 12.5 seconds per seed/input line (ie, 10 seeds computed with our current program takes 120-125 seconds on a single 5800X 4.85ghz core) So, 72,500,000,000 "OPS per seed input line per core" on Zen 3 Thus: 21,750,000,000,000 OPS (2.175e^13) (300 lines of input seeds) of computation done per core, per BOINC task. If everything works as expected, then CPU task time estimates should be close to reality, being 60-65mins on a Ryzen 5000 CPU, and say, 90-120mins on a Ryzen 1000 CPU due to IPC and frequency differences. Also for fun stats, a 3.2GHz Broadwell laptop core is 2.2X slower (271 seconds for 10 seeds) versus my 5800X |
Send message Joined: 31 Jan 21 Posts: 1 Credit: 8,799 RAC: 0 |
i will try it on Rpi 3B+ SD ready :) |
Send message Joined: 18 Jul 20 Posts: 1 Credit: 21,988,398 RAC: 0 |
hi 1X pi4 8gig ready, il set up the other 10 at some point, i's a bit of a pain imgeing all them ssd's :P |
Send message Joined: 15 Jun 20 Posts: 74 Credit: 19,537,761 RAC: 0 |
Thanks for the signs and support and ARM64 hardware being prepared to contribute to this project! We've currently got 4 projects (Kaktwoos-cl update, OneChunk, "new seed video" and Trailer-seed) in progress, but I will keep you updated as we work to close out some of them, and finalize the OneChunk CPU tasks with later testing. |
Send message Joined: 15 Jun 20 Posts: 74 Credit: 19,537,761 RAC: 0 |
One of our coders, Matt, is working on some optimizations for OneChunk while we wait for our BOINC-server side code to be finished. So far, he has given my Ryzen 5800X a 12% performance improvement (110 seconds for 10 input seeds, versus 124-125 before). My Pi 4 (doing two seeds with dungeons and lots of output) went from taking 95 seconds to 88 seconds, or an 8% speed improvement. |
Send message Joined: 12 Apr 21 Posts: 3 Credit: 22,630,281 RAC: 0 |
Are we expecting work for the Pi's yet? or is that still coming soon? |
Send message Joined: 15 Jun 20 Posts: 74 Credit: 19,537,761 RAC: 0 |
Small update, for now. Sadly things are still in progress as our team's work is divided between multiple projects and IRL (school/work) in-between us and them. I personally have been studying for my next IT certification over the past few weeks, and occasionally have been speaking to the MC@Home team to check in and provide reminders on a few things, and also to post here for updates Code hasn't been changed for OneChunk/Filter9000 since my previous announcements, and other members are providing input and confirmation on the exact plan on how to generate work and manage the input+output files to keep track of things. The main delay is the work towards our 1 year anniversary of BOINC and Minecraft@Home as its own group as we know it currently. We are preparing scripts, videos and other material for a celebration in the upcoming weeks. A good amount of that is reviewing previous work and projects for any new insights we can make since our discovery of Pack.png last summer. Please check the main page of our site to join us on Discord for any news on it, and to participate with us and the hundreds of users during the celebration! For updates to Kaktwoos (v2.13), we are waiting for final approvals and merging before sending it to the 'infamous' testbed, which we suggest that you do not get too attached to if you happen to stumble upon it like other BOINCers have. Additionally, another long-term CPU project (terrain checking) is being theorized, involving a very frustrating "Shot T" which contains nothing but a few grass and dirt blocks, in a pit of some sort. Supposedly, it may take a year or more to find Shot T with 400+ "Core i5" level processors, assuming the code is functional and ported to BOINC after our developers can do as many optimizations as they can. If it indeed takes a long time after we eventually get it onto BOINC, then we may begin cracking "Custom World" example seeds which are another major interest to us. Various shots for the Trailer seedfinding and eventual recreation project are here: https://imgur.com/a/ozHEde3 We also recently found the Youtuber 'Seannaners' original Let's Play seed, and I am happy that we did so. Especially after our work on both the recreation and the dungeons, which lead to a relatively quick find after we temporarily paused the project in December. |
Send message Joined: 15 Jun 20 Posts: 74 Credit: 19,537,761 RAC: 0 |
Hello all! So, remember when I said we had some issues with the work generation, validation, and the immense amount of tasks we expect to store? Well, we've made progress towards making OneChunk more manageable and actually practical on our infrastructure. Some of our admins have recently collaborated with 'Sia', a platform for distributed cloud storage to test out alternative workgen data-input methods. We thought, our BOINC' network is distributed, so why not use another distributed service to solve some problems we kept worrying about? Boysanic (Boincboi), Matt, I and others on our Discord are discussing some work that has been done the past few days. Boysanic is testing out a basic, but good method of converting our pure-text 9GB seed input file into a 3.5GB binary input file, which we plan to be put on Sia. They have also made adjustments to a validator to easily read our changed data and ensure it compares to the original between conversions. This compression ratio could also apply to the seeds your clients output (or for outputs of future projects with similar data created), but we expect our 1.5 million tasks should not take up much space on your devices, or our servers once this project is officially launched. Essentially, while generating work (gradually) we'll stream in the data for seeds directly using Sia based file with an offset, bypassing the need to thrash our I/O and RAM storage. It's basically free cost-wise as our file isn't too large, and the many arguments over the method of 'dynamic workgen' can be put to rest I figure. This may be a week or two, but I am happy to help test and assist with the last touches coding-wise on our BOINC project. Mainly to enable CPU-tasks and add the new platforms (ARM64, ie Raspberry Pi) I mentioned in the original post. |
Send message Joined: 25 Apr 21 Posts: 1 Credit: 23,463,520 RAC: 0 |
lol, as you mentioned Sia, i thought this is one of the most stupid things i've heard for long. luckily i'm a dumbass and scrambled it with chia. because of this review, i got gasping https://www.youtube.com/watch?v=FtXkhIVZBmw |
Send message Joined: 27 Jun 20 Posts: 4 Credit: 1,502,630 RAC: 739 |
So far so good. Can you push a android version of the app ? got many android devices on the start. |
Send message Joined: 26 Jun 20 Posts: 3 Credit: 95,445,698 RAC: 0 |
4x Rpi4 64 ready to go! Maybe + 20x Rpi3b+ ready next month. |
Send message Joined: 27 Jun 20 Posts: 4 Credit: 1,502,630 RAC: 739 |
4x Rpi4 64 ready to go! Maybe + 20x Rpi3b+ ready next month. I checked this before i bought them. An AMD onboard chip mainboard got more compute/energy efficiency than 1 rpi. Even the endurance is higher. Check it out. https://browser.geekbench.com/v5/cpu/compare/9179618?baseline=9313591 If u already own a PSU and a hdd, there will be need only 8 gb ram + cpu + Board. For cheap must be real. https://www.funkykit.com/reviews/amd-ryzen-3-3200g-performance-analysis-on-b450-chipset/ |
Send message Joined: 26 Jun 20 Posts: 3 Credit: 95,445,698 RAC: 0 |
Yes you're right, but i already have them lying around. Had them working before with 32 bit Raspian for a few years. Now i want to update all sd cards to 64 bit raspberry os and get them mounted in a case that i have build. 80% done. Thanks for your advice ;-) |