House keeping

Questions and Answers : Web site : House keeping
Message board moderation

To post messages, you must log in.

AuthorMessage
Jord
Volunteer moderator
Help desk expert
Avatar

Send message
Joined: 24 Jun 20
Posts: 85
Credit: 207,156
RAC: 0
Message 115 - Posted: 30 Jun 2020, 15:24:41 UTC

- You can remove forums you don't need from /html/ops/create_forums.php and then rerun the script.
- You don't have any spammers or trolls yet, but eventually you will. So make people moderators. Preferably your own guys.
- If you don't want to use the forum rating, you can disable that. (set <no_forum_rating> in config.xml)
- In the future, when you don't want people without credit to post in the main forums, set that a minimum credit amount is required, or a minimum RAC. These people can then still post in the Help Desk forums.
- Adjust the description of the forums, now things are saying the default "BOINC".
ID: 115 · Report as offensive     Reply Quote
Profile chip
Project administrator

Send message
Joined: 14 Jun 20
Posts: 78
Credit: 1,321,619
RAC: 0
Message 130 - Posted: 1 Jul 2020, 20:25:51 UTC - in response to Message 115.  
Last modified: 3 Jul 2020, 18:15:59 UTC

- You can remove forums you don't need from /html/ops/create_forums.php and then rerun the script.
- You don't have any spammers or trolls yet, but eventually, you will. So make people moderators. Preferably your own guys.
- If you don't want to use the forum rating, you can disable that. (set in config.xml)
- In the future, when you don't want people without credit to post in the main forums, set that a minimum credit amount is required, or a minimum RAC. These people can then still post in the Help Desk forums.
- Adjust the description of the forums, now things are saying the default "BOINC".


- I was struggling with imagination when we bootstrapped these forums, so I'll consolidate them down and rethink the descriptions.
- I've set up a couple of our guys as moderators, I'll find a few more volunteers and encourage them to stay active. I know you're very active over on the BOINC boards, so I've granted you some access here also.
- I'll have to have a think about the credit thresholds for this, raised an issue for it in the meantime.
- Thanks for the ratings system suggestion, I'll implement that change. I've implemented that change.
ID: 130 · Report as offensive     Reply Quote
Jord
Volunteer moderator
Help desk expert
Avatar

Send message
Joined: 24 Jun 20
Posts: 85
Credit: 207,156
RAC: 0
Message 138 - Posted: 1 Jul 2020, 22:41:58 UTC - in response to Message 130.  
Last modified: 1 Jul 2020, 23:30:01 UTC

- I've set up a couple of our guys as moderators, I'll find a few more volunteers and encourage them to stay active. I know you're very active over on the BOINC boards, so I've granted you some access here also.
Thanks, I guess. ;-)
I'm not just active at the BOINC forums, I'm a moderator (and administrator) there as well, I just use a different anonymous account for that. But thanks for the trust. :)
(I also do translation to Dutch for the client, manager and website, even some project's websites. And I update the User Manual and other documentation where necessary. Plus I can do all fun things with issue tickets on BOINC Github)

As for the credit requirement, check out https://boinc.berkeley.edu/trac/wiki/ProtectionFromSpam. Don't use Akismet though, it'll make it very difficult to post URLs, even to your own forums. A lot of the automation (thread locking, removing of spammers etc.) were ideas of mine David Anderson implemented at the BOINC forums, and then when mature enough put out for everyone else.
ID: 138 · Report as offensive     Reply Quote
Jord
Volunteer moderator
Help desk expert
Avatar

Send message
Joined: 24 Jun 20
Posts: 85
Credit: 207,156
RAC: 0
Message 157 - Posted: 2 Jul 2020, 14:36:26 UTC
Last modified: 2 Jul 2020, 14:56:06 UTC

Also not required yet, but keep in mind for the future, and as edited in perhaps it already works as normal.

Now everyone still has all the results of all the tasks they ran since you opened the project visible in their account, Tasks listing. When this grows to several thousands of users with several thousands of pages you can understand this will take quite an impact on your database. It's not necessary to keep these, all you need is the result file in the database and the rest can be deleted. Input files, output files, etc. Normally on a completely set up project this will be done automatically, after 24 hours. The 24 hours can be adjusted if you want to keep results in view longer.

So at your leisure read through https://boinc.berkeley.edu/trac/wiki/FileDeleter and https://boinc.berkeley.edu/trac/wiki/DbPurge

(edit: I checked some of my own tasks and see still lots of Unsent. Which can be attributed to having more tasks in queue than you have computers taking them off, plus you still have 200k tasks RTS so adding unsent tasks you'd have to inject them directly into the feeder... anyway, it probably all works as normal)
ID: 157 · Report as offensive     Reply Quote
Jord
Volunteer moderator
Help desk expert
Avatar

Send message
Joined: 24 Jun 20
Posts: 85
Credit: 207,156
RAC: 0
Message 173 - Posted: 3 Jul 2020, 19:27:47 UTC
Last modified: 3 Jul 2020, 19:28:18 UTC

Two tasks that didn't validate due to too many errors:
https://minecraftathome.com/minecrafthome/workunit.php?wuid=1257094
https://minecraftathome.com/minecrafthome/workunit.php?wuid=1257331

Now, max # of error/total/success tasks = 3, 10, 6
This means a maximum of 3 errors can be had, which in both tasks is 4 (4 independent computers returned it with computing error).

331 I see I got yesterday evening, I ran it to 50% then shut down. Continued it today, in the mean time a couple of Intel GPUs erred on it. I was the 6th to return it but it already had 3 (and even 4) errors.
094 is a different story though, i t had erred 3 times already when it was sent out to me, I returned it correctly, one other computer erred on it as well. Why wasn't it sent out another time, or why was it sent to me in the first place if max error # is 3?
ID: 173 · Report as offensive     Reply Quote
Sergey Kovalchuk

Send message
Joined: 24 Jun 20
Posts: 26
Credit: 1,106,925
RAC: 1
Message 174 - Posted: 3 Jul 2020, 21:32:34 UTC

ID: 174 · Report as offensive     Reply Quote
Jord
Volunteer moderator
Help desk expert
Avatar

Send message
Joined: 24 Jun 20
Posts: 85
Credit: 207,156
RAC: 0
Message 199 - Posted: 5 Jul 2020, 16:36:59 UTC

My next case: https://minecraftathome.com/minecrafthome/workunit.php?wuid=1268362
Two Didn't Need, one Unsent and me. I didn't even know that was possible. There should always be a replication of two for validation.
ID: 199 · Report as offensive     Reply Quote
Profile chip
Project administrator

Send message
Joined: 14 Jun 20
Posts: 78
Credit: 1,321,619
RAC: 0
Message 214 - Posted: 6 Jul 2020, 16:22:54 UTC - in response to Message 173.  

Two tasks that didn't validate due to too many errors:
https://minecraftathome.com/minecrafthome/workunit.php?wuid=1257094
https://minecraftathome.com/minecrafthome/workunit.php?wuid=1257331

Now, max # of error/total/success tasks = 3, 10, 6
This means a maximum of 3 errors can be had, which in both tasks is 4 (4 independent computers returned it with computing error).

331 I see I got yesterday evening, I ran it to 50% then shut down. Continued it today, in the mean time a couple of Intel GPUs erred on it. I was the 6th to return it but it already had 3 (and even 4) errors.
094 is a different story though, i t had erred 3 times already when it was sent out to me, I returned it correctly, one other computer erred on it as well. Why wasn't it sent out another time, or why was it sent to me in the first place if max error # is 3?


Enabled the reliable hosts stuff, should solve it for now
ID: 214 · Report as offensive     Reply Quote
Sergey Kovalchuk

Send message
Joined: 24 Jun 20
Posts: 26
Credit: 1,106,925
RAC: 1
Message 216 - Posted: 6 Jul 2020, 17:47:06 UTC - in response to Message 214.  
Last modified: 6 Jul 2020, 17:59:56 UTC

Enabled the reliable hosts stuff, should solve it for now

at the wrong time, it’s too early to ban hosts, first get a stable application and distribution to suitable hosts.

can set the task the following parameters
max # of error / total / success tasks 10, 10, 3

error = total, can be reduced
success = quorum + 1 (accept too late, but successful)

[edit] it is necessary to regenerate dead WU - they are normal data and might contain solutions
ID: 216 · Report as offensive     Reply Quote
Jord
Volunteer moderator
Help desk expert
Avatar

Send message
Joined: 24 Jun 20
Posts: 85
Credit: 207,156
RAC: 0
Message 217 - Posted: 6 Jul 2020, 19:47:23 UTC - in response to Message 214.  
Last modified: 6 Jul 2020, 19:56:48 UTC

Enabled the reliable hosts stuff, should solve it for now
It's causing my computer to not be eligible to get work. If that's what you want, then so be it. But that's not what reliable hosts is there for. My host is sporadically on and running BOINC, I have an average turn-around time for this project of just under an hour (0.04 days). When it does get work it'll run it and return it in 40 minutes. Trouble is, my host hasn't had the chance yet to reach 10 consecutive valid results to be allowed to be reliable. 9 is the most it had in a row before there were two that couldn't be validated due to the problem I pointed out. So my count started anew, but your reliable hosts setting made it that my host cannot become reliable as it didn't make the requirements.

Hosts whose average turnaround is at most reliable_max_avg_turnaround and that have at least 10 consecutive valid results e are considered 'reliable'. Make sure you set this low enough that a significant fraction (e.g. 25%) of your hosts qualify.
You may run the project as you seem to find fit, I don't mind. But just filling in values without reading the documentation isn't going to be of help here. And it's not needed. I asked you a couple of times already to email the BOINC Projects email list if you feel you're in over your head, as the other project admins are more than willing to help out.

Do know that anyone with a new computer signing up now can't get work as their computer isn't reliable. And it's not explained either in the scheduler messages. Mine just says it asked for work but didn't get any.

06/07/2020 21:56:14 | minecrafthome | Requesting new tasks for AMD/ATI GPU
06/07/2020 21:56:14 | minecrafthome | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
06/07/2020 21:56:14 | minecrafthome | [sched_op] AMD/ATI GPU work request: 8820.00 seconds; 1.00 devices
06/07/2020 21:56:15 | minecrafthome | Scheduler request completed: got 0 new tasks
06/07/2020 21:56:15 | minecrafthome | [sched_op] Server version 715
06/07/2020 21:56:15 | minecrafthome | Project requested delay of 7 seconds

Go away, shoo.
ID: 217 · Report as offensive     Reply Quote
Profile chip
Project administrator

Send message
Joined: 14 Jun 20
Posts: 78
Credit: 1,321,619
RAC: 0
Message 221 - Posted: 7 Jul 2020, 13:23:20 UTC - in response to Message 216.  
Last modified: 7 Jul 2020, 13:38:32 UTC

it is necessary to regenerate dead WU - they are normal data and might contain solutions


We'll definitely reinstate any failed workunits. I first need to determine if it's safe to remove the error mask and change the thresholds on the failed workunits rather than re-creating them.

It's causing my computer to not be eligible to get work. If that's what you want, then so be it.

at the wrong time, it’s too early to ban hosts

You may run the project as you seem to find fit, I don't mind. But just filling in values without reading the documentation isn't going to be of help here. And it's not needed. I asked you a couple of times already to email the BOINC Projects email list if you feel you're in over your head, as the other project admins are more than willing to help out.


That's a surprise and wasn't my intention. I don't want to ban hosts. The purpose of the change was to increase the priority of duplicate jobs so that they'd be processed by reliable hosts; this was to ensure incomplete workunits are prioritized to complete validation as per this thread.

I thought I fully understood the documentation, my comprehension of the options are that reliable hosts are selected first to run work which have a priority equal to or higher than the value provided by .
Results with priority at least reliable_on_priority are treated as "need-reliable". They'll be sent preferentially to reliable hosts.


Do know that anyone with a new computer signing up now can't get work as their computer isn't reliable. And it's not explained either in the scheduler messages. Mine just says it asked for work but didn't get any.

This may have been due to a failed version bump of the kaktwoos app to 2.02 which has since been reverted. It looks as though all computers are recieving work now as far as I can see from the data. I may be mistaken though, in which case we can look at reverting another change to resolve this.

Trouble is, my host hasn't had the chance yet to reach 10 consecutive valid results to be allowed to be reliable. 9 is the most it had in a row before there were two that couldn't be validated due to the problem I pointed out.

first get a stable application and distribution to suitable hosts.

can set the task the following parameters


error = total, can be reduced
success = quorum + 1 (accept too late, but successful)


Kaktwoos 201 has 9273 successes and 2550 failures, 2368 of the 2550 failures are attributable to 6 problem hosts, most of which do not have OpenCL headers installed.

hostid  num_fails  reason
302     481    nvidia opencl headers missing
471     424    HD 4000, now specifically disabled due to lack of OpenCL 2.0 compat.
1196    413   old APU, incompatible drivers
510     405    old drivers, user couldn't upgrade, now disabled
1436    375   outdated drivers
1035    272   missing libraries which we need to statically compile see (https://minecraftathome.com/minecrafthome/forum_thread.php?id=25)


We're working on a solution to send a message to the user via the scheduler in situations where hosts appear to have the correct drivers and match the plan class, but do not have the correct drivers.

I will increase the total error count on the workunits to account for this instability though which should be a band aid in the meantime.

ID: 221 · Report as offensive     Reply Quote
Jord
Volunteer moderator
Help desk expert
Avatar

Send message
Joined: 24 Jun 20
Posts: 85
Credit: 207,156
RAC: 0
Message 222 - Posted: 7 Jul 2020, 13:48:18 UTC - in response to Message 221.  

For the replication, errors and such see https://boinc.berkeley.edu/trac/wiki/JobReplication
ID: 222 · Report as offensive     Reply Quote
Jord
Volunteer moderator
Help desk expert
Avatar

Send message
Joined: 24 Jun 20
Posts: 85
Credit: 207,156
RAC: 0
Message 225 - Posted: 8 Jul 2020, 13:27:35 UTC
Last modified: 8 Jul 2020, 13:30:12 UTC

By the way chip, you can also use the BOINCNetwork on Discord: https://discord.gg/wPRafUq
ID: 225 · Report as offensive     Reply Quote
Profile chip
Project administrator

Send message
Joined: 14 Jun 20
Posts: 78
Credit: 1,321,619
RAC: 0
Message 226 - Posted: 8 Jul 2020, 17:12:40 UTC - in response to Message 225.  

By the way chip, you can also use the BOINCNetwork on Discord: https://discord.gg/wPRafUq


Funnily enough I'm already reasonably active in that Discord, they've been helpful for a few queries
ID: 226 · Report as offensive     Reply Quote
Sergey Kovalchuk

Send message
Joined: 24 Jun 20
Posts: 26
Credit: 1,106,925
RAC: 1
Message 292 - Posted: 18 Jul 2020, 19:28:50 UTC - in response to Message 138.  

As for the credit requirement, check out https://boinc.berkeley.edu/trac/wiki/ProtectionFromSpam. Don't use Akismet though, it'll make it very difficult to post URLs, even to your own forums. A lot of the automation (thread locking, removing of spammers etc.) were ideas of mine David Anderson implemented at the BOINC forums, and then when mature enough put out for everyone else.

server under attack by spam bots
Users - Registered in past 24 hours 3215

admin intervention is required
ID: 292 · Report as offensive     Reply Quote
Jord
Volunteer moderator
Help desk expert
Avatar

Send message
Joined: 24 Jun 20
Posts: 85
Credit: 207,156
RAC: 0
Message 295 - Posted: 18 Jul 2020, 20:52:18 UTC - in response to Message 292.  
Last modified: 18 Jul 2020, 20:59:38 UTC

While it doesn't prevent the bots from making accounts (they can even make accounts with a recaptcha in place), I would advice to only allow users with credit > 100 to make a profile and add an URL. That way the spammers will be able to make accounts, but not fill profiles and URLs with their crap.

Now, if you look at users 661-680 they'll have spam in their profiles.
On the BOINC server we have those as well, but these get deleted automatically with the delete_spammers script that runs 24/7. So then you have a user called AkilahSpoffo who doesn't exist anymore. And that for all the spam accounts made today with spam in their profile or URL. We delete these accounts as we don't have credits on the BOINC server/forums.
ID: 295 · Report as offensive     Reply Quote
Sergey Kovalchuk

Send message
Joined: 24 Jun 20
Posts: 26
Credit: 1,106,925
RAC: 1
Message 296 - Posted: 18 Jul 2020, 22:54:38 UTC - in response to Message 295.  

Probably bots use not a web interface with recaptcha, but RPC (BM interface)
this can be blocked by another Turing test - an invitation code printed on the home page near the join button,
something like minecraft or kaktwoos
for some reason this is not in the document ProtectionFromSpam
ID: 296 · Report as offensive     Reply Quote
Jord
Volunteer moderator
Help desk expert
Avatar

Send message
Joined: 24 Jun 20
Posts: 85
Credit: 207,156
RAC: 0
Message 300 - Posted: 19 Jul 2020, 11:26:31 UTC - in response to Message 296.  

for some reason this is not in the document ProtectionFromSpam
Because it's an RPC/Account Control thing, not a protection from spam thing.
https://boinc.berkeley.edu/trac/wiki/AccountControl
https://boinc.berkeley.edu/trac/wiki/WebRpc
ID: 300 · Report as offensive     Reply Quote
Profile PDW

Send message
Joined: 24 Jun 20
Posts: 3
Credit: 100,026,604
RAC: 16
Message 478 - Posted: 25 Nov 2020, 18:18:08 UTC - in response to Message 292.  

server under attack by spam bots
Users - Registered in past 24 hours 3215

admin intervention is required

You are now over 7.2 million users with more than 20 thousand a day signing up.
Not sure they are all waiting for work though.
ID: 478 · Report as offensive     Reply Quote

Questions and Answers : Web site : House keeping