Danbooru

The database is unavailable

Posted under Bugs & Features

evazion said:

Site is back up. The database server had a failed fan and two bad sticks of RAM that were replaced.

It took a long time to diagnose because at first it appeared that the database server was overloaded and stuck at 100% CPU usage. Normally when that happens it means the site is under heavy load because of bots or scrapers. I spent a long time trying to reduce bot traffic until I finally figured out it was actually bad RAM. I'm still not sure how exactly bad RAM led to 100% CPU usage.

This was probably also the source of the random "The database is unavailable" errors over the past weeks. This tended to happen when the automatic database backups ran. It was probably triggering the bad RAM and causing the database to crash and restart. It was hard to diagnose because I never suspected bad RAM could be the issue.

Thank you!

blindVigil said:

You can't get Gold or Platinum by contributing. Gold is purchase only, which of course isn't available right now, and Platinum is currently only available through raffles. Contributing will get you Builder or Contributor (or potentially even Approver), which are better.

Gold and platinum are still sometimes given to people who don't quite make the cut for builder, but deserve some recognition. Much more rare than builder+, though.

ANON_TOKYO said:

Gold and platinum are still sometimes given to people who don't quite make the cut for builder, but deserve some recognition. Much more rare than builder+, though.

I do remember seeing a couple members being promoted to Gold a few months ago.

sammyG said:

Cool and thanks for the hard work, Evazion.

Going forward please avoid presenting your opinions as absolute or definitive regarding technology. Recognize that you do not hold any expertise on this subject, and humility is valuable.

Try incorporating feedback from others into your worldview.

evazion said:

Site is back up. The database server had a failed fan and two bad sticks of RAM that were replaced.

It took a long time to diagnose because at first it appeared that the database server was overloaded and stuck at 100% CPU usage. Normally when that happens it means the site is under heavy load because of bots or scrapers. I spent a long time trying to reduce bot traffic until I finally figured out it was actually bad RAM. I'm still not sure how exactly bad RAM led to 100% CPU usage.

This was probably also the source of the random "The database is unavailable" errors over the past weeks. This tended to happen when the automatic database backups ran. It was probably triggering the bad RAM and causing the database to crash and restart. It was hard to diagnose because I never suspected bad RAM could be the issue.

I did see that it was down for a moment. Thanks for fixing it!

Artxe said:

Going forward please avoid presenting your opinions as absolute or definitive regarding technology. Recognize that you do not hold any expertise on this subject, and humility is valuable.

Try incorporating feedback from others into your worldview.

Says the one making absolute and definitive claims of another user’s lack of expertise. The only actual claim about technology that user made was about how IP lists are refreshed. If they’re saying something wrong, it would be more productive to correct it instead of making sweeping generalizations about the user.

evazion said:

Site is back up. The database server had a failed fan and two bad sticks of RAM that were replaced.

It took a long time to diagnose because at first it appeared that the database server was overloaded and stuck at 100% CPU usage. Normally when that happens it means the site is under heavy load because of bots or scrapers. I spent a long time trying to reduce bot traffic until I finally figured out it was actually bad RAM. I'm still not sure how exactly bad RAM led to 100% CPU usage.

This was probably also the source of the random "The database is unavailable" errors over the past weeks. This tended to happen when the automatic database backups ran. It was probably triggering the bad RAM and causing the database to crash and restart. It was hard to diagnose because I never suspected bad RAM could be the issue.

Thank you for your work 👍

I'm not really an IT person, but my guess is that the bad RAM probably forced the CPU to compensate in some fashion, increasing the usage. What I could find on a quick search suggests a few possible mechanisms where it could lead to high CPU usage, for example errors when reading/writing to the RAM could cause the CPU to retry the operations, or the lack of usable RAM could force the computer to use more virtual memory. There were a few other failure mechanisms I saw listed but not described nearly in enough detail for me to really understand them.

evazion said:

Site is back up. The database server had a failed fan and two bad sticks of RAM that were replaced.

It took a long time to diagnose because at first it appeared that the database server was overloaded and stuck at 100% CPU usage. Normally when that happens it means the site is under heavy load because of bots or scrapers. I spent a long time trying to reduce bot traffic until I finally figured out it was actually bad RAM. I'm still not sure how exactly bad RAM led to 100% CPU usage.

This was probably also the source of the random "The database is unavailable" errors over the past weeks. This tended to happen when the automatic database backups ran. It was probably triggering the bad RAM and causing the database to crash and restart. It was hard to diagnose because I never suspected bad RAM could be the issue.

Thanks for your hard work,this site is an absolute blessing,I can look at fanarts without going through the toxicity of Twitter lol

1 2 3