Danbooru

Change log - 1/13

Posted under General

This may end up being a controversial change but I had to do something to fight the explosion in database load lately:

API calls (that is, JSON/XML requests) to post listings are now throttled on a per IP address basis, limited to 1 call per second. If you try to make calls more often than this you'll get a blank 503 page.

Updated by ekkk

The saga continues...

Someone keeps attempting searches like source:http://img35.pixiv.net/img/shikishima_e/21274264* which unfortunately is fairly slow. It's spread across multiple IP addresses which makes me think this isn't some isolated service just leeching the site.

For this reason, post XML/JSON feeds are now restricted to members only. An easy way is to just pass in login and password_hash params along with your GET request.

Provided you are authenticated, I've removed the request throttle, although please still limit your request rate.

This is fantastic. For months now I haven't been able to perform any sort of in-depth tag searches (say, searching my favorites for more than one tag), and now I can. No more timeouts, no more failures.

As far as I'm concerned this is a hugely positive change for actual site usage.

I do have some self written python mirror scripts running, too. I will gladly rewrite them.

Can you please be more specific as to "limit your request rate". Which queries do hurt the DB? Is bandwith an issue? (Eg. running many curl processes)

Edit: Maybe i should clarify. I am mirror/backing up my profile _only_. And _only_ actual changes.

Updated

1