Danbooru

'Too Many Requests' message

Posted under General

Is there anyone really familiar with Javascript/JQuery/Ajax that could code up a function wrapper for all of the PUT/POST requests on this site, so that it does an increasing backoff upon getting a 429 response....? I've already converted my Python code so that it does that, but unfortunately I'm still pretty new to the whole Javascript thing... :/

I modified my own program to back off in increments of 250ms (so 250, 500, 750, 1000, ...). It's stuck in a throttle loop right now, and the last two times it got something through, that value had reached ~32 seconds (which sums up to ~30 minutes between successes)... But I was able to make/edit this post, so...?

Ah, well I mentioned this is forum #130819, but the forum_posts controller is one of the few that isn't affected by the API throttle.

Also, from the same topic (topic #9127), it was mentioned that 15 seconds works, so I start my backoff there, so it goes 15s, 30s, 45s, etc. So far, it usually doesn't take more than a minute total to get a successful request through.

The amount of tokens deducted from the limit (as reported by response headers) seems random, so I doubt throttling like that would help. I'm starting to suspect that it's not caused by time going async - because then tokens would be deduced by the same amount if requests came at same intervals. Instead, the limit seems stuck somewhere around -95 when you spam requests. Table corruption may also be the cause, but it doesn't make much sense either, given the observed behavior.

keonas said:

It's starting to trigger even when waiting 25s in between edits.
This is getting ridiculous.

Using the Find Similar/Find Artist tool contributes to it as well, FYI. Can't tell you how many times I've had to sit staring at the screen glumly waiting for my timer to reset and hoping no one else uploaded it.

I'm glad I'm not the only one running into this nonsense. I swear the time required to wait is getting even longer.

This had me worried until I thought to check the forums. I thought I'd been hit with some kind of penalty for something I wasn't aware of.

Edit: Now that I think about it, wtf kind of message is "please rate limit yourself"? It reads really Engrishy. Something a little more descriptive and coherent would be nice, once this issue is fixed.

Updated

I can't mark things as translated, can't edit tags, and now I can't even upload things despite not being at my upload limit. And I have no idea when the timer resets, because it has yet to do so for me since it started.

Edit: okay, uploading works again, but the cool down timer is still too goddamn long.

Updated

BrokenEagle98 said:

Is there anyone really familiar with Javascript/JQuery/Ajax that could code up a function wrapper for all of the PUT/POST requests on this site, so that it does an increasing backoff upon getting a 429 response....? I've already converted my Python code so that it does that, but unfortunately I'm still pretty new to the whole Javascript thing... :/

It would be conceivably possible for one to override jquery's $.ajax to retry on failure, but it would pretty hacky and probably fragile.

chinatsu said:

Some functions such as adding posts to a favgroup don't even appear to provide any warning. Clicking to add to a favgroup just does nothing — super annoying!

There are a lot of functions that haven't been properly tested under error conditions. Post if there are any others.

CodeKyuubi said:

Using the Find Similar/Find Artist tool contributes to it as well, FYI. Can't tell you how many times I've had to sit staring at the screen glumly waiting for my timer to reset and hoping no one else uploaded it.

The only things that are supposed to be rate limited are things that update something on the server. Post if there are other things that are limited that should not be (in normal operation). Dtext preview is another one I think.

evazion said:

There are a lot of functions that haven't been properly tested under error conditions. Post if there are any others.

Creating or deleting a saved search counts as an API call, and fails silently when it hits the rate limit.

I noticed that there's no warning when adding a commentary fails due to the rate limit, though I quickly realized what happened since the commentary didn't appear on the post immediately as it usually does.

1 2