Danbooru

Userscript what to bear in mind?/Bitcoin

Posted under General

Bonsai7

Greetings,
I wrote a Script for myself to download Pictures from here in Python, it uses Beautifulsoup and urllib.
it roughly works like this:

page = requests.get(url, headers=header)
for link in soup.find_all(\"a\", href=True):
image.append(link.get('href'))

for every picture on the page do:
page2 = requests.get(image_done, headers=header)
urllib.request.urlretrieve(image,path)

In summary if a page has 10 pictures I have 21 Requests(1 to get every href on the page,10 to get every imageadress and 10 to download the image) per Page.

Since I have no intention of harming the service I would like to know if its ok using my Script to download a good amount of pictures from here?

I know that bandwith costs money and I would not mind to donate once in a while, so I would like to know if Danbooru has a Bitcoin wallet address? I am aware that you can donate with simply upgrading your Account, however debit/credit card are out of question.

BTW: I would not mind sharing the Sourcecode here, but I doubt someone would be really interested in it since there are already lots of solutions.

Even if it pretty late: I did change the script and started using the api which makes it way more easier anyway.

Updated

Reply
ehh

In summary if a page has 10 pictures I have 21 Requests(1 to get every href on the page,10 to get every imageadress and 10 to download the image) per Page.

You could reduce the amount of requests you make by using the data-file-url attribute, e.g.:

for i in soup.select("article[data-file-url]"):
    image = requests.get(i['data-file-url']).content
    ...

But it's probably simpler to just use the API.

Reply
kittey

Bonsai7 said:

Since I have no intention of harming the service I would like to know if its ok using my Script to download a good amount of pictures from here?

If you don’t want to harm the service, use the API, as ehh mentioned. Generating full HTML pages puts a lot of extra strain on the servers and is completely unnecessary if you only want to retrieve machine-readable data. Don’t parallelize and don’t try to circumvent any limits you might encounter and you should be fine.

Btw, the only way to increase the limits imposed on your account is by upgrading it via credit/debit card. If that’s not an option for you, you’ll have to make do with the basic member account limits.

Reply
1