Danbooru

Read the rules before proceeding!

Topic: Image search

Posted under General

albert

I've added a link on the post/show page. Thanks to piespy for implementing a great tool.

  • ID: 5318
  • Permalink
  • piespy

    albert said:
    I've added a link on the post/show page. Thanks to piespy for implementing a great tool.

    Yay! You're welcome, I'm glad it's useful.

    Now maybe I should think about making it look a little nicer... but since I suck at CSS and HTML, I'll have to ask for some volunteers here. Please contact me via mail or on #[email protected] if you would like to help make it integrate with Danbooru visually as well.

  • ID: 5319
  • Permalink
  • jxh2154

    Very interesting tool, thanks!

    One suggestion since it's integrated with danbooru now. When we search from danbooru, the "best match" is always the image itself. Is there a way for it to not return itself as a result when searching from Danbooru?

  • ID: 5320
  • Permalink
  • piespy

    I could certainly make a special case for that, and remove the query image from the result list if the query URL is a Danbooru preview.

    ... done.

  • ID: 5321
  • Permalink
  • jxh2154

    Now that was fast. Awesome, thanks.

  • ID: 5322
  • Permalink
  • KeliraTelian

    albert said:
    I've added a link on the post/show page. Thanks to piespy for implementing a great tool.

    ...I'm probably being horribly blind and stupid, but where exactly did you put the link?

  • ID: 5332
  • Permalink
  • jxh2154

    Fencedude said:
    ...I'm probably being horribly blind and stupid, but where exactly did you put the link?

    "Similar" at the bottom of the left side menu.

  • ID: 5333
  • Permalink
  • KeliraTelian

    jxh2154 said:
    "Similar" at the bottom of the left side menu.

    Oh. Duh.

    Anyway, nifty.

  • ID: 5337
  • Permalink
  • piespy

    I've changed it to make use of the latest feature that allows going to the post directly. However, that way you can't easily see the post number in the URL, I hope that doesn't make it harder for the mods or whoever wants to use this to parent duplicates.

    Removing deleted posts from my DB using the new deleted post list will have to wait a bit, until then you'll occasionally find deleted posts. Though now you'll at least see why they were deleted instead of only getting a blank result page despite seeing the thumbnail in the similarity results.

  • ID: 5366
  • Permalink
  • Shuugo

    piespy said:
    I've changed it to make use of the latest feature that allows going to the post directly. However, that way you can't easily see the post number in the URL, I hope that doesn't make it harder for the mods or whoever wants to use this to parent duplicates.

    Removing deleted posts from my DB using the new deleted post list will have to wait a bit, until then you'll occasionally find deleted posts. Though now you'll at least see why they were deleted instead of only getting a blank result page despite seeing the thumbnail in the similarity results.

    afaik you can use both methods, and from the /post/index it's still with post ids

  • ID: 5367
  • Permalink
  • piespy

    Yes, but I only have the MD5 of the image. So either I can do an MD5 search that gives you one thumbnail to click on and takes you to the URL with the post ID (that's how it worked before), or I can use the new feature and go directly to the post by MD5 to avoid the extra click but then you don't see the post ID.

  • ID: 5370
  • Permalink
  • スラッシュ

    Yes this does make things slightly awkward for parenting if the old image is the higher quality one... Maybe albert can make the post number show up somewhere on the page or something if it's an MD5 URL?

    For now, it works to take look at the next/previous image and just take what's in between there. Bit convoluted though!

  • ID: 5373
  • Permalink
  • Shuugo

    piespy said:
    Yes, but I only have the MD5 of the image. So either I can do an MD5 search that gives you one thumbnail to click on and takes you to the URL with the post ID (that's how it worked before), or I can use the new feature and go directly to the post by MD5 to avoid the extra click but then you don't see the post ID.

    You could also query the api to get the post id and display it on the thumbs page you have and on the url as a null variable like adding ?md5=foo&pid=100.

  • ID: 5375
  • Permalink
  • piespy

    Shuugo said:
    You could also query the api to get the post id and display it on the thumbs page you have and on the url as a null variable like adding ?md5=foo&pid=100.

    Well, nobody is really interested in the MD5 I think, so if I were to query the api I'd just link to the post page normally. However, I don't know what image a user is going to click on and querying the api for all 16 potential search results seems wasteful and would also delay the display of the search results until the API calls complete.

    Perhaps the query could be done as javascript executed when clicking the "Danbooru post" link and redirecting to the post URL then, but I don't know how well that would work, if at all.

    A decent work-around for now is copying the URL of the tag or note history, since it ends in the post ID.

  • ID: 5377
  • Permalink
  • albert

    I'll just print the ID somewhere on the page.

  • ID: 5378
  • Permalink
  • piespy

    Due to popular demand (well, one person asked for it), I've added a rudimentary XML interface to the search, e.g. to find posts similar to a Danbooru image use it like this:
    http://haruhidoujins.yi.org/db-search.xml?url=http://danbooru.donmai.us/data/preview/be8c6eb6b9760f4ab835b042f4069296.jpg. It's not very well tested so it might occasionally produce bad XML; let me know if that happens.

    If there's an error, the result is of a different format than the normal queries.

    This only returns the raw DB query output. In my script I consider the search successful if the standard deviation of the similiarities is at least 5, and then all matches half a std.dev. above the average are considered relevant.

    Updated by piespy

  • ID: 5396
  • Permalink
  • piespy

    I've added a feature that allows you to directly check pixiv.net images now by inserting the appropriate referrer header. So you no longer have to download the image and then reupload it to the search engine, just pasting the URL should work (until they "fix" their servers to no longer allow this, since it bypasses the need to login...)

  • ID: 5430
  • Permalink
  • dovac

    Is it possible for other danboorus to run a copy of your image search system?

    I've been interested in adding in such a system for moe.imouto.org.

  • ID: 5545
  • Permalink
  • piespy

    I'd be happy to make a search for moe.imouto.org available on my system too if you like, since I've already got it all set up. It could be a separate image DB or merged with the Danbooru DB so that searches will find images from either site.

    You can also run the image query server yourself, though at the moment it's a rather labor-intensive process to set up since it needs a bunch of scripts and must be compiled from source. Well, talk to me on IRC (PM [email protected]) or something if you want to run it yourself. For a small image set like moe.imouto.org it wouldn't take any system resources at all (a couple MB of memory and diskspace at most).

    Though I'll definitely be happy to add it to my system, that'd be less work altogether. Even if it's separate DB I could add an option to search both Danbooru and moe.imouto.org if people just want to find a pic and don't care who has it. Even then you can use the xml query if you want to make your own interface for it.

  • ID: 5548
  • Permalink
  • Shuugo

    piespy said:
    I'd be happy to make a search for moe.imouto.org available on my system too if you like, since I've already got it all set up. It could be a separate image DB or merged with the Danbooru DB so that searches will find images from either site.

    You can also run the image query server yourself, though at the moment it's a rather labor-intensive process to set up since it needs a bunch of scripts and must be compiled from source. Well, talk to me on IRC (PM [email protected]) or something if you want to run it yourself. For a small image set like moe.imouto.org it wouldn't take any system resources at all (a couple MB of memory and diskspace at most).

    Though I'll definitely be happy to add it to my system, that'd be less work altogether. Even if it's separate DB I could add an option to search both Danbooru and moe.imouto.org if people just want to find a pic and don't care who has it. Even then you can use the xml query if you want to make your own interface for it.

    It's possible for konachan too? Since wallpapers are often resized to fit other's screens it would be really handy tool. If you don't want to add that to your server I can try running it locally

    Thanks in advance

  • ID: 5550
  • Permalink