Danbooru

How to upload ugoira, the new pixiv animation system

Posted under General

piespy said:

Yeah, it's a Matroska file with extremely limited options regarding video and audio content and other stuff.

You need a recent enough MKV toolkit to generate webms. At least my mkvmerge (6.7.0) autodetects webm file suffixes and adjusts the output format appropriately.

Ok, updated code with timecode added for mkv. https://cure.ninja/pixiv

Toks said:

I fixed Danbooru's parsing code so it won't try to download these and fail by instead downloading Pixiv's icon for IE.

Thanks!

That'll probably go down as my funniest "oops" moment on Danbooru. This was before the bookmarklet existed, so I was like, "how do I upload this?" Couldn't right click, saving it as a PNG didn't save any animation data, so I pasted the source in the source box and chuckled at my cleverness.

Next thing I know, it's an IE icon with tags for bouncing parsee boobies, and everyone's asking me who I got my crack from.

Updated

I added support for uploading webm to Danbooru for the next version. webm should be preferred when uploading ugoira since it has better quality and file size than gif and better file size and browser support than apng.

I also updated my ruby script to allow converting ugoira to webm instead of just gif.

A bit of feedback for Toks' script.
Fuck your rmagick, that took hours. Damnit
Not sure if loop is supported by wemb but if yes, it doesn't happen. Gif loops alright
It would probably be obvious by the time you actually get rmagick to run, but new ruby users will also need to run "gem install mechanize"
Otherwise, it's awesome. Thanks ^^

Edit:I forgot. Is apng support possible? We'll want wemb on danbooru, but for private use, I'd like it if there was apng as an option

So after spending about an hour and a half trying to get magick to work in ruby I gave up and used the perl script. Has a couple issues though. Using open was crashing my perl, webm settings didn't set a bitrate so quality was kinda ass, and a couple other things didn't quite work right (needed to go up a level to do mkvmerge (and you didn't tell us to install mkvmerge)).

Pastebin of fixes I made is here: http://pastebin.com/UFNx4Djn

Thanks for the effort of writing it, but I would include in your installation instructions making sure to download ImageMagick itself (and telling it to install the precompiled PerlMagick) and download and place mkvmerge in the installation folder. Also using ppm instead of cpan worked better for me (a lot of the stuff was already installed in ActivePerl, though running cpan once to get the compiler is obviously needed).

Edit:I forgot. Is apng support possible? We'll want wemb on danbooru, but for private use, I'd like it if there was apng as an option

If you have all the jpg files you can batch convert to png (irfanview is best for this) and download the gui version of apngasm to roll one yourself (I had issues with it being backwards but it's probably a simple fix I just didn't care to deal with).

The perl script is set to use the console version but it's not set up properly, you'd have to edit my pastebin's version line 150 to this to make it work (and put the console version of apngasm in the same folder as the script):

system("../apngasm ".$pixivID.".png 000000.png 1 $fps")

The webms are notably kinda blurry compared to the apngs you're right. Not sure it's fixable though.

Edit: Also the png file sizes are kinda huge for the more complex files, also in part because of the dithering in the base images from pixiv's end.

Updated

Borrator said:

Edit:I forgot. Is apng support possible? We'll want wemb on danbooru, but for private use, I'd like it if there was apng as an option

I generally don't speak ruby, but it's rather trivial when following Toks' steps.
https://gist.github.com/Type-kun/e7e5f1f49cf22d907da3
new "apng" format is added.

Prerequisites:
1) You still need rmagick installed (to convert jpgs to pngs)
2) You'll need to download non-gui apngasm for your OS flavor and drop it somewhere in your PATH, so it can be called directly from command line (or change the system() call in the script to point to its location).

Note that assembling is rather slow. If size is not critical, you can switch the compression to zlib by adding "-z0" argument to system() call - that will speed things up, but the image will be larger. There's also a newer version , but they don't have win32 one pre-built, and I'm not even sure it works the same way as its predecessor.

Updated

Thanks. I'd say size is more important than assembly time unless you're on a toaster. Especially since apngs are huge. Also, it'd probably be more natural to add "or apng" to commentary line, just for the sake of completeness

Type-kun said:

I generally don't speak ruby, but it's rather trivial when following Toks' steps.
https://gist.github.com/Type-kun/e7e5f1f49cf22d907da3
new "apng" format is added.

Looks good, merged into my gist.

Borrator said:

Fuck your rmagick, that took hours. Damnit

bobjoephil said:

So after spending about an hour and a half trying to get magick to work in ruby I gave up and used the perl script.

Installing imagemagick under windows is suffering.

One thing I can do is change it to only require rmagick when needed, meaning you shouldn't need to install it if you just want to output to webm. But for gif and apng output it's still necessary, no way around it.

Borrator said:

Not sure if loop is supported by wemb but if yes, it doesn't happen. Gif loops alright

Danbooru will force webms to loop when they're uploaded here.

Borrator said:

Thanks. I'd say size is more important than assembly time unless you're on a toaster. Especially since apngs are huge. Also, it'd probably be more natural to add "or apng" to commentary line, just for the sake of completeness

Changed the comment, thanks. Also, I AM on a toaster, so the time difference is pretty noticeable :3

There is also -z2 compression option for Zopfli, and -iNN for number of iterations. I saw examples on the web doing it like "... -z2 -i1000", though I'm not sure which compresses better in the end, you'll have to experiment yourself. As compared to gif, when I tried different approaches on a few pixiv posts, sizes varied from "smaller than gif" to "twice as large as gif", so there's no universal formula here, I guess.

png is bigger in general, and all apngs I've made have been literally enormous. Btw I can't get it to work. Output is none at all. I have the folder where apngasm.exe is stored at the start of "path" environment variable. Am I doing something wrong?
edit: nevermind I had the damned gui.
edit: it works beautifully, thanks ^^ now I just gotta leave it overnight for longer ugoiras
last edit I swear: Test gives like 5 minutes for an "exposition" ugoira (you know, portrait with moving head and changing expression to test your animation skills - those are the largest ones, and also the ones where you are most inclined to want png's loselessness). Still, output size is 18.4 MB, so I definitely wouldn't trade off size for compilation time. For comparison's sake, gif is10.4MB and webm is 1.88MB

Updated

For people struggling with imagemagick under windows, here's how I did that in the end:

0) Go to http://rubyinstaller.org/downloads/ , download both ruby AND ruby devkit. I used Ruby 2.0.0-p481 and DevKit-mingw64-32-4.7.2-20130224-1151-sfx.exe, adjust accordingly for 64x architecture.

0.5) after unpacking/installing devkit, open cmd, cd to devkit install dir and run:

  • ruby dk.rb init
  • ruby dk.rb install

Following steps assume everything installed fine and both ruby install and devkit work fine. Before installing rmagick, you might try to do this first:

  • gem install rubyzip
  • gem install mechanize
  • gem install json

If all of that worked without errors, devkit is most probably installed properly.

1) Go to http://ftp.sunet.se/pub/multimedia/graphics/ImageMagick/binaries/ , and download version 6.7.9-9 for windows, with subtypes "Q16" and "dll", .exe-format installer. I used ImageMagick-6.7.9-9-Q16-windows-dll.exe. Again, you might have to download x64 version instead.

2) Run the installer. DO NOT install in paths containing spaces (ffs, it's 2014, not 1994... let me cry for a bit here). I decided to go simple way and installed in C:/ImageMagick. On "select additional tasks" screen, check boxes like this - you must install headers, because rmagick compiles at install time.

3) Open cmd. run the following commands:

  • set IM_PATH=C:\ImageMagick
  • set PATH=%IM_PATH%;%PATH%
  • set CPATH=%IM_PATH%\include;%CPATH%
  • set LIBRARY_PATH=%IM_PATH%\lib;%LIBRARY_PATH%

then cross your fingers and

  • gem install rmagick

It should ultimately say:
Building native extensions. This could take a while...
and then:
Successfully installed rmagick.

Take a note that I experimented for about 1.5 hrs before finally installing it, so there MIGHT be a step I've forgotten. Also, it's win 32x, win not necessarily work on 64x versions. You'll either have to find out missing ones yourself or ask here :3

Updated

Type-kun said:

...

Since it might help someone, I guess it's worth mentioning what my problem turned out to be. Ruby's auto installer had put a 32bit version despite me having 64bit windows (which is not strange, but you never know), and I had put 64 bit dev kit now knowing that. The dev kit didn't give any errors when installing, so check things like that.
Once I had the 32bit dev kit, the guide in the wiki mentioned by Toks worked just fine, with only one environment variable change

Borrator said:

By the way, I just realised this should be doable. Can someone make a bookmarklet for using Toks' script with webm on current page? (we'll be editing in location ourselves, of course)

You want a bookmarklet that runs my ruby script?

Impossible, bookmarklets can only run javascript within your browser, not executable scripts on your computer.

Well, technically, you can write a server application that calls any local program, using language of your choice, and make it listen to some <port>, then do ajax-style POST request to localhost:<port> from Javascript. Admittedly, that's an incredibly complex solution :3

Type-kun said:

Well, technically, you can write a server application that calls any local program, using language of your choice, and make it listen to some <port>, then do ajax-style POST request to localhost:<port> from Javascript. Admittedly, that's an incredibly complex solution :3

Would be simpler to incorporate my script into Danbooru so we don't have to reinvent the wheel with the "write a server application" thing.

1 2 3 4 5 6 7 8 9 10