Danbooru

A possible way to get png from Twitter

Posted under General

To be precise, a way to get png that were auto-converted to jpg.I saw a tweet and uploaded it with the bookmarklet (post #3062198). The picture was a jpg and uploaded as a jpg. However, later I saw the same image on Discord, with the same filename and dimensions, but in png. Eventually I found that indeed there's a png version on Twitter - the same filename but a different extension, and :large rather than :orig, because the latter gives me an empty page. However, trying to upload it gave me a long list of errors:

Show

An error occurred: error: Downloads::File::Error - HTTP error code: 404 Not Found
/var/www/danbooru2/releases/20180321005618/app/logical/downloads/file.rb:148:in `block in is_cloudflare?'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/dalli-2.7.6/lib/active_support/cache/dalli_store.rb:113:in `block in fetch'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/dalli-2.7.6/lib/active_support/cache/dalli_store.rb:371:in `block in instrument'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activesupport-4.2.8/lib/active_support/notifications.rb:166:in `instrument'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/dalli-2.7.6/lib/active_support/cache/dalli_store.rb:371:in `instrument'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/dalli-2.7.6/lib/active_support/cache/dalli_store.rb:112:in `fetch'
/var/www/danbooru2/releases/20180321005618/app/logical/cache.rb:18:in `get'
/var/www/danbooru2/releases/20180321005618/app/logical/downloads/file.rb:146:in `is_cloudflare?'
/var/www/danbooru2/releases/20180321005618/app/logical/downloads/file.rb:138:in `uncached_url'
/var/www/danbooru2/releases/20180321005618/app/logical/downloads/file.rb:39:in `block in download!'
/var/www/danbooru2/releases/20180321005618/app/logical/downloads/file.rb:38:in `open'
/var/www/danbooru2/releases/20180321005618/app/logical/downloads/file.rb:38:in `download!'
/var/www/danbooru2/releases/20180321005618/app/models/upload.rb:421:in `download_from_source'
/var/www/danbooru2/releases/20180321005618/app/models/upload.rb:117:in `block in process_upload'
/var/www/danbooru2/releases/20180321005618/app/logical/current_user.rb:10:in `scoped'
/var/www/danbooru2/releases/20180321005618/app/models/upload.rb:113:in `process_upload'
/var/www/danbooru2/releases/20180321005618/app/models/upload.rb:158:in `process!'
/var/www/danbooru2/releases/20180321005618/app/controllers/uploads_controller.rb:58:in `create'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/abstract_controller/base.rb:198:in `process_action'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_controller/metal/rendering.rb:10:in `process_action'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:117:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:505:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:92:in `__run_callbacks__'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:778:in `_run_process_action_callbacks'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:81:in `run_callbacks'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/abstract_controller/callbacks.rb:19:in `process_action'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_controller/metal/rescue.rb:29:in `process_action'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_controller/metal/instrumentation.rb:32:in `block in process_action'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activesupport-4.2.8/lib/active_support/notifications.rb:164:in `block in instrument'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activesupport-4.2.8/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activesupport-4.2.8/lib/active_support/notifications.rb:164:in `instrument'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activerecord-4.2.8/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/abstract_controller/base.rb:137:in `process'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionview-4.2.8/lib/action_view/rendering.rb:30:in `process'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_controller/metal.rb:196:in `dispatch'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_controller/metal.rb:237:in `block in action'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_dispatch/routing/route_set.rb:74:in `dispatch'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_dispatch/routing/route_set.rb:43:in `serve'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_dispatch/journey/router.rb:43:in `block in serve'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_dispatch/journey/router.rb:30:in `each'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_dispatch/journey/router.rb:30:in `serve'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_dispatch/routing/route_set.rb:817:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/rakismet-1.5.4/lib/rakismet/middleware.rb:10:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/rack/agent_hooks.rb:30:in `traced_call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/rack/browser_monitoring.rb:32:in `traced_call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/rack-1.6.8/lib/rack/etag.rb:24:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/rack-1.6.8/lib/rack/conditionalget.rb:38:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/rack-1.6.8/lib/rack/head.rb:13:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_dispatch/middleware/params_parser.rb:27:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_dispatch/middleware/flash.rb:260:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/rack-1.6.8/lib/rack/session/abstract/id.rb:225:in `context'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/rack-1.6.8/lib/rack/session/abstract/id.rb:220:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_dispatch/middleware/cookies.rb:560:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activerecord-4.2.8/lib/active_record/query_cache.rb:36:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:653:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:88:in `__run_callbacks__'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:778:in `_run_call_callbacks'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:81:in `run_callbacks'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_dispatch/middleware/remote_ip.rb:78:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/railties-4.2.8/lib/rails/rack/logger.rb:38:in `call_app'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/railties-4.2.8/lib/rails/rack/logger.rb:20:in `block in call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activesupport-4.2.8/lib/active_support/tagged_logging.rb:68:in `block in tagged'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activesupport-4.2.8/lib/active_support/tagged_logging.rb:26:in `tagged'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activesupport-4.2.8/lib/active_support/tagged_logging.rb:68:in `tagged'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/railties-4.2.8/lib/rails/rack/logger.rb:20:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/actionpack-4.2.8/lib/action_dispatch/middleware/request_id.rb:21:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/rack-1.6.8/lib/rack/methodoverride.rb:22:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/rack-1.6.8/lib/rack/runtime.rb:18:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/activesupport-4.2.8/lib/active_support/cache/strategy/local_cache_middleware.rb:28:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/rack-1.6.8/lib/rack/sendfile.rb:113:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/railties-4.2.8/lib/rails/engine.rb:518:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/railties-4.2.8/lib/rails/application.rb:165:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/rack-1.6.8/lib/rack/tempfile_reaper.rb:15:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/rack-1.6.8/lib/rack/commonlogger.rb:33:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/rack-1.6.8/lib/rack/chunked.rb:54:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/rack-1.6.8/lib/rack/content_length.rb:15:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/newrelic_rpm-4.7.1.340/lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:606:in `process_client'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/unicorn-worker-killer-0.4.4/lib/unicorn/worker_killer.rb:92:in `process_client'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:701:in `worker_loop'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:549:in `spawn_missing_workers'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:563:in `maintain_worker_count'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:293:in `join'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/gems/unicorn-5.4.0/bin/unicorn:126:in `<top (required)>'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/bin/unicorn:23:in `load'
/var/www/danbooru2/shared/bundle/ruby/2.5.0/bin/unicorn:23:in `<top (required)>'
/home/albert/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/bundler-1.16.1/lib/bundler/cli/exec.rb:75:in `load'
/home/albert/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/bundler-1.16.1/lib/bundler/cli/exec.rb:75:in `kernel_load'
/home/albert/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/bundler-1.16.1/lib/bundler/cli/exec.rb:28:in `run'
/home/albert/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/bundler-1.16.1/lib/bundler/cli.rb:424:in `exec'
/home/albert/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/bundler-1.16.1/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/home/albert/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/bundler-1.16.1/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
/home/albert/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/bundler-1.16.1/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
/home/albert/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/bundler-1.16.1/lib/bundler/cli.rb:27:in `dispatch'
/home/albert/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/bundler-1.16.1/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
/home/albert/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/bundler-1.16.1/lib/bundler/cli.rb:18:in `start'
/home/albert/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/bundler-1.16.1/exe/bundle:30:in `block in <top (required)>'
/home/albert/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/bundler-1.16.1/lib/bundler/friendly_errors.rb:122:in `with_friendly_errors'
/home/albert/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/bundler-1.16.1/exe/bundle:22:in `<top (required)>'
/home/albert/.rbenv/versions/2.5.0/bin/bundle:23:in `load'
/home/albert/.rbenv/versions/2.5.0/bin/bundle:23:in `<main>'

I tested it with a tweet of another artist who I know usually posts in png, with the same result (and a similar list of errors).First of all, is this a known thing?
And second, can this be used to improve the bookmarklet? If the cause of those errors is found and corrected, it can let you grab the hidden png versions automatically if they exist.

Actually nevermind, they are just lossy-lossless. I should've looked more closely.

I faintly remember there being a discussion about that exact case a while ago but my search-fu fails me.

After the existence of :orig versions was discovered by someone, bookmarklet uploads were changed to always pull it, but them someone discovered the lossy-lossless PNGs and now Danbooru uses something smarter to choose either the :orig or :large version, IIRC.

1