Danbooru

Danbooru 2 Issues Topic

Posted under General

This topic has been locked.

reiyasona

Moonspeaker said:

As of recently, Danbooru's been appending Unicode FFFD (the ?-in-a-diamond replacement character on my browser) to the end of all the commentaries I've opened to work on.

@Moonspeaker
Can reproduce! The "�" character is present for me, too.

Updated

reiyasona

Type-kun said:

This is incredibly weird since nothing relevant was changed. Commentary query for post #2413015 seems to show that everything is stored correctly. Code responsible for writing commentary text to the edit form wasn't changed for a long time. @Albert?

@Type-kun
For post #2413015 I get the response:

<textarea name="artist_commentary[original_description]" id="artist_commentary_original_description" cols="40" rows="6">瑞鳳にぐちぐち言われる瑞鶴って好き\u0</textarea>

\u0 = invalid (non-UTF-8) character = �

As you said, the entries seem to be stored correctly. Incorrect character encoding could be the issue here.

I found something related on Stackoverflow:

This specific character � is usually the sign of an invalid (non-UTF-8) character showing up in an output (like a page) that has been declared to be UTF-8. It happens often when

  • a database connection is not UTF-8 encoded (even if the tables are)
  • a HTML or script source file is stored in the wrong encoding (e.g. Windows-1252 instead of UTF-8) - make sure it's saved as a UTF-8 file. The setting is often in the "Save as..." dialog.
  • an online source (like a widget or a RSS feed) is fetched that isn't serving UTF-8

Updated

reiyasona

I tested the following Browsers:

  • Firefox: � present
  • Edge: � present
  • Chrome: all clear

I think that Chrome might just ignore invalid characters to bypass problems and appear more user friendly.
This doesn't look like a client side (browser) problem.

RaisingK said:

BrokenEagle98 said:

It seems like the artist commentary was edited several times for post #2413015. In the prior versions, the mysterious character does show up.

And it's still there, if you go to edit the commentary. At least, it is for me.

I believe that Type-kun just tested whether the � character is saved or not.
It can be saved, but deleting it again afterwards doesn't change anything. The invalid character simply reappears.

-> link to GitHub issue #2617

Updated

tapnek

reiyasona said:

@tapnek Do you still have this problem?

Just reproduced it. Got signed out of the ordinary website and have to log back in by clicking "Sign In Securely". No password or username bwwdws. The workaround I have for this is just by logging in via plain HTTP simultaneously, which breaks the point of securely logging in.

reiyasona

tapnek said:

Just reproduced it. Got signed out of the ordinary website and have to log back in by clicking "Sign In Securely". No password or username bwwdws.

@tapnek

First question: Did you initially login via HTTPS or HTTP?
Second question: Did you access Danbooru via HTTPS or HTTP after you came back?

I suspect that you initially accessed Danbooru via HTTP, clicked "Sign In Securely" and then logged in. On your return you accessed Danbooru via HTTP again. You appeared as "signed out" until swapping to HTTPS by selecting the "Sign In Securely" option.

tapnek said:

The workaround I have for this is just by logging in via plain HTTP simultaneously, which breaks the point of securely logging in.

A better workaround is this one:

reiyasona said:

chodorov said:

I had no clue Danbooru supports HTTPS. Is there a chance at there being a user settings checkbox to have it enforced always when you are logged in?

@chodorov
Simply install HTTPS Everywhere. In many cases it will automatically navigate you to an available HTTPS version of a given website.

tapnek

reiyasona said:

First question: Did you initially login via HTTPS or HTTP?
Second question: Did you access Danbooru via HTTPS or HTTP after you came back?

I suspect that you initially accessed Danbooru via HTTP, clicked "Sign In Securely" and then logged in. On your return you accessed Danbooru via HTTP again. You appeared as "signed out" until swapping to HTTPS by selecting the "Sign In Securely" option.

That's exactly how it went, only I didn't appear signed out. I was signed out on the HTTP part and had the privileges of an anonymous user.

reiyasona

tapnek said:

That's exactly how it went, only I didn't appear signed out. I was signed out on the HTTP part and had the privileges of an anonymous user.

The reason for this behavior is that the HTTPS version of Danbooru creates several "secure" cookies in your browser. These cookies are only transmitted over an encrypted connection in case you initially signed in via HTTPS ("Sign In Securely" option).

The authentication cookies used by HTTPS Danbooru are the following:

  • user_id: secure cookie; can be stored up to 20 years in your browser
  • password_hash: secure cookie; can be stored up to 20 years in your browser
  • _danbooru_session: cookie without secure flag; is destroyed on session end (browser close)

If you access Danbooru over HTTP after you closed your browser, the _danbooru_session cookie will already be destroyed. You still have the user_id and the password_hash cookies, but they can only be used for the HTTPS version of Danbooru. This behavior is protecting your login data because you can't accidentally stumble upon a HTTP link to Danbooru that transmits all your cookies in plain text. But, of course, it can also lead to confusion among users. A more user friendly and yet secure solution could involve the introduction of a redirect cookie for users who initially logged in via HTTPS. This cookie would tell the server to redirect a user to a HTTPS version of Danbooru, if he/she accidentally clicked on a HTTP link.

As a workaround solution I can only recommend HTTPS Everywhere.

Update 1: Created issue #2620 on GitHub
Update 2: Created issue #2621 on GitHub

Updated

Jarlath

FYI, the Unicode character glitch appeared in post #2418745, which was originally uploaded via the bookmarklet from Twitter - it didn't exist until user fTFzI8YOOJyy7jw9 added the English translation to the commentary, which appended the character to the Japanese commentary.

So it seems to happen anytime the commentary is changed, and definitely isn't due to Pixiv or whatever.

Type-kun

Jarlath said:

So it seems to happen anytime the commentary is changed, and definitely isn't due to Pixiv or whatever.

It happens during the page output, actually. If you open the html code, you'll find "textarea" elements for editing the commentary with that weird character already present - though it seems that Chrome automatically filters it out, probably due to security concerns. Like I said earlier, it's very weird since nothing was changed in the source code as far as I can see.

Dbx

I think my https everywhere extension turns pixiv images into https urls. Danbooru doesn't create a friendly source link for them and instead directly links to the image.

reiyasona

Dbx said:

I think my https everywhere extension turns pixiv images into https urls. Danbooru doesn't create a friendly source link for them and instead directly links to the image.

@Dbx

I can confirm this behavior. There is no normalized_source rule that catches HTTPS links. The "http:/" part should read "https?:/".

def normalized_source
  case source
  when %r{\Ahttp://img\d+\.pixiv\.net/img/[^\/]+/(\d+)}i, %r{\Ahttp://i\d\.pixiv\.net/img\d+/img/[^\/]+/(\d+)}i
    "http://www.pixiv.net/member_illust.php?mode=medium&illust_id=#{$1}"

  when %r{\Ahttp://i\d+\.pixiv\.net/img-original/img/(?:\d+\/)+(\d+)_p}i,
       %r{\Ahttp://i\d+\.pixiv\.net/c/\d+x\d+/img-master/img/(?:\d+\/)+(\d+)_p}i,
       %r{\Ahttp://i\d+\.pixiv\.net/img-zip-ugoira/img/(?:\d+\/)+(\d+)_ugoira\d+x\d+\.zip}i
    "http://www.pixiv.net/member_illust.php?mode=medium&illust_id=#{$1}"

I will create an issue on GitHub when I'm home again.

Quick / temp fix: Click on your HTTPS Everywhere symbol while being on the pixiv.net domain and deactivate the "pixiv (partial)" rule (written in green letters).

Update: Created issue #2622 on GitHub.

Updated

1 124 125 126 127 128 129 130 131 132 315