Upcoming Changes for Upload and Approval Complaints

OOZ662 said:

My quartile score seems to be pretty horrendous, I guess because I've been keeping an eye out for obscure images outside the Touhou/KanColle/Current-Top-Animes ring (though many of my approvals still fall inside it) and mainly using the modqueue which favors old posts over sniping new potential score goldmines. I guess I'll need to switch up a bit.

This sort of thing seems to be the problem this sort of system will enforce.

It assumes strong correlation between content quality and score, and it's easy to come up with counter-examples.

Consider, say Wakabayashi Toshiya's works, and the fact that they are quite popular, but at the same time, rarely get a score above 2, which is the cutoff for this measurement.

According to this system, it was wrong to have uploaded them, or for any of them to have been approved. Meanwhile, many of the things that were ultimately flagged and deleted after approval were getting higher scores than those posts.

Translated doujins and webcomics are a pretty major draw for this website, with some people coming or at least staying here exclusively for translated works that come in a daily trickle, rather than just sketches.

Even if you lowered the threshold for non-Touhou doujins, you're talking about a very granular measure if you're saying that it's fine to approve a page with 1 score, and not with 0 because nobody bothered to upvote that one page... (And I don't think people will be wanting to read doujins with multiple individual page holes in them...)

If anything, if you're going to use scoring, you should make it possible for more people to rate content, since the main reason advertised reason to become a gold member is to see the loli-tagged stuff, which would inherently skew the base of people who have scoring power. If you want to keep the privilege level, make gold and above have triple voting power or something, but give it out to others.

Likewise, you should create some sort of reward for voting for things, one way or the other (maybe including a neutral option), possibly with greater reward for images below a certain voting threshold, and a search option for low vote images, so that there is some reason for people to actually vote on otherwise obscure copyrights.

Changing the post scoring system just for this, is probably going a bit too far for the issue at hand.

If the formula scoring approvers is tweaked to take other things besides post score into account (maybe without even revealing the formula or approver scores at all) and there's still adequate manual (i.e. by a human) review of an approver's performance and preferences, then I don't really see any major problems with this.

I thought I'd just pop in and add that there are 17235 posts, right now, that are 3+ points and are deleted.

On a positive note, I do like the idea of more granular rights.

EDIT: Dtext is hard for dummies like me.

NWSiaCB said:

Translated doujins and webcomics are a pretty major draw for this website, with some people coming or at least staying here exclusively for translated works that come in a daily trickle, rather than just sketches.

This is certainly my favorite thing about this site.

Doujins and webcomics are more likely to get upvotes if they are translated. Heck, they're more likely to get clicked on if they are translated. The problem is that the translators cannot be translating posts 24/7. They have lives and more important things to do then translate stuff for free. It doesn't seem fair for a post to go unapproved because the translators haven't gotten around to translating it, especially if it's part of an ongoing series.

ShadowbladeEdge said:

I thought I'd just pop in and add that there are 17235 posts, right now, that are 3+ points and are deleted.

It'd be important to exclude banned_artist from that count, as they typically have high score prior to being deleted.

I'd also probably exclude high scoring posts that were simply not approved due to timing out on the queue. The argument here is that low scoring posts that are approved are a bad thing. High scoring posts that are not approved by anyone doesn't affect this metric. I guess a case could be made if a vast majority of unapproved high-scoring posts need to not be approved for quality issues, and that this new system will motivate people to certainly approve them. A cursory glance through those posts though doesn't present much of anything that would be a major detriment.

I would say that duplicates also need to be considered specially. I don't know if policies have changed or not, since I've been much less active in the forums lately, but it used to be that same or lesser quality duplicates were to be parented and not approved. The new rules may motivate people to break that policy and approve them anyway since duplicates tend to have the same scores as the better quality versions (often high). The duplicate policy if it's still being practiced could also harm approvers because it used to be the case that lesser-quality duplicates would get flagged for deletion, even though there was no reason to not approve them in the first place since the higher quality version wasn't around yet.

I'll also second the special consideration of comics and manga. I've never been too keen on approving them, and that probably helps me, but they have never been banned either. I know that at least for some people, they are a valuable part of the site, but obviously not something that will garner a ton of upvotes.

I'm also curious as to how the quartile scores are calculated and meant to interpreted. I'm used to seeing values of 1-4 for quartiles or maybe 0-5 (with 0 and 5 being the minima and maxima), but here I see numbers up to 22. see edit below

EDIT: Rereading my post, it'd also be important that banned_artist approvals not count against approvers either, since they frequently get banned and the posts deleted long after things are posted and approved.

EDIT2: Nevermind I understand the quartile score now. (after digging into the code, I realized I should have just hovered over the header). It's actually the score of the 25th percentile (or first quartile) of all approved posts by that approver in the last month. That makes sense and is why a person with very few approvals can have a very high score, so long as all of the approved posts have very high scores. Conversely a low score means that at least a quarter of your approvals have that score or below in the last month. Sensible enough.

Shinjidude said:

It'd be important to exclude banned_artist from that count, as they typically have high score prior to being deleted.

You're right, that would cut it down to about 7250. Also I was more trying to get at that just because something has a high score it isn't indicative of it's quality. Kinda taking what people are saying about low scores in the other direction.

Hmm, I also think my cursory look wasn't quite what I thought it was. I didn't notice that in the link above, you had status:deleted_score:3.. which I simply added "-banned_artist" to. The underscore makes the query not work properly, and as a result the quality didn't look too bad.

My previous argument was that most of what popped up there didn't look so bad at first glance (mostly not stuff I'd go out of my way to flag myself). Running with the query working (status:deleted score:3.. -banned_artist) looks much worse and rightly contains a lot of unapprovable stuff. If the new policy caused all of that to be approved it'd be a detriment. Unfortunately we don't have much besides score to automatically gauge on, so I'm not sure what would need tweaked to make the metrics better.

Maybe there is a way to weight people who vote up higher scoring posts and vote down lower scoring posts higher than those that do the opposite? That's probably easier said than done though, and could have very unintended consequences if a large number of the voting population disagrees with the quality metrics the moderating team prefers to enforce.

Shinjidude said:

Hmm, I also think my cursory look wasn't quite what I thought it was. I didn't notice that in the link above, you had status:deleted_score:3.. which I simply added "-banned_artist" to. The underscore makes the query not work properly, and as a result the quality didn't look too bad.

Yeah I dicked that one up pretty good. Should be fixed now though.

Going on a tangent, if we're going to have staying an approver tied to post scores in any way then scores need to be invisible before they're approved. Even discounting malicious intent it could just subtly get into peoples heads.

As a lover of statistics I'd also love to see the lifetime versions of these stats. Just for fun.

Fred1515 said:

Changing the post scoring system just for this, is probably going a bit too far for the issue at hand.

If the formula scoring approvers is tweaked to take other things besides post score into account (maybe without even revealing the formula or approver scores at all) and there's still adequate manual (i.e. by a human) review of an approver's performance and preferences, then I don't really see any major problems with this.

This system, however, seems deliberately built to avoid having to manually do anything, so that it's "not anyone's fault" that any given person was demoted.

Shinjidude said:

EDIT: Rereading my post, it'd also be important that banned_artist approvals not count against approvers either, since they frequently get banned and the posts deleted long after things are posted and approved.

Posts from banned artists don't get deleted anymore. Deleting and banning have been separate for years. See that there are only a few hundred posts under status:deleted banned_artist whereas there are almost 10,000 under banned_artist.

Toks said:

Posts from banned artists don't get deleted anymore. Deleting and banning have been separate for years. See that there are only a few hundred posts under status:deleted banned_artist whereas there are almost 10,000 under banned_artist.

That's weird because it definitely took off 10k posts from my score search when I excluded banned artist.

Shinjidude said:

Maybe there is a way to weight people who vote up higher scoring posts and vote down lower scoring posts higher than those that do the opposite? That's probably easier said than done though, and could have very unintended consequences if a large number of the voting population disagrees with the quality metrics the moderating team prefers to enforce.

Going back to what I said earlier, there could be a system whereby users are encouraged to vote more often, by making their votes count slightly more (possibly having non-integer values?) as they vote for things that later get approvals or upvotes when they have few votes at the time of the user's voting.

If an image is "non-controversial" (basically nobody votes against it and it's approved,) then it ups the positive score of their voting record. If they vote up something that gets serious negative feedback, and doesn't get approved, there is a negative on their voting record. If it's within a margin of not getting much support or attracting many detractors, or the user votes some sort of neutral response, they get a much lower positive score just for participating.

To encourage use, maybe some sort of reward of having a temporary privilege of an additional search term or an additional comment on images or something for "good voting practices" if they hit some voting threshold in the past month.

Different user status ranks might have a multiplier to make ranks above basic user still desirable.

ShadowbladeEdge said:

That's weird because it definitely took off 10k posts from my score search when I excluded banned artist.

Off status:deleted score:3..? Can't be, status:deleted score:3.. banned_artist only has 100 posts.

Just a note before someone gets that idea - unlike favorites, danbooru doesn't keep the upvote info forever, so it'll be impossible to grade upvotes based on timestamp and/or user level.

NWSiaCB said:

Going back to what I said earlier...

I'd also worry that this would give people who only care about overrepresented copyrights (Touhou, Kancolle, etc.) even greater weight for little reason. The people who care more for original or less popular (for no quality reason) works, would not get the same boost as those that upvote things that always tend to get upvotes based on who is being depicted rather than the intrinsic quality. Older or lesser known works tend to get little attention compared to juggernaut franchises.

Toks said:

Off status:deleted score:3..? Can't be, status:deleted score:3.. banned_artist only has 100 posts.

Oops, seems like a cut off the ".." after the 3 when I negated banned_artist. I am batting 1000 today on this shit.

Shinjidude said:

I'd also worry that this would give people who only care about overrepresented copyrights (Touhou, Kancolle, etc.) even greater weight for little reason. The people who care more for original or less popular (for no quality reason) works, would not get the same boost as those that upvote things that always tend to get upvotes based on who is being depicted rather than the intrinsic quality. Older or lesser known works tend to get little attention compared to juggernaut franchises.

The weight could be calibrated based upon how many votes a post gets - posts with more votes are worth less, even if you get positive scores. If a post that only gets 5 votes, all positive, is approved, then they may get 10 positive voting feedback points, but if you upvote a post with 50 votes, with 5 negative, and approved might get you only 2 points.

To help with it, you might try throwing in a search button for low-vote posts to help users find posts with low vote counts.

The system, however, is based upon what people do after you vote, so there'd need to be some sort of review time, like the website counts up the votes after a week before it starts giving out the actual voter score. You'd also want to avoid encouraging people from just being the first voter in on a new Touhou post, by making it worth less if 30 people vote just after you.

Empowering the userbase with such a thing as approvals is a big trust challenge. I will try to remain positive.
Kudos for this new initiative. Not for the idea itself, but for the intention of improving the site.

NWSiaCB said:

...

Hmm even though I proposed a similar idea above, the more I think of it, the less I like going down this route. If we end up making scoring as complicated as the US tax code, like the US tax code, it's going to become full of unintended motivations and loopholes people could (and likely would) exploit.

Modifying the scoring system doesn't seem to be a bad idea especially if it's going to be used in this way, but the changes need to be simple and transparent. Long ago before I was a janitor, there was a thing where mods and admins had greater weight given to their votes, but that proved to be unpopular even to some mods (and might be counterproductive if one of the goals is to be able to demote approvers with less than sterling quality metrics). Something simple along those lines though would be less gameable, more easily implementable, and less likely to have unintended consequences.

I've been out of it because of long-running personal issues.

Has there been a lot of issues with the current approval system?

I will say that an issue with the scoring system is that blacklisted tags tend to get disproportinately high scores since the only users that can see/favorite it get the automatic +1 to a post.

I'm inclined to think that the quality of a janitor/moderator should be decided on a case by case basis. There isn't a huge amount of staff, and any automatic system is going to cause issues.

I also think its counter-intuitive since approvers are asked to ignore score/favorites when approving images, having this be the metric for quality goes against this idea and creates pressure to also factor in the type of posts and which are likely to be score-bait.