Concern over mod queue coverage

I'm not albert so I don't have hard numbers, but I feel like the moderation is slipping on checking new mod queue uploads.

I admit I miss some images too, and I don't see every single one like I used to. A lot of my time here is spent on the forums and in dmail. But I still spend considerable time in the queue and I've noticed some troubling things. This is all based on my own impressions, but I feel like I've been doing this long enough to note trends.

For one, the number of Hides on the oldest posts. I always, always begin modding from the oldest stuff in the queue to make sure it gets at least one pair of eyes. It used to be that the oldest posts in the queue had at least 5-6 hides when I looked. Well, right now all the oldest posts have only 2 hides. At the very least, 60% of the mods who consistently use Hide haven't seen those images. I doubt everyone suddenly stopped using Hide.

Next, I'm approving a higher percentage of 3 day images than in the past. It used to be that when I started at the back of the queue, it was generally all crap. Maybe a few missed gems. But that percentage has steadily risen lately and while right this second there's not a huge amount I'd approve back at 3 days, over the past couple weeks there sure has been.

Lastly, there are too many images in the deletion appeal thread that look like they should have been accepted the first time around. Like the queue, it used to be that maybe 10% of the appeals were justified. That percentage has risen significantly lately.

Just putting this out there as my own opinion as an admin, to hopefully spur people to be more attentive to what seems, at least to my eyes, to be a problem lately. I'm sure some mods are a dedicated and on top of it as ever but on the whole I'm a little worried.

I know some people will seize on this as an excuse to criticize the mod process, so I'll say right now that I do think the method is sound and I completely oppose removing or significantly altering moderation. It's just an issue of manpower in my eyes.

Its probably worth considering adding new Janitors, if the existing ones are losing interest.

I've stated personally that I don't work through the queue unless I have to, but I make sure to check out whatever shows up on the front page while I'm around, and I work the Appeals thread at least once a day.

But yes, I think I approved a half dozen posts in a row in the Appeals thread earlier, which is way above average.

I haven't seen the queue in years, so I can't comment on that, but I have noticed the same trend in the appeal thread. So it seems reasonable to assume there is some real reason behind it. I believe albert said he'd monitor the approval and overall activity back when he recruited janitors, so presumably he has some hard data to be analysed.

I can't comment for anyone else, but myself I do admit I've been swamped for some time now and I'm generally happy if on any given day I can stay up to date with the forum and scan incoming uploads every two to three days. But I've never been a major approving force anyway, having concentrated on tagging to which my approvals are mostly incidental, so I doubt I could change things much even if I stopped doing anything at all.

But it is true that things are ever faster-moving; comments, which used to be a significant source of "maybe I should approve this", I've given up completely on, as it's impossible even to read as fast as new comments are coming. Still, I'm not sure the answer is to have more people rather than seek ways to filter out the unnecessary traffic. I estimate the daily uploads at around 50 pages (~1000 posts); with 30 more or less active approvers it should be perfectly feasible to handle it. If we fail to, it means our tools are not good enough, or that we need to choose people more carefully. I'd be very wary to expand the approvers group beyond the 30 we already have, as it's already close to the maximum size at which meaningful moderation is still possible.

Well, the thing to do is to see if any of the current Janitors are not approving at all (or only to such a small amount it doesn't matter), reduce them back to Contributor and promote some other people.

When I go through the queue I see a lot of hidden images that are really good. It leaves me to wonder if people just click "Hide All" every morning or something.

I try to go through the queue and the deletion appeals at least once a day, but sometimes I do accidentally hide something good. I tend to just shrug and think someone else will catch it, but lately I've been unsure enough of this that I've gone back into the "already hidden-list" and approved it then.

I've also noticed like you said that the maximum number of "hides" an image seems to get these days, including me, is 4. It used to be 7 or 8 at one point, and while I know some people don't use the hide functionality, it's still a remarkable trend.

In short, maybe it's time for some new Janitors?

Also, perhaps relatedly, or perhaps this is subject for a new topic, we need to figure out the comments section. It's almost impossible to keep tabs on that, in my opinion.

葉月 said:
I estimate the daily uploads at around 50 pages (~1000 posts);

FWIW, going by the uploads report page there are on average ~220 posts uploaded per day by privileged and lower users and ~325 uploads per day from contributors and up.

Fencedude said: Well, the thing to do is to see if any of the current Janitors are not approving at all (or only to such a small amount it doesn't matter), reduce them back to Contributor and promote some other people.

I definitely would like to see this. I doubt the janitors that don't use their powers would care if they're dropped back to contrib (they wouldn't be losing anything they use), and it'll let us see how many of the hypothetical ~30 are actually doing anything. I guarantee it's not close to 30, certainly not right now.

I'm somewhat with Hazuki in thinking we don't need *more*, we just need to make sure the ones we have, even if it stays the same number, are active.

I have been doing an abnormal amount of traveling lately, usually ending up on a sub-par connection, though I try to always clear the queue when I get back. I thought it was primarily due to that fact, but I have also noticed the queue being quite a bit longer than normal as of late.

jxh2154 said:
I'm somewhat with Hazuki in thinking we don't need *more*, we just need to make sure the ones we have, even if it stays the same number, are active.

This. Just change the inactive people.

Also in "our" defense, the uploads aren't good either. Most of them are: game cgs, duplicates, bad quality caps, unrelated pics, etc.
This isn't a excuse for the people that don't give a shit about the queue, but adding more janitors isn't going to change much what is approved and what not.

As a side note, may I go crazy nazi mode with the comments moderation? It's full (and I mean FULL) of crap and I want to nuke most of it.

I think it's been acknowledged repeatedly by many people that the comment moderation queue is virtually impossible to deal with, and as such, no one has been. I certainly wouldn't mind if someone wanted to take it on though.

I have sort of shied away from moderating comments at all, for a simple reason: if we delete the stupidest comments, we remove the evidence by which the worst commenters can be identified as repeat offenders. I wish the deleted comments were removed from the posts, but still visible in the archive for individual users, with an annotation that they've been removed.

Hmm. To be totally honest, I've done almost nothing in my capacity as a Janitor in the past several months -- I've been generally rather bogged down by irl concerns recently, and am currently hardly doing anything on danbooru except checking the forum from time to time. However, I would prefer not to be demoted to Contributor status, as I feel I can still "use my powers for good", as it were, in deletion appeals, etc. An erstwhile Janitor being inactive doesn't make them any less qualified to make those judgments, so I don't see any reason why current Janitors (myself included) should be demoted. Of course, I also don't think that my being a Janitor should prevent any other qualified people from being Janitors, so I'd oppose a limit on the number of Janitors as well.

On the other hand, continuously increasing the pool of Janitors could lead to a certain amount of red tape, since (active) Janitors can delete and undelete images at will - also, the standards of moderation might be somewhat loosened.

Any thoughts?

The problem is the more people we have, the blurrier the standards. At a certain number of people we'll reach the situation where pretty much everything goes through, which negates the point of having a mod queue in the first place. It's not personal or a question of "bad 0xCCBA696, no cookie for you", but simply the inevitable laws governing any group or organisation large enough. The only way to avoid that is to be very strict about staying below the critical size. All this, however, is still waiting for albert to show some numbers to meditate upon. No-one's saying you'll be sacked yet, or that you won't be allowed to return once the dreaded real life stops bothering you.

And I'd like to point out it's not merely a theoretical concern, we've already run into instances of mysterious somebodies approving complete shit and being more or less untrackable. And every additional person with approval rights makes it that much harder to hold anyone accountable.

I'm pretty sure we have a feature that allows admins (or at least albert) to see who approved which post, am I wrong?

AFAIK, it's very flakey, in particular undeletions aren't tracked at all, and there were a number of other circumstances in which it got confused, according to what jxh2154 said.

EDIT: Plus there are absolutely no aggregate views or reports allowing the admin to look at the approval history of any single mod. And bear in mind that just being able to tell who approved a particular pic doesn't yet equate to holding people accountable. Even if no-one strays much, the overall resulting standards can still get lower and blurrier.

葉月 said:
It's not personal or a question of "bad 0xCCBA696, no cookie for you", but simply the inevitable laws governing any group or organisation large enough.

Haha, yeah, I understand that completely.

But what I seem to be understanding from what you're saying is that it is more important that the tastes of those in charge of the mod queue are focused than it is that they are in accordance with any given standard (such as for example albert's taste, or whatever). I think rather the opposite - if someone is judged to be "trustworthy" in terms of their tastes (which was the whole point of the mod queue in the first place), then if their particular sphere of likes goes slightly outside the collective likes of the rest of the janitors, I doubt it will be into particularly questionable territory. Conversely, just because there are a small number of janitors (and consequently that the tastes of the janitors are focused) doesn't mean that those tastes may not be somewhat questionable.

Of course, those two goals are not mutually exclusive and are in fact rather in line with each other, but I think we should be clear about what the goals are here:

1) We should have a group of janitors such that good images are generally approved
2) We should have a group of a janitors such that shitty images are not approved

#1 can be accomplished either by increasing the number of eyes on the mod queue, or making sure that the eyes on the mod queue are on the mod queue significantly often. #2 can be accomplished only by weeding out those eyes which see good in things which are generally considered bad.

Of course, it's a bit harder work to weed out bad eyes when there are more eyes in total, but if albert has any stats on, say, unapproved posts and who originally approved them, that could certainly help the process along.

As pretty much everyone in this thread has said, that is the crux here - there need to be "aggregate views or reports". However, I don't think that "every additional person with approval rights makes it that much harder to hold anyone accountable". What it does do is make oversight and coordination more difficult, just due to the number of people involved, I guess.

Well, in theory more good people onboard don't hurt. In practice it doesn't work like that. One thing is that every small aberration in anyone's judgement is amplified by the number of people. So if you make a questionable decision once in every T amount of time (and we all do), it'll result in 3.3x as many bad approvals per T when we have 100 mods than when we have 30. Similarly, there's the (always non-zero) probability of appointing the wrong people. And if we have twice as many people, it's twice as likely that at least one shouldn't be a janitor. Or, conversely, they need to be only half as bad to do the same level of damage. And under certain threshold you can't actually pinpoint the wrongness, even if you're seeing its results. That's one thing.

The other is the group cohesion, trust and responsibility. The more people in charge, the less any one of them feels responsible for their actions. It's nothing new. OTOH, the more people we have, the less I can trust that if I leave crap to expire, no-one else will approve it, unless I personally screen each and every other approver. That also touches the issue of the social horizon and the maximal group size one can still internalise.

Thridly, the issue of focus is just as central as ensuring everyone has a decent taste. It doesn't just matter that everyone has a "good" taste, they should also have pretty compatible ones, otherwise we can get into a situation similar to a ring species. Ie. even if any given mod has others who understand perfectly his/her standards, the ones on the opposite ends of the spectrum don't understand each other. And at this point we could either splinter, or get into a situation in which the mods themselves don't understand the moderation standards, which would mean a complete collapse of the whole mechanism. And again, the more people we have, the bigger the risk of blurring it out to the point of incompatibility.

And lastly, in the field I come from (computer science), it's been long observed that the general productivity of a group peaks out very fast, and above that adding more people decreases both the volume and the quality of the results. So the only way forward is to work as hard as possible to make sure everyone is maximally productive, both by choosing the right people and by giving them the right tools.

Yup, all very good points. Though I think a dialect continuum would be a better example, since janitors aren't exactly trying to interbreed with each other ;)

Muey said: I'm pretty sure we have a feature that allows admins (or at least albert) to see who approved which post, am I wrong?

Regular approvals are visible to me and albert.
Undeletions are not visible to me, but may be to albert.
Unapprovals are not visible to me, but may be to albert.
Reapprovals are... I don't remember actually. I don't think I can see them, but I don't come across many.

None of the above are visible to mods or janitors.

I think not having this information out there for everyone is a sound idea, although yes even I, as an admin, don't really have data on who is approving what and how much. Albert as the site owner should have much more.

Perfect is the enemy of good enough. Instead of coming up with six perfect janitors, it may be acceptable to have a dozen acceptable ones and just delete bad art that goes through. It's not as if the current system isn't infallible.

I'm not sure what the best way to search for new janitors is. My problem with the quiz is that even the most casual visitors could take it, become a janitor, and then never visit the site again. I think judging by their favorites is more practical. More than 100 favorites means they visit the site with some regularity. We can give them a trial run and if they approve too much crap or if they aren't very active they get demoted.

I'll start recruiting tomorrow.