Data migration

I'm wondering how the data migration going on now is going to affect safebooru.donmai.us, because frankly, I'm a little worried, considering that is the only reason I am able to trudge through all the images looking for what I wanted. Thanks if you can answer.

It should finish within the next couple hours. Then I'll need an extra hour or so of downtime to sync over any images over the past two days. At that point the sonohara server will no longer be pushing data to hijiribe, so I imagine things will get better. I'm hoping to finish setting up the new server by Sunday evening and hopefully by Monday morning things will be back to normal.

At some point I will have to replace the sonohara server, but it should go much faster since the new server is entirely driven off of SSDs.

As for what prompted this migration, the server had about 600GB of hard drive space left and I decided I needed something more future-proof. The new server has 4x1TB SSD drives in RAID5 with enough space for eight drives total, which I think should cover space needs for the next several years. It'll also have faster CPUs and more RAM.

Will safebooru server stay safe? I have to draw the line somewhere, after all.

If you won't disable moderated uploads will you at least disable the cull script until a day after the migration is done? I'm sure we'd all like to be able to actually moderate the images before they're deleted.

Yes, please. All of the pics I uploaded before I got contributor were deleted, and that's about 80 ._.

The migration should be finished now. Post any issues you see with either Danbooru or Safebooru here.

I'll disable the post pruner until Monday night then.

The API is giving me a lot of 500 Internal Server Error.

Not sure if this is just me, but starting from ~13 minutes ago (current time: 6:39 PST, Feb 16, 2014), I keep getting errors when trying to save notes.

Fixed a bug with the api.

Now the JSON results look like:

[{"has_comments"=>false, "parent_id"=>1466216, "status"=>"pending", "has_children"=>false, "created_at"=>"2014-02-17 05:23:15", "has_notes"=>false, "rating"=>"q", "author"=>"Gumi555", "creator_id"=>115224, "width"=>640, "source"=>"", "score"=>0, "tags"=>"tagme", "height"=>801, "file_size"=>99294, "id"=>1616856, "file_url"=>"/data/80548dbd62dfd5c709d6ea72bcdb2f81.jpg", "preview_url"=>"/ssd/data/preview/80548dbd62dfd5c709d6ea72bcdb2f81.jpg", "md5"=>"80548dbd62dfd5c709d6ea72bcdb2f81"}, {"has_comments"=>false, "parent_id"=>nil, "status"=>"pending", "has_children"=>false, "created_at"=>"2014-02-17 05:21:57", "has_notes"=>false,

RaisingK said:

Now the JSON results look like:

The non-legacy api seems fine, looks like just the legacy posts api is broken. Other legacy apis besides posts also seem to be working correctly.

Since the migration, some posts don't have an 'md5' key.

example: http://danbooru.donmai.us/posts.json?limit=100 returns 100 items, only about 30 of them have md5.

Previously, all posts had 'md5'.

tsixe said:

Since the migration, some posts don't have an 'md5' key.

example: http://danbooru.donmai.us/posts.json?limit=100 returns 100 items, only about 30 of them have md5.

Previously, all posts had 'md5'.

That's an intentional change for loli/shota posts: issue #2096

Toks said:

The non-legacy api seems fine, looks like just the legacy posts api is broken. Other legacy apis besides posts also seem to be working correctly.

Is this still an issue?

albert said:

Is this still an issue?

Yes, /post/index.json still gives that '=>' output.