Contents (4)
·
✅ We're in Tokyo Now: 22 Minutes of Downtime | 11.8
Offline 00:16–00:39 UTC on Nov 8, no data lost. Faster in Asia and North America, slower in Europe. Try it and tell us how it feels.
Hey Pickful community,
We’ve moved Pickful’s servers from Europe to Tokyo. The downtime was shorter than we planned, and every bit of data came across.
✅ The result
- 22 minutes down: offline from 00:16 to 00:39 UTC on Nov 8. We had planned for about an hour.
- Nothing lost: your posts, comments, points and messages are all here.
- Scheduled tasks back on track: they resumed at 00:44 UTC. Scheduled posts and X syncs will go out as planned.
🔒 How we avoided losing or doubling anything
Moving a database can go wrong in two ways:
- Losing data: new data keeps coming in during the move and never makes it into the copy.
- Doing something twice: both servers run the same task, for example paying the same bounty twice.
Here’s how we avoided both:
- Paused the job queues first, so tasks like posting to X or paying bounties stopped starting.
- Let running jobs finish, then stopped the Europe server. Nothing was cut off halfway.
- Took a “fingerprint” of the stopped data: a count of each key record type (users, posts, points ledger, payout states, queued jobs).
- Compressed the whole database and sent it to Tokyo. Both ends computed a checksum, and we only went on if they matched.
- Fingerprinted the restored copy in Tokyo and compared it with Europe item by item. The site started only after an exact match.
- Pointed Cloudflare at Tokyo to bring the site back, and resumed the job queues last.
If any step had not matched, the process would have stopped on its own. Before the real move, we ran a full rehearsal, and the steps were independently reviewed.
🚀 What changed in speed
The server itself is faster. The new machine has more capacity, and the database is now on PostgreSQL 18. Our median time to handle a request dropped from 60 ms to 30 ms. For the slowest 10% of requests, it dropped from 768 ms to 113 ms.
We’re closer to Asia. When you open Pickful, you first reach the nearest Cloudflare edge. Your feed, logged-in pages, posting and comments all have to go from that edge back to our server.
- Through the Hong Kong edge, that trip went from 431 ms to 173 ms.
- Through US West edges, it now takes about half as long. US East and Canada are about 30% faster.
Some places got slower. Through European edges, the same trip went from about 155 ms to about 340 ms, because the server is now farther from Europe. That’s the trade-off of choosing Tokyo: most of our users are in Asia.
These are early numbers from the first hour and a half after the switch, compared with the same hours the day before.
💬 Try it and tell us
People in different places reach Pickful through different Cloudflare edges, so how it feels to you is exactly what our numbers can’t show. Let us know in the comments:
- Where you are and how you connect: home broadband, mobile data, or a work network
- How it feels: opening the home page, scrolling your feed, posting. Is it faster, slower, or about the same?
- Anything that seems off: pages that won’t load, missing content, notifications that didn’t arrive
The old Europe server stays around until about Nov 22, so we can act quickly if anything comes up.
No app update needed. Web, iOS and Android all work as usual.
Pickful Team
11.8
Repost this post?
Share with your followers.
Reply