Daily snapshots of Hivemind

avatar
(Edited)

If you don't have a private steemd, the initial sync. time of the Hivemind takes around 6-7 days. This is indeed quite a time to wait.

In order to make that process faster. I have decided to distribute daily Hivemind database snapshots. If you're interested in running a Hivemind node, then you can import this snapshot before running hive sync.

I have made a test install on a fresh server and it took 4 hours to catch up with the block production. (Includes download time, gunzipping, restoring, etc.)

Instructions

Update: See the detailed instructions to restore this snapshot.

Limitations (Important)

  • Dump file is compatible for PostgreSQL 10.

  • There is a simple Python script creates snapshots, every day at 12:30 CET. It takes around 1.5 hours to complete the backup. Therefore, if you want to download the file, check the state.txt on the server. If it says, 'backup in progress', wait until it's complete. Then proceed with downloading. Otherwise, what you download will be a corrupted file.

  • The snapshot includes an additional index on posts_cache table, on author column.

Vote for me as a witness

I do my best to support the blockchain with my skills. If you like what I do, consider casting a vote on via Steemconnect or on steemit.com.

hivemind postgresql busy
0
0
0.000
22 comments
avatar

About 38.42$ has been spent to promote this content.

0
0
0.000
Reply
avatar
(Edited)

Now I have to install postgres on one of my machines to test it!

20190117_174951.jpg

0
0
0.000
Reply
avatar

Awesome man, I was going to ask you to bum a snapshot when I am ready to run one. My full node is pure Steemd & RocksDB. I am not ready to make it a HiveMind based node but I will likely be setting up a Hivemind instance soon.

0
0
0.000
Reply
avatar

Nice! Let me know if you encounter any issues.

0
0
0.000
Reply
avatar

What kind of resources / setup do you need in order to be able to run a full node?

0
0
0.000
Reply
avatar
(Edited)
0
0
0.000
Reply
avatar

Not possible atm. My disk is not that big, yet. However decompressing shouldnt take that long. I have synced a fresh hive install in 4 hours including download, decompress, and restore.

0
0
0.000
Reply
avatar
(Edited)

I experienced the same as wehmoen, download took 10 minutes and decompress has been running for a good 7 hours now, pv says eta 7 hours. So uncompressing might actually take 14 hours.

I wonder if dumping the logs in the console isn't slowing the whole operation.

Might also be that I'm running on hdd and not ssds

0
0
0.000
Reply
avatar

Thank you so much :D I was syncing and it was taking forever downloading your dump now

0
0
0.000
Reply