I’m using the Ethereum cryptocurrency in a way which requires something called
full Ethererum archival node where each new blocks (somewhat a batch of recent transactions) is added to the blockchain database with all previous states and virtual machine execution steps. More specifically, my case requires running the full open source OpenEthereum software for such task.
This means the underlying Nosql database is powered by Rocksdb and that the data is stored in 6 column format with built in Snappy compression.
The problem is OpenEthereum suffers from heavy
Io workers thread synchronisation issues which means the rebuild speed of the database is capped by single thread performance of the cpu. And Ethereum grew so large that it would now take more than a year to complete a new full archival node sync using the current fastest per thread cpu (along writing several petabytes in terms of ssd durability).
This performance issue is a very complex problem to fix. So much, that it s one of the reason the company which created the software divested from it. So despite it s widespread usage, it s likely to remain unfixed especially, since the lack of manpower saw the replacement of some parallelized algorithms with serial and easier to maintain ones.
Such situation means that while the data is public for free through many websites (using their own Ethereum archival node copy), it s access as a database is not and that I m not interested by the data from the database but to use the tool which requires it.
My node crashed last month when attempting to read a block deep in the past because a recent transaction referred it.
The issue lies at the Rocksdb backend and seems unrelated with OpenEthereum (except for some way to fix it like rebuilding just the corrupt entry by hand). One of my block blased sst file contains a masked crc32 checksum error on a single row which prevents Rocksdb to decompress it. Using the Rocksdb s dump tool, I discovered all my backups contains the error which means the issue went unnoticed for years, but neverless that this is the sole entry being corrupted.
Of course, I failed to find a multi column version of https://github.com/facebook/rocksdb/wiki/RocksDB-Repairer. But one of the easiest solution for rebuilding the file automatically using OpenEthereum would be to remove the damaged block or Ethereum transaction along all the data/sst which were happenned after it (I mean somewhat to reset the database up to the last valid block). But being new to Nosql, not only I ve no ideas on how to write a program opening the database and using
DeleteRange() but I don t understand whether it would require understanding the binary format of OpenEthereum for storing index references.
So where to start for stripping the database?