...
The issue for Bonsai archive is here and the PR is here. For context there was originally a draft PR which was a start at delivering the feature, and a re-based branch of that PR https://github.com/jframe/besu/tree/multi-version-flat-db-rebase
Design
Bonsai makes use of additional DB segments to store flat data in addition to the state trie. These segments include:
...
A query can then be performed to request the state at a given block.
Finding the nearest state entry for a block
With the above DB lookup scheme we can query state for any block at which that account's state changed. However, we also need a way to find the state of an account at a block where that account wasn't updated. Take the following example:
...
The archive feature therefore implements getNearestBefore() and getNearestAfter() logic. The get-nearest functions perform a lexicographical search for the nearest entry in the DB to the requested key (in this case, 0x4D95FBAF35Fc5A815983F9df94821C1c089DC02f000000000000000b). On finding that the nearest previous lexicographical entry is 0x4D95FBAF35Fc5A815983F9df94821C1c089DC02f000000000000000a, Besu returns that state at that block. This provides the logic needed to satisfy calls such as eth_getBalance at an arbitrary block.
New DB segments
Clearly the size of the flat DB for a Bonsai archive node is going to be considerably larger than that of a regular Bonsai node. This is the tradeoff made by a user who wishes to have access to all historic state of a chain. In order to reduce the performance impact on importing new blocks, the Bonsai archive approach uses different DB segments for historic entries:
...
The original DB segments will typically only contain current state, therefore retaining the block-import performance of a regular Bonsai node. Asynchronously, the node will move old state from these segments to the archive segments. When state is queried, the node initially checks in the current state segment for the account and block. If it isn't found, the node then checks in the correct archive segment to see if it exists there.
The archive state (i.e. which block has the node archived state for) needs to be persisted to the DB as well. To do this, an additional entry exists in the ACCOUNT_INFO_STATE_ARCHIVE segment to record which block the node has archived up to. Note - archiving a block simply means moving all state that relates to changes in that block, from the primary DB segment(s) to the archive DB segment(s). If a block only includes new state (new account, new contract deploy etc) then no entries will be moved to the archive segments as part of archiving that block.
Configuration
Bonsai archive is delivered as a new data storage format. Initially it will be experimental. To create a Bonsai archive node use --data-storage-format=X_BONSAI_ARCHIVE
Rocks DB column families
Currently the column families are identified by an individual byte. You need to know that e.g. 0x0a maps to "TRIE_LOG_STORAGE". The Bonsai archive work might be a good time to update the column families from bytes to string names to make querying the DB directly easier. This would likely mean moving the DATABASE_METADATA version from 2 to 3 and would require decisions about whether to upgrade existing DBs or just use the new names for new DBs in order to make direct queries of the database for problem diagnosis. For example ldb --db=. --column_family=$'\x0a' ... is used to perform queries against the TRIE_LOG_STORAGE DB segment. The 2 new archive segments have been named with their full name. This makes direct DB queries much easier, as per the example given above querying --column_family=ACCOUNT_INFO_STATE_ARCHIVE. It may be useful in the future to change the existing DB segments to use their full names, but since this would require DB migration for existing state is has not been done as part of the archive work.