Modularization of Hyperledger Besu - can we make Besu more flexible by factoring it into decoupled components which can be exchanged for alternate implementations.?

Goal of this document: 

• Starting a conversation about modularizing Besu.

...

We are getting various signals that the future of blockchain technologies is all about modularity.  If L2 chains on top of L1 chains are the future, how can we make an L1 client that can be composed from various implementations of sub-components.?  We also see evidence of this elsewhere as well - The Merge separated consensus from execution. MEV actors like Flashbots separate proposing a block from building it. Even within clients, we see teams like Erigon re-writing their client in different languages, and combining the best performing subcomponents regardless of language.

Apart from the general direction of blockchain, software has been trending away from monolithic implementations, in order to maximize developer efficiency, and reduce change fatigue. Smaller components can reach stability more easily than large monoliths can. 

Potential Benefits

Releases - finer grained components could have a finer grained release process, speeding up the release cycle.

Reduces Cognitive Complexity cognitive complexity - better defined scope for contributors to target a specific part of the codebase.  New developers can focus more narrowly, and get up to speed faster with fewer distractions.

...

1. Engineering effort around Besu
   1. Large engineering effort  effort - we will need to always prefer incremental delivery over greenfield or big-bang approaches.
   2. Series of workshops to define the work
2. Technical project organization
   1. Communication planning
      1. Internal - how do we make sure all Besu contributors can keep their finger on the pulse of this initiative.
      2. External - do we need to convey this to external users or interested parties. If so, how?
   2. multi stakeholders discussion, federating people around modular besu. Examples of stakeholders this would benefit:
      1. MEV searchers
      2. rollup implementers
      3. infrastructure providers like Infura or Alchemy
      4. developers

Step 1: What are Besu Minimum viable components ? Or should we think about it as a combination of modules (brainstorming / workshop session)

Situations to assess:

...

Besu Minimum Useful Components

Hypothetical situations that would benefit from component composition:

• EVM and state are needed and not the Consensus. Ex: RollupRollups, Hedera Hashgraph
• Set up call with Revenant / Chupa
• Ask folks to bring ideas around modularization
• Find minimum viable components 
• Catalog all components 
• Scope MVP (minimum viable platform), EVM testing tools
• Transaction pool and block gossip needed. Mev searchers. 
  • Possibly EVM needed to for gas use analysis.
• All-in-one mainnet client that provides ethereum proof-of-stake as its only consensus mechanism.
• State Synch Testbed, rapid prototyping for data stores which can be populated with state changes from a moving chain.

Potential First Steps

• Catalog all components 
• Test approach on one or more modules (see the timing and scoping/lessons learned)situation listed above. 
• Extrapolate out rough timeline on MVP scope and modules timing vs the catalog 

...

• EVM Engine
• Consensus Protocols
• Peer-to-peer communications
• JSON-RPC Communications
• Data Storage
• Block Productioncatalog.
• Scope MVP (minimum viable platform)

...

Debrief of meeting with Erigon

...