Modularization of Hyperledger Besu - can we make Besu more flexible by factoring it into decoupled components which can be exchanged for alternate implementations.

Goal of this document: 

• Starting a conversation about modularizing Besu.

...

We are getting various signals

Erigon, Turbo Geth, MEV proposer /producer ..

• Pros: 
• Cons:
• Risks:
• Hyperledger 
• Mitigations:
• Horizon: 
• pre-merge ? 
• post merge?
• Duration: 
• 9 months - but we should divide in increment (module per module). 
• For example, how long to modularize one core component of Besu?

General

Two step approach 

• Open-minded approach, engineers attempting the modularization to see the real scope

Engineering effort around Besu

Cross team (internal and external) + Infurathat the future of blockchain technologies is all about modularity.  If L2 chains on top of L1 chains are the future, how can we make an L1 client that can be composed from various implementations of sub-components.  We see evidence of this elsewhere as well- The Merge separated consensus from execution. MEV actors like Flashbots separate proposing a block from building it. Even within clients, we see teams like Erigon re-writing their client in different languages, and combining the best performing subcomponents regardless of language.

Apart from the general direction of blockchain, software has been trending away from monolithic implementations, in order to maximize developer efficiency, and reduce change fatigue. Smaller components can reach stability more easily than large monoliths can.

Potential Benefits

Releases - finer grained components could have a finer grained release process, speeding up the release cycle.

Reduces Cognitive Complexity - better defined scope for contributors to target a specific part of the codebase.  New developers can focus more narrowly, and get up to speed faster with fewer distractions.

Increases pace of innovation - experiments and prototypes become much easier, faster, and lower risk to pursue.

End User Control - software modularity should lend itself easily to greater customizability for the end user. Whether this is exposed to them has yet to be discussed.

General Concerns and Challenges, Possible Mitigations

1. Engineering effort around Besu
   1. Large engineering effort  
   2. Series of workshops to define the work
2. Technical project organization
   1. Communication

...

1. 1. planning
      1. Internal - how do we make sure all Besu contributors can keep their finger on the pulse of this initiative.
      2. External - do we need to convey this to external users or interested parties. If so, how?
   2. multi stakeholders discussion, federating people around modular besu

...

• Large engineering effort 

Use case: MEV, rollup

Modules: EVM, tx pool (pluggable storage, tracing/analytics/API layer?) 

Modular infra would allow us to select specific core parts of Besu needed to run an ethereum client by our customers with consensys plugins on top.

Reducing the dependency on HL?

Modularization

Pros: speeding up release cycle, better defined scope for contributor to target a specific part of the codebase

• Inversion of control (justin)
• Paradigm to help modularizing the codebase that we need to 

Solution

• That is not adding tech debt
• Easy to maintain from a ConsenSys perspective
• Providing consensys some controls over extra features
• Allowing for a platform to help create features more easily (new modules)
• Hackathon/Bounties “best MEV plugin on Besu”

1. 1. . Examples of stakeholders this would benefit:
      1. MEV searchers
      2. rollup implementers
      3. infrastructure providers like Infura or Alchemy
      4. developers

Step 1: What are Besu Minimum viable components ? Or should we think about it as a combination of modules (brainstorming / workshop session)

...

• EVM Engine
• Consensus Protocols
• Peer-to-peer communications
• JSON-RPC Communications
• Data Storage
• Block Production

Modularization technical braindump

(brainstorm) Modular Client for the Merge - draft

1. Approach #1 - “Besu as Debian” - create distribution artifacts from modules
2. Approach #2 - “Execution Engine Shell” - create an execution engine project leveraging hyperledger/besu modules
3. ?

Synthesis of the conversation on Discord

Participants: Gary, Danno, Sajida, Tim

https://discord.com/channels/@me/804833347816914944/885936379618545724

...

Debrief of meeting with Erigon

...