Documentation Standards Task Force

Team Members

NameEmailRoleTask
Bobbi Muscarabobbi@ledgeracademy.comTask Force DirectorOrganize Group
John Carpenterjohn@globalblockchainsummit.comTask Force MemberOrganize and Help Create Documentation Standards
Sara Moixsaraisabelmoix@gmail.comTask Force MemberHelp Organize and Support
Benjamin Weymouthbenjamin.weymouth@gmail.comTask Force Member Help Organize and Support 


Determine The Current Status of all Projects/Tools/Libraries

  • The purpose of this section is to review the existing documentation hosting setup for multiple Hyperledger Projects.
  • Key Takeaways: 
    • Most Projects use ReadtheDocs
    • Most of those ReadtheDocs projects use either Sphinx, Restructured Text for markdown or a theme enhancer like MKdocs
    • A few projects use a non-traditional documentation hosting service, or do not use any documentation hosting service. 
    • Might be prudent to standardize / harmonize the documentation since most projects utilize ReadtheDocs
ProjectDocumentationFile Type / Documentation ServiceNotesLocationWiki Page CategoriesRepos
FabricREADTHEDOCS.RST FILE and .MD FILESBuilt with Sphinx using a theme provided by Read the Docs.https://hyperledger-fabric.readthedocs.io/en/latest/

Documentation & Resources

Repository


BesuREADTHEDOCS.MKDOC ( Material design) and .MD files

Hyperledger Besu and its documentation are licensed under Apache 2.0 license / This Readthedocs.org documentation is maintained with love by Hyperledger Besu community.

Made with Material for MkDocs 
https://besu.hyperledger.org/en/stable/

Documentation

Documentation on Hyperledger Besu can be found here: https://besu.hyperledger.org/

Repositories

https://github.com/hyperledger/besu/

https://github.com/hyperledger/besu-docs


SawtoothREADTHEDOCS

.RST FILE and .MD FILES


Sawtooth 1.2 documentation has not yet been completely converted to the new site’s format. You can find the documentation in sphinx-doc rst format at:https://sawtooth.hyperledger.org/docs/1.2/

Documentation

Repositories


IrohaREADTHEDOCS

.RST FILE and .MD FILES

Built with Sphinx using a theme provided by Read the Docs.https://iroha.readthedocs.io/en/develop/

Documentation

Repositories


Indy READTHEDOCS.RST FILE and .MD FILES

Built with Sphinx using a theme provided by Read the Docs.


Documentation Index

https://indy.readthedocs.io/en/latest/

Documentation

  • An index of Indy documentation can be found here.

  • The documentation of the nitty gritty of the underpinnings of the code is located in the “docs” folder of each repo.

  • Note: Indy Documentation is in the process of migrating to: indy.readthedocs.io

Repositories


AriesREADTHEDOCS.RST FILE and .MD FILESReadtheDocsRead me bring you to edx classes.

Documentation

For those new to the Aries community, Trust over IP and verifiable credentials, Linux Foundation provides two courses about the concepts and technology:

The latter is obviously more focused on Aries, with the first chapter providing a summary of the former course, and a series of hands on labs based on Aries implementations.

Repositories

Note that while the frameworks listed below are written in a specific, identified language, for the the business layer applications built on top of the frameworks can be implemented in any language.

https://github.com/hyperledger/aries

https://github.com/hyperledger/aries-rfcs

https://github.com/hyperledger/aries-cloudagent-python

https://github.com/hyperledger/aries-framework-dotnet

https://github.com/hyperledger/aries-framework-go

https://github.com/hyperledger/aries-framework-javascript

https://github.com/hyperledger/aries-vcx

https://github.com/hyperledger/aries-agent-test-harness

https://github.com/hyperledger/aries-mobile-test-harness

https://github.com/hyperledger/aries-mobile-agent-react-native

https://github.com/hyperledger/aries-mobile-agent-xamarin

All Aries repositories - https://github.com/hyperledger?utf8=%E2%9C%93&q=aries&type=&language=


BevelREADTHEDOCS.RST FILE and .MD FILES

© Copyright 2021, Hyperledger Bevel.Revision1e64937f.

Built withSphinxusing athemeprovided byRead the Docs.

https://hyperledger-bevel.readthedocs.io/en/latest/introduction.html

Documentation

Repositories


CactusREADTHEDOCS

RST FILE and .MD FILES

markdown or reStructuredText files with the standard theme. 


Documentation

Repositories

https://github.com/hyperledger/cactus/blob/main/README.md


CaliperREADTHEDOCS /MKDocs

.RST FILE and .MD FILES

Note: also uses yml for Mk Docs 

powered by MkDocs and Material for MkDocs

="Jekyll v3.9.2" />

https://hyperledger.github.io/caliper/Documentation: https://hyperledger.github.io/caliper/

Repositories

Source code: https://github.com/hyperledger/caliper

Documentation: https://hyperledger.github.io/caliper/


CelloREADTHEDOCS.RST FILE and .MD FILES
https://github.com/hyperledger/cello/blob/main/README.md

Documentation

Latest Docs

Repositories

Source Code: https://github.com/hyperledger/cello/

Documentation https://cello.readthedocs.io/en/latest/


FireflyJUSTThe Docs

https://just-the-docs.github.io/just-the-docs/ 


** Just the Docs is different from ReadtheDocs. Might be easier to stick to one Layout. 

This site uses Just the Docs, a documentation theme for Jekyll.https://hyperledger.github.io/firefly/ https://hyperledger.github.io/firefly/

Repositories


Grid
  • Platform used unclear. 

** Could be JusttheDocs, but it's unclear which documentation platform or theme they are using. We know Jekyll is used but that could be any number of Documentation hosting services. 

<!-- Begin Jekyll SEO tag v2.8.0 →

Jekyll v3.8.6

The Hyperledger Grid documentation is available on the Grid website: Hyperledger Grid Documentation

Repositories


Transact

docs.rs

http://github.com/hyperledger/transact 


** Uses a different service: Docs.rs, the documentation hosting service


docs.rs/transact/0.4.5

https://crates.io/crates/transact

https://docs.rs/about

Documentation

The Hyperledger Transact API documentation is available on crates.io (click "Documentation"):  https://crates.io/crates/transact

Repositories


Ursadocs.rs

https://github.com/hyperledger/ursa-docs


**Unclear: the documentation is on the github and on Docs.rs 


https://rust-lang.github.io/mdBook/

Documentation

Repositories


































Examine the current process used for documentation


It's logical to dive deeper into some case studies reflecting the above documentation grid. This section dives into Hyperledger Fabric, Sawtooth, Besu- and looks for elements to highlight with each comparison. 

This current process review focuses on a variety of documentation for the HyperLedger Fabric project, as a case study indicative of the successes and pitfalls to learn how other pages might be standardized. This initial commit is a fact finding mission with generalized conclusions. These conclusions should be discussed, edited and converted into direct action items. This is not an exhaustive review, but an initial fact finding summary. It is mean to foster open discussion and create a place for new ideas on the betterment of Hyperledger project documentation. 

Main Recommendations

  • Current / Future Hyperledger Projects should utilize a standard documentation pattern to be agreed upon. 
  • The Fabric documentation pattern is as follows: ReadtheDocs exists as the main source of non-code truth, GitHub for all code truth, and a Hyperledger Wiki page for Community related items and badging. 
  • All Projects can leverage Discord: include or “pin” documentation relevant posts. (currently all are not pinned) 
  • Re-factor all documentation content to adhere to the various sources of truth in the agreed upon documentation pattern. 
  • Standardize graphics across all documentation, especially in the ReadtheDocs
  • Harmonize the Read the Docs- especially in the glossary section for concept lookups and graphical standardization. 
  • Graphics are present in the Read the Docs page, but not present in the glossary- would be a “nice to have” to complete the glossary section. 

Hyperledger Fabric Documentation 

HyperLedger Fabric  documentation can be found in a variety of sources, depending on your use case. Here are the main documentation standards: 

  1. HyperLedger Fabric Read the Docs: https://hyperledger-fabric.readthedocs.io/
    • Designed for Multiple Audiences: Developers and Business Professionals
    • Generally follows topical organization with bullet points 
    • Has code, helpful imagery, written explanations 
    • Sorts concepts into tutorials, very comprehensive spread of information 
    • Doesn’t include source code files, at times will link to GitHub files for more detail
    • Some areas are informative, rich knowledge base
    • Use Cases area could be more informative (Recommend re-fresh / re-formatting of both pages, especially wiki page. currently just links to hyperledger wiki) 
  2. HyperLedger Fabric Main GitHub page: https://github.com/hyperledger/fabric
    • Designed for developers: whether smart contract, application or enterprise blockchain developers 
    • Has a Readme file for general information, versioning, installation 
    • Links back to the HyperLedger Fabric Wiki 
  3. HyperLedger Fabric Wiki: https://lf-hyperledger.atlassian.net/wiki/display/fabric/
  4. HyperLedger Fabric Discord Documentation Channel: https://discord.com/channels/905194001349627914/945038395825070141
    • The Discord channel is designed for asking and answering questions, fostering discussion regarding HyperLedger Fabric Documentation 
    • Not a single source of truth for any audience, however helpful for business, developer or community members 
    • Lots of useful links, but nothing is pinned for quick access
    • As Discord becomes bigger, “pinning” will be eminently helpful 
  5. HyperLedger Fabric LandScape Page: https://landscape.hyperledger.org/projects?selected=hyperledger-fabric 
    • The LandScape Page holds notable metrics, badging, links and an aggregate of the Fabric Twitter Page
    • Metrics include the programming languages used, and the number of recent commits.
    • The links include a comprehensive list of the various Fabric GitHub repositories 
    • The Twitter aggregate section includes the latest 3 tweets from the Hyperledger Foundation

Some Key Conclusions:

Most current Fabric links point to the Read the Docs page as the source of truth. Thus it should be considered the de facto standard in a general sense. Fabric uses Read the Docs as the main source of truth for descriptions, GitHub as the source of truth for Code, and the HyperLedger Wiki page as the source of Truth for community related items. Projects should be customized as required. But some information in Fabric is spread out over multiple pages.  Thus, it would be more efficient to be grouped by audience for easier readability. For example, some details are on the wiki page that aren’t on Read the Docs- and vice versa. Use of graphics on the Read the Docs is sometimes inconsistent. The TaskForce could hone in on any existing inconsistencies in style, graphics, tables, bullet points, readability, links, and content usefulness for all projects to better serve future and current HyperLedger projects. 


Comparison of Fabric Vs. Besu Documentation


This comparison is an initial analysis of the level of standardization between the Hyperledger Fabric Vs. Besu ReadTheDocs. ReadtheDocs was chosen because the ReadTheDocs is main source of documentation truth, and most deep in knowledge / helpfulness. 

Key Takeaways: 

    • Besu introduces itself as a product first, whereas Fabric introduces blockchain before a product introduction. 
    • Should we standardize this? Should we introduce the product or concept? 
    • Recommendation: Plenty of space in the ReadtheDocs for conceptual resources, let’s introduce the product first. (technical documentation typically assumes the developer knows the fundamentals)
    • FAQ style Vs. Concept style. Besu uses FAQ, Fabric conceptual. 
    • Practical Vs. Theory: Besu very quickly is practical. Fabric documentation puts you through lots of theory before set-up steps. 
    • Main structure: Fabric is structured much differently from Besu. 
    • Audience: Even though the Besu documentation is deep and thorough, it is noticeably simpler in structure for a first-time user. 
    • Social Media: Besu main page is lacking social media links, and Fabric displays them- particularly the Discord is important for community engagement. 

Comparison of Fabric Vs. SawTooth Documentation

This comparison is an initial analysis of the level of standardization between the Hyperledger Fabric Vs. SawTooth ReadTheDocs.  

The terminology used between the two varies: while both begin with a key concepts / key terms introduction. Fabric emphasizes the idea of a Blockchain, while Sawtooth emphasizes the idea of a distributed ledger. While every blockchain uses a distributed ledger system, not all distributed ledgers conform to the characteristics of a blockchain. Therefore greater clarity may be gained if all documentation emphasized blockchain as the standard starting point, which utilizes practices related to distributed ledgers. 

Key Takeaways: 

  • No standard navigation style or layout exists between Fabric and Sawtooth. Each project has a different look and feel to their ReadtheDocs
  • Layout and conceptual navigation is different between the projects: this could lead to slower development and adoption
  • Both Fabric and Sawtooth emphasize concepts first, instead of a QuickStart or rapid development deployment project example
  • Fabric has a much deeper conceptual dive into blockchain concepts, while Sawtooth has an initial conceptual dive. 
  • Code formatting looks cleaner and more legible on Fabric Documentation. 
  • Sawtooth begins with a Glossary, Fabric ends with a Glossary. (Should be consistent between the two.) 
  • The Sawtooth documentation may be outdated and require an update to conform to the latest release (per Hardik) 

Comparison of Fabric Vs. Cosmos Documentation

This comparison is an initial analysis between the Hyperledger Fabric documentation and the Cosmos Blockchain documentation.  

Both Cosmos and Fabric begin with a key concepts / key terms introduction. Fabric emphasizes the idea of an Enterprise level Blockchain, while Cosmos emphasizes the idea of application-specific blockchains.  

Key Takeaways: 


  • Cosmos has an updated / web3 look and feel to the UI. 
  • Landing page comparison: Cosmos provides a direct and immediate choice between conceptual and Quickstart / Developer Tutorials.
  • Introduction comparison: similar themes addressed: history, backgrounder, orientation to product
  • Key Concepts: both start with Key Concepts: blockchain, smart contracts, etc. Fabric is more comprehensive from a conceptual standpoint. 
  • Cosmos documentation assumes the audience knows blockchain basics, whereas Fabric is more newcomer friendly. 
  • Tutorials comparison: Fabric are built-in, while the Cosmos tutorials takes me to an entirely new page, new UI, new layout. (not user-friendly) 
  • Cosmos Tutorials means data is duplicated, developer has to start from scratch. 



Review of ReadtheDocs Syntax, Sphinx, and reStructured Syntax

Since many Hyperledger projects use ReadtheDocs, it is important to have an understanding of the underlying structure of the platform. Thus, this section highlights some of the syntax, capabilities and limitations of the ReadtheDocs documentation hosting system. structure.  

ReadtheDocs Highlights: 

  • Hyperledger documentation on ReadtheDocs is populated by files from the various Hyperledger project GitHub pages
  • Those GitHub pages are synced with the ReadtheDocs platform and Pull Requests sync up to an index.rst file
  • The index.rst file, as the extension might imply, is built upon Sphinx and reStructured syntax. (RST stands for reStructured) 
  • Sphinx and reStructured have their own syntax, formatting and are designed specifically for formatting documentation 
  • Matrix Sphinx is a tool that allows us to create documentation using simple markup languages and generate multiple output formats: HTML, LaTeX, PDF, Confluence, ePub, and plain text.

ReStructured Text: 

  • reStructured is not a full fledged markdown language, but does allow for some customization 
    • Myriad options for text formatting is available 
    • images and images with hyperlinks are available 
    • various themes are available
    • backgrounds for code snippets like dark mode are available 
    • Tables, References / Bibliographic citations and a Table of Contents are available 

What is an RST file? Here is an RST Example from Hyperledger Fabric: 


Sphinx 

Sphinx allows for customization of themes, they can be found here. Should they be standard for all projects? 


Proof of Concept for ReadMe: 

A POC was performed for ReadtheDocs using the tutorial found here. All the templates don't show the full power of RST Syntax, while this POC shows buttons and styling that is possible. The POC was to ensure that certain features that are in the web3 community may be applicable to Hyperledger projects. It was found that features are available but ReadtheDocs relies on third-party plug ins and that deeper / more web3-style layouts would require customization work. 

POC Key Takeaways: 

A Proof of Concept / Tutorial Page to dive into the customization options and reStructured Syntax for a ReadTheDocs page.

  • A GitHub Repo was created for a POC of ReadtheDocs functionality
  • Linked up Github with ReadtheDocs so that a Pull Request syncs to the documentation page
  • ReStructured Text Syntax was utilized to test image-based button linking 


Image: pull requests sync easily to the Sphinx platform. 


POC Next Steps / Future Work: Custom CSS and JS 


We can change, or add a common theme to all pages. Currently some pages use standard themes (Ex: Fabric) while others use custom themes (Ex: Besu). 

Apparently, Sphinx can also support custom CSS and JS- thus further customization could be performed. However, the caveat being that this customization would need to be synced up between projects. If different projects used different layouts, functionality it defeats the purpose of standardization. 



 


Create Standards / Best Practices for Community

What does the community think? Harmonization between maintainers and style guide. 


Quote from Community Member: 

On the other hand, I understand that Besu is separate from ConsenSys and it may require a complete isolation of content, and the style/markdown guidelines may need to diverge from ConsenSys's in the future. Our team is happy to continue this discussion here and at the next community call.

Survey For Maintainers and Community  

Consistency between These points: 

1. Location of Documentation: we recommend ReadtheDocs

2. Markup Language

3. Template

4. licensing 

5. Weighing standardization (common template) with project uniqueness. 

6 How to standardize contributions ( tutorials?) 

Team Goals:

Determine to scope of Documentation Standard Task Force

Collect information  both externally and Internally

Create Standards / Best Practices for Community

Create Documentation Best Practice badging system.

Internal:




6/6


6/2

Task: Check out https://docs.cosmos.network/ for a comparison

  1. Documentation Standards

OTHER SIDE

Suggestion:

Create a Technology overview for all products

Get 4 or five bullet point references consistent for all project 

Task: Create a Matrix for badging documentation effort of projects:


OpenSSF Best Practices Badge application complete with Documentation.

Different doc: Dev and User, general knowledge quick jump to spinning up the project. 


Start: Glossary Gentle

Read the Doc : Conceptual.  comprehensive  How to move from audience members' needs?

Different sub headings for users, Sub Topic - developers only - Cosmos read docs is example

Documentation tends to cover basic information that users should or shouldn't be aware. 

STANDARDIZED GLOSSARY

General intro

App dev guide

System guide 

State audience

Quick Start information 

Contribu6tor .md file as an initial entry point. Text or points to files on read the docs.

Project/ Badging

Github Documentation

Read the Docs

Wiki 

Discord

Project/ Badging

Github Documentation

Read the Docs

Wiki 

Discord

General - Product Introduction



Conceptual



End User 



Developer















Fabric
Sawtooth
Indy
Iroha
Besu
Firefly



Documentation

  • The project MUST provide basic documentation for the software produced by the project. (N/A allowed.) (Justification required for "N/A".) [documentation_basics]

    Details:This documentation must be in some media (such as text or video) that includes: how to install it, how to start it, how to use it (possibly with a tutorial using examples), and how to use it securely (e.g., what to do and what not to do) if that is an appropriate topic for the software. The security documentation need not be long. The project MAY use hypertext links to non-project material as documentation. If the project does not produce software, choose "not applicable" (N/A).Rationale:Potential users need documentation so that they can learn how to use the software. This documentation could be provided via the project website or repository, or even via hyperlink to some external information, so we do not specify exactly where this information is.

  • The project MUST provide reference documentation that describes the external interface (both input and output) of the software produced by the project. (N/A allowed.) (Justification required for "N/A".) [documentation_interface]

    Details:The documentation of an external interface explains to an end-user or developer how to use it. This would include its application program interface (API) if the software has one. If it is a library, document the major classes/types and methods/functions that can be called. If it is a web application, define its URL interface (often its REST interface). If it is a command-line interface, document the parameters and options it supports. In many cases it's best if most of this documentation is automatically generated, so that this documentation stays synchronized with the software as it changes, but this isn't required. The project MAY use hypertext links to non-project material as documentation. Documentation MAY be automatically generated (where practical this is often the best way to do so). Documentation of a REST interface may be generated using Swagger/OpenAPI. Code interface documentation MAY be generated using tools such as JSDoc (JavaScript), ESDoc (JavaScript), pydoc (Python), devtools (R), pkgdown (R), and Doxygen (many). Merely having comments in implementation code is not sufficient to satisfy this criterion; there needs to be an easy way to see the information without reading through all the source code. If the project does not produce software, choose "not applicable" (N/A).

Other

David Boswell

Hart Montgomery Arun .S.M. kamlesh nagware – Thanks for your feedback and I see that you've each mentioned that having more resources at the graduated level would be helpful.  Here are some ideas to consider, although it would also be really helpful to hear from people involved in projects about what additional help they'd be interested in.

  • I think there is probably something around documentation and translation support for graduated projects that would be worth doing.  If possible, perhaps we can make some budget available to bring in a technical writer, for example, to help with project docs?
  • We could also make the resources available to Incubation projects time bound – for instance, an Incubation project could have a year to grow and mature and at the end of that time the TSC decides to either move it to Graduated or move it to the Labs.
  • The project placement on the Hyperledger site does not currently provide a distinction between Graduated and Incubation projects since they're treated the same way now.  We're looking at updating the site this year to give more prominent positioning to Graduated projects, so the priority placement item will become more of an incentive.
  • The same is true of Workshops – this is a new concept so we have not yet seen the benefits to a project of running a workshop.  After we've done a few of these we can analyze the outcome and if we can see a bump in users and contributors to a project after a workshop then this will become an incentive for Graduation status too.

    Project participation and interface:

    Criteria:

    • criteria.md - Criteria for "passing" badge
    • other.md - Criteria for other badges (silver and gold)

    Development processes and security:


    Besu: the documentation is released when a new version of the Besu software is released.
    General guidelines

    1. Be consistent - Consistency helps users follow and understand the documentation. By being consistent with your word choices, visual formatting, and style of communication, users know what to expect when they read the documentation. For example, use consistent sentence structures when writing step-by-step instructions.

    2. Be simple but technically correct - Avoid technical jargon and assume the user isn’t an Ethereum expert. When an understanding of a complex Ethereum concept is required, you can refer users to external resources. For example, to explain how the EVM works, link to a resource such as the Ethereum Wiki.

    3. Be proactive and suggest good practices - Anticipate users’ needs and guide them through a process. This often takes the form of notes or tips alongside the main explanation. Put yourself in the user’s shoes and consider what questions you’d have when reading the documentation.

      Documenting good practices is also important. For example, instruct users to secure private keys and protect RPC endpoints in production environments.

    4. Be informative but concise - Provide enough information to help users develop a mental model of how the software works without forcing them to read too much text or redundant detail. Cut down your text by using simple words and concise sentences.

    5. Be inclusive - ConsenSys documentation aims to be inclusive to all users. Refer to the Google inclusive documentation guide and the Microsoft bias-free communication guide as starting points.

    Writing style guide

    ConsenSys documentation follows the Microsoft Writing Style Guide, which is a straightforward reference for natural and clear writing style. The following are some important style recommendations:

    • Abbreviations - Avoid abbreviations and acronyms unless they’re well-known or often repeated in the documentation. Use “for example” instead of “e.g,” and “that is” instead of “i.e.”
    • Active voice - Use active voice where possible. Use “you” to create a personal tone.
    • Code samples - Provide code samples that can be copied and pasted in a console or editor with minimal editing, and work as expected.
      • When writing code samples in a programming language, refer to the programming language’s style guide.
      • Always provide code samples as text in a code block; never use screenshots that would force the user to type it manually.
      • When breaking up lines in a command sample, add line breaks (\) to ensure it can work when pasted.
      • Don’t include the console prompt (>,$,#,%, or the full user@mycomputer Develop %) or other characters that would prevent the command to run when pasted.
      • If values must be replaced in a sample, use placeholders such as <your IP address>.
    • Contractions - Use common contractions, such as “it’s” and “you’re,” to create a friendly, informal tone.
    • Sentence case for headings - Use sentence case instead of title case for headings.
    • “We recommend” - In general, don’t use first person. However, use “we recommend” to introduce a product recommendation. Don’t use “ConsenSys recommends” or “it is recommended.”

    Refer to the Microsoft Guide for any other questions on style.

    Documentation system guide

    Refer to the Divio documentation system guide for how to structure, classify, and arrange the different functional elements of documentation (for example, tutorials, how-to guides, and references).

    1. Manually create a release on GitHub

    When a new version of the software is released, the documentation team manually creates a release on the corresponding GitHub documentation repository.

    Documentation versioning follows the same Calendar Versioning (CalVer) pattern as the software to help users match the documentation version to the software version easily.

    The GitHub release creation process tags the Git repository with the new version (for example, 21.1.4).

    2. Automatically build the documentation on RTD

    When Read the Docs (RTD) detects a new tag on a documentation repository, RTD automatically generates the following documentation versions:

    • Latest - Corresponds to the latest commit in the main branch of the documentation repository.
    • CalVer - Corresponds to the tag in the main branch that was created during the release (for example, 21.1.4).
    • Stable - Corresponds to the last created tag.

    RTD builds all three versions, all showing the same content from the same commit.

    As contributors continue to work on the documentation, RTD rebuilds the latest version from the latest main commit each time a new PR is merged, and the CalVer and stable versions remain behind latest.

    Note: The Besu documentation repository default branch is now named main.

    3. Automatically activate the documentation version on RTD

    By default, RTD doesn’t activate or publish new CalVer versions, but Besu documentation has custom rules for RTD to automatically do so. If you have access to RTD as a Besu documentation maintainer, you can view these automation rules and the history of version activations in the Admin tab of the RTD documentation project.

  • General:
  • Tablet of Contents
  • Tablet of Contents
  • https://github.com/jaredweinfurtner/docs-project-layout
  • Developer Documentation
  • TBD
  • Technical Documentation
  • I have created a template/demo repository for technical documentation:
  • repo: https://github.com/jaredweinfurtner/docs-project-layout
  • generated documentation: https://jaredweinfurtner.github.io/docs-project-layout
  • For technical documentation, a user should not have to browse through the code repository on GitHub.  The user should also have many options to read the technical documentation: Easily navigatable web page, PDF, ePub for eReaders, etc.  The following is a proposal on how to approach technical documentation that is simple for contributors to manage the documentation and simple for readers to take in. 
  • Sphinx
  • Sphinx is a tool that allows us to create documentation using simple markup languages and generate multiple output formats: HTML, LaTeX, PDF, Confluence, ePub, and plain text.  It has an easy to understand hierarchical structure with automatic indices. More importantly, it has a large community building plugins & extensions.
  • To learn more, visit: https://www.sphinx-doc.org
  • Plugins/Extensions
  • The power of Sphinx is in its plugin support.  We are currently using 4 plugins to make life easier for a contributor:
  • sphinx_rtd_theme - Read the Docs (https://docs.readthedocs.io) is used quite a bit in the open-source community and providing a familiar experience using their theme with our documentation for users and contributors has obvious benefits. 
  • sphinxcontrib-drawio - This allows direct reference of *.drawio files directly within the documentation, so no need to export as png every time you make a simple edit eliminating the chance of out-of-sync files and images.
    • reStructuredText:

      .. drawio-figure:: my-awesome-model.drawio

    • Markdown:

      ```{drawio-figure} my-awesome-model.drawio
      ```

  • myst-parser - By default, Sphinx uses reStructuredText (another markup language similar to Markdown).  This plugin allows Markdown support within Sphinx to make it easier to those not wanting to learn another language. For more information, please see: https://myst-parser.readthedocs.io
  • sphinxcontrib-plantuml - similar to the drawio plugin.  It allows PlantUML files to be directly referenced in the documentation.
  • There are so many more plugins that we can use to make our life easier since documentation is never anyone's favorite task:  https://github.com/sphinx-contrib/
  • Docker
  • It is often necessary to generate the documentation locally to verify your changes before committing.  To make life easier for a contributor, I have packaged all of the installation, plugins, and configurations in a docker image so that the same container that builds the documentation locally is the same container that the CI environment builds for deployment. 
  • Source: https://github.com/jaredweinfurtner/sphinx-drawio-docker
    Docker Hub: https://hub.docker.com/repository/docker/jaredweinfurtner/sphinx-drawio-docker
  • Simply run the following in the /docs folder to generate the static html with the Read the Docs theme:

docker run --rm -v `pwd`:/docs jaredweinfurtner/sphinx-drawio-docker make html

  • Structure
  • Inside the /docs folder, the following structure is proposed:

build

├───html
│   ├─  ...
│   └─  index.html
│     
└───latex
    ├─  ...
    └─  docs-project-layout.pdf
 
source
│   ├─ index.rst   
│   └─ conf.py

├───section
│   ├─  overview.md
│   └─  my-drawing.drawio
│  
└───section
    ├─  overview.md
    └─  my-drawing.drawio


  • /build/**
    Contains generated documentation and should be ignored via .gitignore
  • /build/html/**
    Contains static html that can be deployed as any other static html site - for example to: Read the Docs, GitHub Pages, etc
  • /build/latex/**
    Contains mainly cached files for quicker follow-up generation of the pdf, but your *.pdf should be buried in here somewhere
  • /source/index.rst|md
    This is your landing page and should contain the tree structure of your documentation. I highly recommend separating each section & subsection into its own folder so that you can include any diagrams alongside the .rst file.
  • /source/conf.py
    This is the Sphinx configuration file and is the only thing you need to copy & configure into your own project. Simply set the properties listed at the top for your particular project:

# -- Project information -----------------------------------------------------
 
project = 'My Awesome Project'
author = 'Max Mustermann'
copyright = '© 1984-2021 Max Mustermann'
 
# The full version, including alpha/beta/rc tags
release = '0.0.1'

  • GitHub Pages
  • GitHub Pages offers static html hosting of our generated Sphinx documentation with the ReadTheDocs theme.  It nicely binds the project's GitHub repo with technical documentation so the user experience is seamless.  It offers automatic publishing by designating a git branch and a subfolder that contains the documentation (i.e. docs branch with a /docs subfolder).  Normally GitHub uses Markdown to generate the documentation, but it doesn't have the capabilities of Sphinx plugin support (such as draw.io, PlantUML, PDF support).
  • For more information, please see: https://pages.github.com
  • Compatibility with Sphinx
  • Out-of-the-box GitHub Pages is not compatible with Sphinx structure; however, it is easily fixed.  To configure GitHub pages to simply host the statically generated Sphinx documentation (instead of its own processing/theme), we simply add an empty.nojekyll file in the /docs folder. There's even an extension sphinx.ext.githubpages that takes care of that automatically (example).
  • Build / Deployment
  • It is good standard practice to never check-in generated code to a git repo, so we do not check-in the generated Sphinx html from our local environments.  Therefore, it is necessary to generate the documentation (static html) in a CI environment and push it to the docs branch under the /docs folder so that GitHub Pages can automatically deploy it.
  • GitHub Actions
  • GitHub Actions is GitHub's CI system (think Jenkins) and allows us to trigger our static html generation whenever a change is made to the master branch.  We simply generate the static html via the same docker command we use locally during development, move the static html to the root /docs folder for GitHub Pages, add the .nojekyll to tell GitHub Pages to simply host our own static html.
  • Example GitHub Action for GitHub Pages integration (example result: https://jaredweinfurtner.github.io/docs-project-layout/):
    doc-generation.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

# This is a basic workflow to help you get started with Actions
 
name: Documentation
 
# Controls when the workflow will run
on:
  # Triggers the workflow on push or pull request events but only for the main branch
  push:
    branches: [ main ]
 
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
      with:
        persist-credentials: false # otherwise, the token used is the GITHUB_TOKEN, instead of your personal token
        fetch-depth: 0 # otherwise, you will failed to push refs to dest repo
    - name: Generate Documentation
      run: |
        docker run --rm -v `pwd`/docs:/docs jaredweinfurtner/sphinx-drawio-docker make html
        cp -r docs/build/html/* ./docs
        touch docs/.nojekyll
    - name: Commit files
      run: |
        git add --all
        git config --local user.email "business-partner-agent[bot]@noreply.github.com"
        git config --local user.name "business-partner-agent[bot]"
        git commit -m "[bot] docs generation" -a
    - name: Push changes
      uses: ad-m/github-push-action@v0.6.0
      with:
        github_token: ${{ secrets.GITHUB_TOKEN }}
        branch: docs
        force: true

  • External
  • Articles
    https://blog.prototypr.io/software-documentation-types-and-best-practices-1726ca595c7f
  • https://document360.com/blog/software-documentation/
  • https://ifs.host.cs.st-andrews.ac.uk/Books/SE9/Web/QualityMan/docstandards.html
  • https://www.govinfo.gov/content/pkg/GOVPUB-C13-ddd47a2eca7877b72195f405e4a9fb8d/pdf/GOVPUB-C13-ddd47a2eca7877b72195f405e4a9fb8d.pdf
  • Internal
  • From Fabric: 
  • Documentation
    Documentation release process
  • From Besu
  • General guidelines
  • The guiding principles for the Besu documentation are:
  • Be consistent
  • Keep it simple but technically correct
  • Be proactive and suggest good practices
  • Be informative but concise
  • Be inclusive
  • 1. Be consistent
  • Consistency is important to help our end users build a mental model of how Besu works. By being consistent with our word choices, visual formatting, and style of communication it helps users know what to expect when they refer to or search Besu documentation.
  • 2. Keep it simple but technically correct
  • Avoid technical jargon and always assume our end users may not be Ethereum experts.
  • This doesn't mean explaining all Ethereum concepts in our documentation. Explain Besu functionality and when an understanding of complex Ethereum concepts is required refer users to relevant resources.
  • For example, to explain how the EVM works, link to ethdocs.org documentation such as https://github.com/ethereum/wiki/wiki/Ethereum-Virtual-Machine-(EVM)-Awesome-List
  • Simple explanations must still be technically correct.
  • 3. Be proactive and suggest good practices
  • Being proactive means anticipating user needs and guiding them through a process. This most often takes the form of notes or tip messages alongside the main explanation. Put yourself in the user's shoes and consider what questions you would have when reading the documentation.
  • Do not assume required steps are implied. Err on the side of including them if you are unsure.
  • Documenting good practices is also important. For example, instruct users to secure private keys and protect RPC endpoints a production environments.
  • 4. Be informative but concise
  • We seek a balance between providing enough relevant information to help our users develop a solid mental model of how Besu works without forcing them to read too much text or redundant detail.
  • To provide additional detail, use sub-sections.
  • 5. Be Inclusive
  • We believe that we all have a role to play to improve our world and writing inclusive doc is a first step in the right direction.
  • We welcome users and contributors from a wide range of of cultures, so please remember to always observe and respect their preferred language around their identity. 
  • Avoid potential offensive terms and, for instance, prefer "allow list and deny list" to "white list and black list".
  • We suggest to refer to Google inclusive doc writing guide and Microsoft bias free writing guidelines as starting points.
  • Writing style guide
  • We use the Microsoft Style Guide as our general guide to writing technical documentation. We take guidance from it but do not apply every rule. For example, we use title case rather than sentence case.
  • The Microsoft Style Guide aims for natural, simple, and clear communication.
  • Here are some important points we follow:
  • Active voice
  • Use active voice. Use you to create a more personal friendly style. Avoid gendered pronouns (he or she).
  • Contractions
  • Use contractions. For example, don’t.
  • Use common contractions, such as it’s and you’re, to create a friendly, informal tone.
  • Recommend
  • It's acceptable to use "we recommend" to introduce a product recommendation. Don't use "Hyperledger recommends" or "it is recommended."
  • Example: Instead of This is not recommended for production code use We don't recommend this for production code.
  • Directory vs folder
  • Use directory over folder because we are writing for developers.
  • Sentence case for headings
  • Use sentence-style capitalization.
  • Assumed knowledge for readers
  • We have two distinct audiences to consider when developing content:
  • New to Ethereum and Ethereum clients
  • Experienced with Ethereum clients other than Besu
  • Avoid abbreviations
  • Try not to use abbreviations except for well known ones and some jargon. Don't use "e.g." but use "for example". Don't use "i.e." but use "that is".