Potential Plans

Keep the Existing System

Hyperledger would keep our existing hosting Jenkins system and try to modify the Jenkins scripts and setup so that the other Hyperledger projects can set up what they need.

Pros

• The Fabric team has no migration work.

Cons

• Keeps our relatively high level of monetary cost.
• All of the other teams will have to migrate from their existing systems.
• All of the teams that have tried and failed to use our system will need to participate in the requirements gathering process to build a system they can actually use.
• Must work directly with LFIT to make changes to our Jenkins system to have all of the right plugins.

Switch to a Kubernetes Cluster

Hyperledger would stand up a Kubernetes cluster on a cloud provider such as Amazon or Digital Ocean. We would devise a way to delegate limited administration powers to project maintainers so that they can define their own clusters and deploy to those clusters the services they need. They would have to deploy their own CI/CD system and they would have to deploy their own Testnets.

Pros

• Gives us much more fine grained control over running CI/CD pipelines as well as running Testnets.

Cons

• Does not lower our hosting costs and may even increase them.
• Increases the human administration costs because managing Kubernetes clusters to meet everybody's needs is very hands-on.
• Project maintainers have to learn how to use Kubernetes since it is fairly new to most people.
• Requires each team do manage their own CI/CD deployments on the Kubernetes cluster. This does not improve on the visibility problem we have now where each team has their own CI/CD system that is largely closed to the community. 

Switch to Gitlab + Minimal Hosted Runners + Crowdsourced Runners

Hyperledger would switch to running an instance of Gitlab and allow project maintainers to set up and run their own CI/CD and test net clusters. Hyperledger would run a few "shared" runners to ensure that there is always a machine or two available to run jobs for projects that do not have any volunteer runners. This solution allows for projects to not only handle CI/CD but also test nets. Gitlab runners are capable of running just about anything. Project maintainers can create test net projects with a CI pipeline definition that tells the runners to download a docker image of the latest build of a project, and all of our blockchain platforms publish ready-to-run docker images, they can easily stand up a test net for a scaling or soak test. This proposal reduces the human overhead and the financial overhead of running a CI/CD pipeline as well as increasing the level of control each project will have over their CI/CD pipeline.

Pros

• Greatly reduces Hyperledger overhead in terms of hosting dollars as well as human time for administration.
• Reduces how much we have to work with LFIT to make changes to the system. Gitlab will be largely self-service.
• Gives teams much more control over their CI/CD resources.
• Empowers teams to scale up their CI/CD resources through community marketing and/or direct sponsorship.
• Lowers the barrier to entry for teams to set up CI/CD.
• Gitlab does what we need and is free for us to use.
• Increases the visibility into the CI/CD system for all of our projects. There will be one location where all of the builds for all of the projects will be coordinated from.

Cons

• Significant rework of the existing Fabric CI/CD system that may not happen.
• Kelly Olson Sawtooth and Grid rework? Not sure how much effort this will be.
• Iroha rework from their own Jenkins.
• Rework for Burrow, Indy, and Ursa but to a much lesser degree than the others because they are all using Gitlab.
• Crowdsourcing machines as runners may not work as a community function and the minimal hosted runners won't be enough.

Switch to Gitlab + Hosted Runners

Just like the Gitlab proposal above, except that instead of relying on the community to voluntarily contribute runners, Hyperledger would pay for enough hosted runners to handle the full CI/CD load of the community. This has significant cost over the crowdsourced runners solution.

Pros

• Gives teams much more control over their CI/CD resources.
• Reduces how much we have to work with LFIT to make changes to the system. Gitlab will be largely self-service.
• Empowers teams to scale up their CI/CD resources through community marketing and/or direct sponsorship.
• Lowers the barrier to entry for teams to set up CI/CD.
• Gitlab does what we need and is free for us to use.
• Ensures that even small/new teams have access to significant CI/CD/Testnet resources.
• Increases the visibility into the CI/CD system for all of our projects. There will be one location where all of the builds for all of the projects will be coordinated from.

Cons

• Significant rework of the existing Fabric CI/CD system that may not happen.
• Kelly Olson Sawtooth and Grid rework? Not sure how much effort this will be.
• Iroha rework from their own Jenkins.
• Rework for Burrow, Indy, and Ursa but to a much lesser degree than the others because they are all using Gitlab.
• Does nothing to reduce our hosting costs or human administration costs. It may even increase our hosting costs because of the increased demand on the resources.