Hyperledger Caliper
Caliper is a blockchain benchmark framework which allows users to measure the performance of a specific blockchain implementation with a set of predefined use cases. Caliper will produce reports containing a number of performance indicators, such as TPS (Transactions Per Second), transaction latency, resource utilisation etc. The intent is for Caliper results to be used as a reference in supporting the choice of a blockchain implementation suitable for the user-specific use-cases. Given the variety of blockchain configurations, network setup, as well as the specific use-cases in mind, it is not intended to be an authoritative performance assessment, nor to be used for simple comparative purposes (e.g. blockchain A does 5 TPS and blockchain B does 10 TPS, therefore B is better). The Caliper project references the definitions, metrics, and terminology as defined by the Performance & Scalability Working Group (PSWG).
Key Characteristics
A unified blockchain benchmark framework. We provide a common layer to integrate with major existing blockchain framework/platforms, so that the same benchmarks can be run for different blockchain systems Some benchmark test environment will be provided to help different people run tests under the same environment, blockchain management tools like Hyperledger Cello could be integrated later to deploy and operate the environment. Also, users can use their existing environment and configure Caliper to run the test under the environment.
A commonly accepted definition of performance indicators. You cannot compare an apple and a pear directly unless some common criteria are set. We will work closely with PSWG to provide a common definition of performance indicators that users care about, such as TPS, latency, resource utilization, etc.
A set of commonly accepted benchmark cases. The goal of Caliper includes providing a set of easy-understandable benchmark cases so that each blockchain solution can be compared in various scenarios. This calls for much collaboration from PSWG, Requirement WG and other WG in Hyperledger community as well as blockchain practitioners to cover as many use cases that are of user’s interest as possible.
Project Management
Issue Tracking - https://github.com/hyperledger/caliper/issues
Repositories
Source code: https://github.com/hyperledger/caliper
Documentation: https://hyperledger.github.io/caliper/
Additional Materials
- Project health charts
- 2019 Hong Kong Bootcamp Tutorial (outdated, pre-publish version)
License Scan Results
License Audit
2019-03 Caliper License Audit Summary
2019-03 Caliper License Audit Full Report
License Audit Fixes
Hyperledger Caliper License Github Issues
Crypto Audit
Communication
Mailing List
Mail alias: caliper@lists.hyperledger.org
Mail archive: https://lists.hyperledger.org/g/caliper/topics
Mail subscription: https://lists.hyperledger.org/g/caliper
Chat (for questions and ephemeral discussions)
Questions are welcome and best asked in Hyperledger Discord. Learn more about Hyperledger Discord here, get the invite and check out one of the many Caliper project channels.
Meeting
There is a community call every four weeks on Zoom, at UTC time 3PM on Wednesday. The exact meeting details can be found in Hyperledger meeting calendar.
Minutes
October 17, 2018 minutes
October 24, 2018 minutes
October 30, 2018 minutes
November 6, 2018 minutes
November 14, 2018 minutes
November 21, 2018 minutes
November 27, 2018 minutes
December 4, 2018 minutes
December 19, 2018 minutes
January 16, 2019 minutes
March 27, 2019 minutes
April 3, 2019 minutes
- May 22, 2019 minutes
History
Recent space activity
Space contributors
- d_kelsey (223 days ago)
- Ry Jones (224 days ago)
- Sean Bohan (879 days ago)
- Attila Klenik (1462 days ago)
- Nick Lincoln (1863 days ago)
- ...