Purpose: I that section you can find information about executed performance test runs for Iroha, all parameters and results.
Parameters
Network configuration
| Iroha nodes | Virtual machines | Iroha memory | Postgres memory | Storage, per node | |
|---|---|---|---|---|---|
| Configuration A | 50 |
...
| 10 |
...
| (5 |
...
| nodes per machine) | 5 |
...
| Gb | 1 Gb | SSD, 100Gb | |||
| Configuration B | 4 | 2 (2 nodes per machine) | 5 Gb | 1 Gb | SSD, 100Gb |
| Configuration C | 7 | 7 (1 node per machine) | 15 Gb | 1 Gb | SSD, 100Gb |
Iroha configurations
| Max proposal size | Max rounds delay | MST enabled | MST expiration time | Proposal delay | Stale stream max rounds | Vote delay | Block store allocation | ID | |
|---|---|---|---|---|---|---|---|---|---|
| Iroha config A | 50 | 1000 | true | 1440 | 5000 | 60 | 5000 | Filesystem | SN |
| Iroha config B | 1000 | 500 | true | 1440 | 1000 | 100000 | 1000 | Filesystem | BK |
| Iroha config C | 50 | 1000 | true | 1440 | 5000 | 60 | 5000 | Database | SNDB |
Transaction configuration
- Ordered batch 1 – create an account, add tokens and transfer them
- CreateDomain
- CreateAsset
- CreateAccount
- AddAssetQuantity
- TransferAsset
- Ordered batch 2 – grant permissions to set account details and set some data
- GrantPermission
- SetAccountDetail
- Ordered batch 3 – set account details
- SetAccountDetail
- Ordered batch 4 – create MST account, add signatory and mint some tokens
- CreateAccount
- SetAccountQuorum
- AddSignatory
- AddAssetQuantity
- Ordered batch 5 - send MST transfer (from an account created by ordered batch 4)
- TransferAsset
- Ordered batch 6 - get pending transactions (created by ordered batch 4) and confirm them
- GetPendingTransactions
Test types
- Soak testing: running Iroha network with 80% of peak load over a long time period of time (more than 24 hours) – to emulate behaviour under normal load on an extended period of time
- Spike testing: running Iroha network with 100% of peak load over a shorter time period (around 1 hour) – to emulate behaviour at peaks of request during "rush hours"
Metrics
During performance testing, our goal is to understand non-functional characteristics of running Iroha network in terms of speed and efficiency.
For a clear understanding of such characteristics, we are going to collect the following metrics:
- Step 1. Obtaining information about peak load that Iroha network can handle:
- Define the highest amount of transactions for a peak load of Iroha network during one hour (on the peak load the network should work stable, without significant deterioration of other characteristics: TPS, memory consumption, transaction finalization time, etc.).
- Step 2. Collecting data about Iroha characteristics (for each test type)
- The maximum amount of successfully finalized transactions per second (TPS)
- Average time of transaction finalization (from the moment when the transaction reached the Torii port, until the moment when the transaction is marked as finalized and sent to the subscribers via opened streams)
- Average memory consumption by Iroha nodes (first 10% of time interval should be omitted to minimize the influence of "heating up" stage)
- Average memory consumption by Postgres storage (first 10% of time interval should be omitted to minimize the influence of "heating up" stage)
- Average CPU utilization by each node (first 10% of time interval should be omitted to minimize the influence of "heating up" stage)
- Amount of errors from the Iroha network
- Amount of not finalized or missed transactions
Sources
...