Project Plan - HL Iroha and HL Cactus Integration

Owned by Han Xu
Last updated: Aug 30, 2021 by baziorekVersion comment
6 min read

Overview

The main goal of the project is to create a documented Iroha (1.x) package for the Cactus project, which demonstrates the interoperability between Iroha and Cactus. The project also includes 1 documented examples of integration between Iroha and Cactus. The documentation will benefit the community so that people can understand this project.

Mentor and Mentee

MentorMentorMentee

Peter Somogyvari

PDT (UTC-7)

peter.somogyvari@accenture.com

Grzegorz Baz

CEST (UTC+2)

g.bazior@yodiss.pl

Han Xu

EST (UTC-5)

hongkexuhan@gmail.com

Communication channel: Telegram/ RocketChat/ Public Channels/ Emails/ Microsoft Teams

Project repository: 

https://github.com/hyperledger/cactus (main)

https://github.com/hyperledger/iroha

Project pull requests:

https://github.com/hyperledger/cactus/pull/1183

https://github.com/hyperledger/cactus/pull/1169 

Practical presentation of completed project (not the same presentation as on project site) - how to use result of the internship:

Project Plan

Schedule

Task

Deliverable

Week 1 - Week 3

Jun 1 - Jun 18

Jun 18: 1st Quarter Evaluation

  • Install Ubuntu 20.04 LTS and set up environment.
  • Build Cactus.
  • Build Iroha.
  • Understand existing source code.
  • Start the Iroha and Cactus integration.
  • Project Plan.
  • Skeleton for Iroha and Cactus integration.

Week 3 - Week 6

Jun 19 - July 9

July 9: Midterm Quarter Evaluation

  • Implement the Iroha and Cactus integration.
  • Write tests for the Iroha and Cactus integration.
  • Write documentation.
  • ~50% of the Iroha and Cactus integration package.
  • Technical documentation for the package.

Week 6 - Week 10

July 10 - July 30

July 30: 3rd Quarter Evaluation

  • Write tests for Iroha and Cactus integration.
  • Write documentation.
  • Completed package for Iroha and Cactus integration.

Week 10

July 31 - Aug 10

Aug 20-31: Final Evaluation

  • Wrap up documentation.
  • Final report of the project.
  • Final presentation of the project
  • Code for unit tests and integration tests. *
  • Technical documentation for the entire project.
  • Final report.
  • Final presentation.
  • 1 example of Iroha and Cactus integration. (Nice-to-have)
  • 2 examples of Iroha and Cactus integration. (Optional)
  • 2 examples of Iroha and Fabric integration. (Optional)

* Will wrap up all codes-relevant stuff by Aug 5.

Timeline

Week

Task

Weekly Slide

1 (Jun 1 - Jun 6)
  • Attend onboard meeting
  • Attend 6/4 Iroha community meeting
  • Read the project wiki page

W1 Slide.pdf

2 (Jun 7 - Jun 13)
  • Install Ubuntu 20.04 LTS
  • Set up environment
  • Build Cactus
  • Build Iroha
  • Project Plan

W2 Slide.pdf

3 (Jun 14 - Jun 20)
  • Skeleton code for Iroha and Cactus Integration
  • Learn Typescript via Udemy course

W3 Slide.pdf

4 (Jun 21- Jun 27)
  • Finished the Typescript Udemy course
  • Stripped out files and functions for Iroha plugin
  • Worked on current Tx test
  • Tried to compile the package

W4 Slide.pdf

5 (Jun 28- July 4)
  • Made sure the project compiles
  • Used dockerode to run Iroha+Postgres docker image properly
  • Modified based on v1.2.0 Iroha Docker image; posted on DockerHub
  • Tried to run Iroha’s JS library

W5 Slide.pdf

6 (July 5 - July 11)
  • Inquired Iroha community about Iroha docker health check + Iroha JS library response
  • Switched to Iroha’s TS library, got output from all commands/queries
  • Rewrote Cactus’s connector to manipulate Iroha’s TS library (able to pass commands and queries and get back response)
  • Tried almost all Iroha’s commands+queries. (a shared Google Sheet is used to track the progress)
  • Redesigned test cases, pushed to Github.

W6 Slide.pdf

7 (July 12 - July 18)
  • Switched from connector to apiClient to run transactions for everything
  • Progress: Commands -> 14/20; Queries -> 12/15; in total: 26/35 (~74%)
  • Made code cleaner (used enum for Iroha commands/queries)

W7 Slide.pdf

8 (July 19 - July 25)
  • Implementation progress: Iroha commands -> 19/20; Iroha queries -> 14/15; in total: 33/35 (~94%)
  • Working on rebasing
  • Took a look at the Iroha Testnet 
W8 Slide.pdf
9 (July 26 - Aug 1)
  • Finished rebasing
  • Drafted pull request with Peter; fixed most feedback from Peter (11/12)
  • Finished my own Dockerfile; shrinked the size to 120MB; 85% improvement (thanks to help from Iroha community and Greg)
  • Delivered presentation to the Iroha community this morning
  • Explored the issue behind grpc error “GOAWAY unavailable” (get pending transaction)

W9 Slide.pdf

10 (Aug 2 - Aug 8)
  • Responded to all feedback from Peter for my 2nd PR
  • Look into the pending transaction issue

W10 Slide.pdf

11 (Aug 9 - Aug 15)
  • Finished Iroha-Iroha transfer example (including passing randomly generated admin/node key pairs)
  • Finished the irohaBaseConfig interface for the request
  • Tweaked Iroha docker healthcheck via Python SDK
  • Tried best to remove magical strings (e.g., changing the strings to enum; randomly generate admin/node key pairs; uuidv4())
  • Responded to most of the feedback

W11 Slide.pdf

12 (Aug 16 - Aug 20)
  • Fixed the crash issue with Iroha-Iroha transfer example (will not crash if pass in no
    parameter)
  • Updated comments and documentation (e.g., readme.md)
  • Removed magical strings for all files; responded to all PR feedback
  • Completed the “Recommendations for future work” list
  • Added tls:boolean option inside the baseConfig interface (tls is not implemented)

W12 Slide.pdf

Process

  • Communication and updates:
    • Telegram/ RocketChat/ Public Channels/ Emails/ Microsoft Teams are used as tools.
    • Weekly check-in meetings between mentors and the mentee.
  • Quarterly evaluations on Jun 18/ July 9/ July 30/ Aug 20.

Recommendations for future work:

CategoryIndexCurrent implementation and issueExpected improvementsDifficulty level *Priority *Github Issue
Iroha connector plugin








1

gRPC TLS option is not implemented.

Currently, only able to pass in TLS parameters( TLS Cert/ TLS Key/ TLS Port) to iroha test ledger, but those parameters are immediately discarded.

Implement gRPC TLS support for Iroha test ledger.

(Refer to Cactus PR #1190 as an example. )

3/51Link

2

Currently the parameters are hardcoded ( i.e., we can only support new parameters if we recompile the code of the connector and then publish a brand new release of it.) Parameters should be more generic in the future so that parameter changes can be done dynamically.3/52Link
3

fetchCommits is not implemented. 

It is inside the plugin-ledger-connector-iroha.ts, but unable to test it due to its unique characteristic of streaming responses.

fetchCommits could be implemented as something similar to Besu connector's WatchBlockV1.3/54Link

4


4a. There is no implementation for smart contract.

4b. InvokeContractV1 is not implemented due to the lack of smart contract.

4c. Call Engine is not implemented. (because Call Engine calls smart contract)

For Call Engine, we intentionally reject the transaction within the ledger, and then validate the rejection in the test case. 

4a. Implement smart contract once Iroha fully supports smart contract.

4b. Implement InvokeContractV1 for Iroha connector once Iroha fully support smart contract.

4c. Given smart contract is supported by Iroha, implement Call Engine and test it.

3/5

5 ( will be important once Iroha fully supports smart contract)

Link

Link

5

setSettingValue is not implemented because this command can only be called inside genesis.block file.

(Current connector implementation returns HTTP 405 Error in plugin-ledger-connector-iroha.ts)

When Iroha is able to run SetSettingValue outside of genesis.block, implement and test it.

(It is being said that the Iroha team plans to support running SetSettingValue outside of genesis.block.)

2/57Link
6

removePeer is not fully tested. 

A valid Iroha testnet needs to be constructed to test removePeer.

Able to manipulate Iroha testnet ( An Iroha testnet usually composes of >= 3 Iroha nodes).

Then, construct Iroha test within the test case to test removePeer.

4/56Link

7

getPendingTransaction is not fully tested.

There is an issue with producing a pending transaction: the code will get stuck and fail the test suite. It seems like Iroha ledger itself is struggling to generate the pending transaction. (https://jira.hyperledger.org/browse/IR-1010) In other words, it seems to be an issue with Iroha ledger itself instead of the Iroha Javascript library.

Able to produce a pending transaction in the test case.

Able to validate the pending transaction via getPendingTrasaction query.

4/56Link
8

Prometheus exporter metrics integration is not implemented.

Add prometheus exporter to the Iroha connector plugin.

2/56Link
Iroha-Javascript library1

1a. Utilized iroha-helpers-ts since the original Iroha Javascript library gives “undefined” as output. 

However, iroha-helper-ts is based on an old Iroha Javascript library (12/23/2020 release), which does not support some new features.

1b. Due to this, currently, one argument for compareAndSetAccountDetail, check_empty, is not dealt with inside the plugin-ledger-connector-iroha.ts file. The run-transaction test case also fail to use this argument.

1a. Build our own iroha-helper-ts library (outputs tx status and tx hash) based on the Iroha Javascript library, so that it could be upgraded to match the most recent Javascript library version. Moreover, our own iroha-helper-ts's output could be further optimized to suit test cases better. 

1b. After developing our own iroha-helper-ts library, correct compareAndSetAccountDetail's arguments. Also, correct the test case accordingly.

3/53

Link

Link

Iroha docker container

1

The test cases rely on a modified Iroha v1.2.0 all-in-one (AIO) docker image.

However, Iroha version 1.2.0 is outdated.

Upgrade Iroha to the latest version to improve its functionality and performance.2/56Link
2

An Iroha Python SDK is embedded inside the Iroha AIO docker image for docker healthcheck. Although this healthcheck mechanism works fine, it makes the docker image ~100MB larger. 

Once the Iroha team introduces the gRPC healthcheck/ Iroha metrics page in a stable release (e.g., v1.2.2), implement healthcheck mechanism through gRPC healthcheck or curl the metric page. 
`curl http://127.0.0.1:8080/metrics`		2/56Link
3Each Iroha docker container relies on a corresponding Postgres database container to store information. Replace the Postgres database docker container with RocksDB, which needs just one folder ( a docker volume) to keep data between different runs of image.3/56Link
Documentation1

For docs/source/support/iroha.md, the link to the transaction test is not guaranteed to be valid.

Once the current Iroha connector plugin PR gets merged/released, update the test link to be a valid link. 1/56Link

*For the difficulty level, 1 means the easiest, 5 means the most challenging.

*For the priority listing, 1 means the most important, 7 means the least important. Important ones have been marked as red.