Iroha API for Clients
Status | IN PROGRESS |
|---|---|
Stakeholders | 武宮誠 Egor Ivkov Yuriy Vinogradov Vladislav Markushin |
Outcome |
|
Due date | Sep 11, 2020 |
Owner | Nikita Puzankov |
Background
Previous assumptions about Iroha Network and Web API in general were usage of iroha-client with encapsulation of underlying (plain TCP) protocols.
New needs from dependent projects require discussion of these assumptions and decision how to work with iroha network.
Iroha Bridge module provides an ability to connect different blockchains together and needs an ability to communicate between both sides via web interface.
Current Iroha approach is a proprietary TCP based protocol encapsulated inside `iroha_network` crate. `iroha_client` crate uses `iroha_network` inside and should be used itself by other applications including other blockchain solutions like `Polkadot`.
It turns out that `Substrate's` off chain workers provides only HTTP (more high level) API for communications with external resources (like Iroha Peer).
Requirements from Yuriy Vinogradov
HTTP compatible Iroha Peer API
HTTP client library for Instructions, Queries, Events
Rust client library (for Substrate's Off-chain workers)
JS Client library
Problem
Different clients environments have different restrictions - the strictest one is Substrate's Off-chain worker runtime which only supports HTTP 1.
Also DevOps team and Client side developers prefer to use JSON as a client-peer communication format. Some scenarios require to have an ability to combine Instructions API with Events API (send transaction and start consuming updates on it).
Solution
As a solution we propose this high-level design. HTTP and WebSocket protocols will be used for communication between Clients and Iroha Peers.
Iroha should provide HTTP API.
Endpoints:
Iroha Special Instructions
Iroha Queries
Iroha Events (under consideration - needed for Substrate)
MIME type of the HTTP API will be `application/json`
Clients build their own functionality or use 3rd party clients to establish HTTP connection, sign and construct transactions
JSON RPC or RESTfull approach should be discussed
Additionally Iroha should provide WebSocket API.
Endpoints:
Iroha Events
Iroha Special Instructions (under consideration - was requested by client-side developers)
Iroha Queries (under consideration - was requested by client-side developers)
JSON RPC can be used as high level protocol over WebSocket
Decisions
Iroha will provide HTTP and WebSocket API for clients
JSON format for messages encoding/decoding will be used
Alternatives
Stay with TCP and use middle ware for protocol and messages transformations - not user friendly
Use HTTP 2.0 - will not work with Substrate
Concerns
WebSocket connections should have configuration options to prevent bad performance of Iroha peers under high load
Assumptions
Bridge clients will use Substrate's Off-chain Workers with an ability to communicate over HTTP only
Risks
Performance regress `[8; 5]`
Action items
Comments
So my vote is for the dedicated HTTP server module on Iorha2 not included in the core but has an ability to proxy all ISI and queries if needed.
Looks good, let's wait for 武宮誠review and we will need to plan this feature with Vadim Reutskiy.
>Decisions
> Iroha Bridge module will provide additional HTTP Endpoints on top of existing Torii Endpoints
Nikita Puzankov Why you finally decided on this option if I described disadvantages in the message above and proposed another solution which is also included in the list. You did not mention any reasons why you made this decision and you did not discuss it with us.
Yuriy Vinogradov
It's RFC, Request for Comments - I just proposed this option as a primary because we agreed on it. And it is the solution that we discussed here Re: Iroha Network API, because it is mandatory for Bridge and DEX modules it should be packed with them. Maybe you was confused by part "Iroha Bridge module"?
All the reasons described in the RFC and RFC is a discussion - so please, discuss and comment
As I remember we agreed on that it will be in a separate module. And it was discussed in first comment to this RFC.
The HTTP server is not a part of the bridge. It is a separate functionality. It also will contain not only Bridge ISI, but generally all user ISI. It will be used by many modules - DEX, Bridge - it is already two different parts. I think that our future web wallet or even other types of wallets and apps also most probably will have the intention to use it. So, the inclusion of HTTP code to the bridge module seems not to make sense.
Rewrote
>Decisions
> Cloud Events will be implemented via HTTP instead of WebSocket because of Substrate's restrictions
Nikita Puzankov It is also not correct. substrate supports WebSockets as Kamil Salakhiev mentioned. So we don't have any restrictions here from Substrate side.
You may ask Kamil Salakhievfor proofs, but according to Off-chain workers documentation (I will paste a link one more time here https://substrate.dev/docs/en/knowledgebase/runtime/off-chain-workers#fetching-external-data) there is no WebSocket support. As I asked iceseerin the first issue about client for `no-std` more than a month ago about restrictions and your choice for runtime, as I ask you now - please, give me restrictions and supported abilities of your clients environments, because we already spend a lot of time discussing things without them
Ok, I also found only proof that Polkadot has websocket as a connection to the node, but did not found that we can use it from the pallet or from the web worker. Kamil Salakhiev Can you help to find some docs on how to use web sockets from the pallet or web worker?
https://www.substrate.io/kb/runtime/metadata - use Web socket to connect to the node.
We will need subscriptions for the Web also. I think while we decided to implement JS library independently it will be also easier for them to use WebSockets for subscriptions. For example, WEB should have the ability to be subscribed to transaction status changes.
Comment from Kamil'
Indeed we cannot use websockets from pallets, only http. When I said that ws and http are supported I meant that Polkadot’s RPC supports http and ws transports.
For subscription Web Sockets transport should be implemented. As far as I know HTTP would not allow to implement subscription mechanism
But it cannot be used by off-chain workers, why do we need web sockets then?
HTTP transport is not enough to implement subscription
But if it's the only API available, what would we do?)
Support two transports with the same API.
First priority would be HTTP, later we can introduce WebSockets subscription mechanism
The way it is done in Kagome and Polkadot is that we have two RPC ports: for http and websocket requests. Both use JSON-RPC api for processing requests. If you are interested such apis are described here: https://github.com/w3f/PSPs/blob/psp-rpc-api/psp-002.md.
I would love to have similar document for Iroha2
Kamil Salakhiev once again - who will need WebSockets? We are talking about HTTP API only for Substrate's off-chain workers, because they support only HTTP - why we need to add WebSockets if they are not supported?
use cases:
Subscribe to transaction status (stateless validated, stateful validated, committed or similar statuses)
Subscribe to new blocks
Subscribe to transfer transactions
No, it's functional requirements for "streams api" - who will use them and why it should be implemented via WebSockets?
Those streams apis will be used by clients. For example block explorer needs to be subscribed to the new blocks.
Read this article to understand why WebSockets. In short it allows avoid continuous polling + performance is higher than other solutions. If you have other options what transport to use we can discuss
We are trying to keep 2.0.0 requirements at minimum. So if we do not have such a clients, let's postpone it to 2.1.0.
As long as projects using Iroha2 are okay with it I don't mind to postpone it. Yuriy Vinogradov when do you think it will be critical to have streaming over websockets?
I think Yuriy Vinogradovand Egor Ivkovwill check abilities of WASM and WASI for browsers and will provide requirements on monday
As all this options were described here:
For the prototype we can use HTTP polling, but it will not be very efficient for Production, so we will need to think ways to handle it.
It will be not enough because workers need to receive this information in real-time?
> Iroha Bridge has an additional application as a proxy on top of Iroha API (as a standalone app or service mesh)
I think the standalone app is not an Iroha approach. Might be implemented as a module on iorha2. We also can expect that some blockchains which use other protocols like Protobuf might be interested to connect iroha2, so the module approach seems someway better because it allows to add/exclude networking protocols.
> Iroha Bridge module provides HTTP wrapper on top of Iroha API (on the Peer side)
If the bridge will provide only bridging functions HTTP interface then another HTTP server might be needed if we will need to provide HTTP API for other functions. So the Iroha HTTP server is better to be decoupled from concrete functionality to have the ability to support all need functions for other possible cases.
So my vote is for the dedicated HTTP server module on Iorha2 not included in the core but has an ability to proxy all ISI and queries if needed.