Library usage for HTTP and WebSocket protocols
Background
From the start of the development of Iroha 2, we tried to limit the number of dependencies in order to ensure higher security. With the acceptance of Maintenance Endpoint RFC, the work on the implementation of HTTP and Web Socket API has been started. Taking into account the size of the team and the time constraints it seems reasonable to use already existing libraries for both of the selected protocols (HTTP, WebSocket), and therefore add these dependencies to Iroha 2. This RFC discusses the possibility of introducing these dependencies and the selection of these libraries if they are decided to be added.
Action items
- Egor Ivkovimplement the maintenance endpoint with the selected libraries
Problem
Should we use external libraries for the HTTP and Web Socket protocols or provide our own implementation of them? If we use external libraries, then which ones?
Solution
Decisions
It is suggested to use external libraries for both of these protocols. The pros and cons of this decision are shown below
Pros | Cons |
---|---|
|
|
Requirements for Libraries
These are the high-level requirements that seem reasonable to propose for libraries that we will use:
- Async I/O - as Iroha2 is heavily async and relies on executors for low-level thread management.
- A relatively small number of new dependencies - as new dependencies introduce potential security risks.
- Library rather than framework - as Iroha2 has already a rather specific structure and should not change its style to fit the framework.
- It should be possible to upgrade HTTP connection from HTTP library with WebSocket
- Free open license
- HTTP 1.1 support - as Substrate offchain workers do not support HTTP 2
Proposed Libraries
The proposed libraries are the following:
Protocol | Library | Description |
---|---|---|
HTTP | httparse | A push parser for the HTTP 1.x protocol. Avoids allocations. No copy. Fast. |
route-recognizer | Recognizes URL patterns with support for dynamic and glob segments. Can be easily replaced with our custom solution later. Used to speed up the development process. Also doesn't have any dependencies. | |
Web Socket | async-tungstenite | Asynchronous WebSockets for async-std, tokio, gio and any std Future s runtime. Based on tungstenite crate. |
The usage of these libraries would, in general, mean that we will implement our own web server based on TCP streams, but the messages will be parsed with the help of HTTP library. When the upgrade request for web socket will be received, then the connection manager will be passed to async-tungstenite library.
Alternatives
Alternative | Description | Why not chosen |
---|---|---|
tiny_http as HTTP server library | Low-level HTTP server library | No async support |
http as HTTP type helper library | A general-purpose library of common HTTP types. | Does not support parsing from streams or raw bytes, which is a functionality that we might spend a lot of time writing if it is not supported out of the box. |
async_h1 as HTTP server library | Asynchronous HTTP/1.1 parser. | Does not support WebSocket upgrade. |
hyper as HTTP server library | A fast and correct HTTP implementation for Rust. Actually the fastest HTTP server library according to techempower. |
|
h2 as HTTP server library | A Tokio aware, HTTP/2.0 client & server implementation for Rust. |
|
http-service as HTTP server interface library | The crate http-service provides the necessary types and traits to implement your own HTTP Server. | The crate mainly provides the interface for the servers to implement the same methods, it might be good for architecture in general, but it is an additional dependency and our priority is to only add absolutely necessary ones. |
tide as http server | A modular web framework built around async/await. |
In general it might have been a good choice if they had finished web socket support by now. |
actix / warp / rocket as web framework | Popular web frameworks with both warp and rocket being built on the hyper library, |
|
Custom http and web socket libraries | It is possible to implement a fully custom solution. | There are already existing solutions which correctly implement the standards and are widely used. The team will be able to focus on our unique functionality. |
Concerns
The chosen solution might take more time to implement but at the same time, it gives us flexibility and minimal increase in dependencies.
Assumptions
WebSocket feature is essential to us and we should support it out of the box.
Risks
There are potential risks in adding new dependencies which might have security-related flaws.
Additional Information
Grin developers community had a discussion on this topic which might be worth reading. Grin is a Rust implementation of Mimblewimble blockchain format and protocol.