simplexmq/rfcs/2021-02-28-streams.md

Streams

Managing dedicated SMP queues for fast synchronous communication

Problem

SMP agent protocol implementation provides the most secure way to distribute keys achieving the following qualities:

• compromising the sender agent/device allows to send messages, but not to read them.
• compromising the recipient agent/device allows to receive messages, but not to send them (neither public encryption key is stored - TBC if public key can be restored from the private - nor server authentication key is available).
• compromising the server does not expose any information about messages content, as encrypted section has the same size.

The current hybrid encryption scheme uses a new symmetric key for each message, and because the symmetric key is not persisted at any point, it is difficult to obtain it and send counterfeit messages on behalf of the sender (even in case both the server and recipient are compromised).

There are 2 downsides of the current scheme:

1. RSA encryption/decryption is relatively slow even for 2048 key size and gets much slower for larger key sizes. It is not a problem for the chat messages (and any content updates) that happen infrequently, but it makes the transmission of large files and any other streaming communication slow.

2. If the large file or voice/video calls were to be sent via the same queue as normal messages/content updates, the server and any passive observer would be able to understand when such transmissions happen (even if performance was not a problem).

Proposed solution

For every fast/dense data transmission use a separate short-lived connection with dedicated SMP queues that the agents would provision to receive such transmissions.

Such connections that can be unidirectional or duplex can be managed by agents via additional set of agent commands using "stream" abstraction.

The protocol to create and manage such stream could be the following:

1. The sender client notifies the agent about the stream it has to create for a specific duplex connection (the connection must be duplex for it to be possible): <conn_alias> SNEW <size> (where conn_alias is the existing duplex connection, size is the size for the data to be transferred; 0 for unbounded streams), the response is <conn_alias> STREAM <stream_id> <size>.
2. The sender agent sends to the recipient agent the "envelope" STREAM <stream_id> <size>
3. The receiving agent would notify the client about availability of the stream by sending <conn_alias> STREAM <stream_id> <size> notification.
4. Receiving client would send <conn_alias> SJOIN <stream_id> command to the receiving agent.
5. Receiving agent provisions the SMP queue on the random SMP server (we planned to remove SMP server from NEW command anyway). Possibly, there will be a limited list of servers that support streams (that is un-throttled number of messages), and stream support can be communicated via welcome header and change with AUTH SMP command.
6. Receiving agent sends the envelop to sending agent SJOIN <stream_id> <invitation> (where invitation has the same format as in out-of-band message, but it has symmetric AES key and IV instead of asymmetric RSA key - the encryption scheme should codified in the invitation to make it generic).
7. Sending agent and receiving agent confirm and secure SMP queue as usual. This is still not solving the problem that the sending agent should sign each command. We might extend SMP protocol to create a new queue type that allows authenticating the sender only once per TCP session so that messages can be sent without signing - the server can see that these queues are special anyway, as there will be much faster traffic there.
8. Sending agent, once the queue is confirmed notifies the sending client that the stream is ready to accept data by sending notification <conn_alias> SREADY <stream_id> <max_chunk_size>.
9. Sending agent client can now send packets into the stream up to total <size> limit if it was specified by using command <conn_alias> SPUT <stream_id> <chunk_no> <chunk_size> <binary> (the chunk size should not be bigger than max_chunk_size, the agent would pad it to this size before sending to the receiving agent)
10. Sending agent would send the chunk to the receiving agent in CHUNK <chunk_no> <ts> <chunk_size> <binary> envelope, where the first 4 bytes in decrypted binary is chunk size.
11. Receiving agent would send the chunk to the client as <conn_alias> SDATA <stream_id> <chunk_no> <ts> <status> <chunk_size> <binary> (possibly, with three IDs and timestamps as with messages).
12. Both the receiving and sending clients can terminate the stream with <conn_alias> SCLOSE <stream_id> command. The agent would send SCLOSED <stream_id> envelope and the opposite client would receive <conn_alias> SCLOSED <stream_id> notification - no more data will be accepted into this stream.
13. If the stream was bounded with some size then both the sending and receiving client would receive SCLOSED notification after the last chunk of correct size was sent (and the incorrect chunk would be rejected).
14. If the stream was interrupted then the recipient client can request resuming this stream using <conn_alias> SRESUME <stream_id> <from_chunk_no>, the recipient agent would send "envelope" SRESUME <stream_id> <from_chunk_no> <invitation>.

To be clarified:

1. Possibly the internal stream ID used by clients should be different from stream ID communicated between agents, and in any case it should be connection scoped (same as with messages).
2. There seems to be no need to rotate symmetric keys, disconnections can be handled.
3. Same as we plan for redundant SMP queues with one connection, agents could use multiple queues per stream to increase the bandwidth and/or transmission reliability (e.g. for file transfers, where interruptions are not problematic multiple queues can be used in parallel, while for calls agents can duplicate all transmitted bytes so that losing some queues or TCP disconnects do not result in communication interruption).
4. Possibly the stream creation could be initiated by the receiving agent, to have it more similar to connection creation procedure.