net: Prototype a TLS convenience API based on sockets

[pfalcon]

As discussed during Zephyr Networking Forum meetings, mentioned in #5854 , etc., currently there're not even examples, far less best practices, on using TLS communication over BSD Sockets. (The same applies to DTLS, but this ticket is dedicated to TLS specifically.)

This ticket is intended to bootstrap/track this work, and establish requirements for implementation.

[pfalcon]

So, there could be 2 approaches:

1. Make it more or less specific to TLS, or
2. Make it more generic and reusable

I'd lean towards 2, because going route 1 unlikely to save much, but doing adhoc things is much less "interesting".

So, the idea of 2 would be to introduce a concept of generic "stream" object, with corresponding interface of operations. Implementation of that will be polymorphic per a specific stream type, and backed by a typical "vtable" of function (method) pointers. A stream object would start with a pointer to such a vtable (shared by all instances of a stream of particular type), followed by type-specific data. Right, poor-man's C++. I'm not sure if we'll want foo->method(foo, ...) syntax to call methods, or wrap them in macros like STREAM_CALL(method, foo, ...). Arguably, native C syntax is cleaner even if there's foo repeated.

The operations of the stream interface would be:

• read - As it's stream, not just socket, corresponding terminology is used (read, not recv). The signature is the typical POSIX one.
• write - Ditto.
• flush
• close

I'd like to specifically draw attention to the importance of flush() operation. This isn't usually considered with native sockets, but would be crucial for "userspace" objects like we discuss here to get proper data transfer semantics. Actually, one of my biggest criticisms of TLS support implementation in net_app - that there's a "bookkeeping" thread per TLS connection - seems to be routed in the lack of flush operation, so the purpose of that thread is to flush app's data on timeout. Why? App can tell itself when data needs to go down the wire. (Maybe there're other uses of that thread; my point is that I implemented TLS support without extra threads, and it worked.)

There's also a close() method, for convenience, but of course, there's no open() or similar method in the stream interface, by the same reason that there're no virtual constructors in C++ - when you create an object, you always create a specific type of object, and then perform polymorphic operations on it.

Finally, there're 2 types of stream objects - "leaf" streams, whose constructors accept params to create a stream, or at least to convert object of another type into a stream, and "wrappers", which take another stream as a param, and apply "transformation" on it. A socket stream would be an example of 1st kind. TLS would be example of 2nd. This is part of "choice 2, make it generic and reusable" scenario. For example, we could serve HTTP over SPI - why not, just write a stream implementation for SPI. Also, we could serve HTTP over TLS over SPI, nothing new would need to be written "over TLS" part will be the same as for a socket stream.

[pfalcon]

@GAnthony, @jukkar, @tbursztyka, @mike-scott, @mbolivar : Comments/criticism welcome.

[galak]

@d3zd3z adding David to this as well

[pfalcon]

An interesting issue in regard to stream concept is poll() support for them. There can be 2 approaches:

1. Say that polling is not part of stream interface, and clients interested in poll()ing, need to maintain correspondence between underlying socket fd and a stream object themselves (and used the underlying fd for poll() operations).
2. Try to somehow address that on the stream interface level. On obvious way would be to add getfd() method, but fairly speaking, that's were proverbial C++ overheads start. Another approach would be to make underlying fd a "data interface", which has less (performance) overhead, but "fragile", and potentially more memory overhead. E.g., one way would be to make stream object layout to start with vtable pointer (will be there anyway), followed by the fd field (then it will be duplicated for each wrapper stream - again, as an alternative to a change of virtual functions returning just a structure field).

[galak]

@d3zd3z pointed at:

https://www.openbsd.org/papers/linuxconfau2017-libtls/#slide-0

https://github.com/libressl-portable/openbsd/tree/master/src/lib/libtls

Something to look at for ideas.

[mbolivar]

@mbolivar : Comments/criticism welcome.

Thanks! I'd be interested to see the results, but I'm afraid my input isn't likely to be useful.

[pfalcon]

Thanks! I'd be interested to see the results, but I'm afraid my input isn't likely to be useful.

Ok, I assume that at least means you don't see anything terribly wrong with it ;-).

[GAnthony]

Keeping in mind the need to accommodate the secure socket offloading, I'd like to share some links which were shared in an earlier internal email thread:

The SimpleLink SDK proposes a secure socket design that I thing is
quite elegant.

All secure connection setup is done via setsockopt() calls, then the
rest of the application logic
is the same as if using unsecured sockets. The security handshakes occur
"under the covers" during the socket connect() and accept() calls.

See Section 2.2, "Secure Socket Layer",
http://www.ti.com/lit/an/swra509a/swra509a.pdf

For an example application, see:
http://dev.ti.com/tirex/content/simplelink_academy_cc32xxsdk_1_13_00_29/modules/wifi_secure_socket/wifi_secure_socket.html

The TCPClient function at the bottom shows some actual code.

Even if we decide not to use socket-like APIs for TLS setup the way TI does, the main point here is that, though the Zephyr NET_OFFLOAD solution would allow secure socket data transmission to be offloaded, the setup (at least for SimpleLink) must occur via these setsockopt calls, which would probably need to be hooked from this new proposed "TLS convenience API".

For that reason, I think the connection/configuration APIs should be included in the scope of this proposal, similar to what is done here:
https://www.openbsd.org/papers/linuxconfau2017-libtls/#slide-16

[GAnthony]

A couple comments on creating a new "stream" API:

1. First we argue that networking protocols should be based on sockets; now the argument is no, they are to be based on generic stream objects. And if that's the case, should this also be the new interface for all device drivers?

That's OK, but we should realize, if that's the case, it's a much larger scope than making a "TLS convenience layer" ;-)

2. The connection aspect: how would listen, accept: work with streams. Would UDP or RAW_SOCK work under the stream interface?

[pfalcon]

Let me quick reply to these so far:

First we argue that networking protocols should be based on sockets; now the argument is no, they are to be based on generic stream objects.

No, we don't argue all that, at once. The arguments are actually layered:

1. The first argument is that sockets is the de-facto and de-jure (POSIX) API for network programming, used for decades, and that the best approach is to have that as system-level API, to get the benefits of that standardness and history (which includes porting existing application written against Sockets API, which are many).

2. However, during few recent years, TLS is getting more and more important and prominence, which is above-system-level interface. And the second argument is that applications should be able to deal with either plain or TLS-wrapped protocols with as at least as possible code duplication and separation (i.e. 2 different implementations of the same protocol shouldn't have 2 different implementations - one TLS-enabled and one not). So, we construct appropriate interface, sitting above sockets to achieve that.

A key point is that 1) (sockets API) is "one for everyone out there", while 2) (Zephyr's TLS API-on-top-of-sockets) is a just a particular implementation of a TLS API, among many available. They're layered like that, and sufficiently independent, e.g. someone may use another TLS API, or another socket implementation (e.g. offloaded) with the proposed TLS API.

[pfalcon]

2. The connection aspect: how would listen, accept: work with streams.

Simple answer: they won't. The stream interface abstracts bidirectional peer-to-peer communication in terms of bytes (vs e.g. in terms of records). Socket interface is superset of stream interface, but we don't define it. That means that a server will be implemented in terms of plain sockets, and once connection is accepted, only then in will be wrapped with TLS (that's of course how it works anyway, the point is that we don't try to abstract that in any way).

Would UDP or RAW_SOCK work under the stream interface?

Nope, because they aren't "communication in terms of bytes". UDP would be "communication in terms of records (datagrams)", but I'd need to do more homework on DTLS. One concern, as pointed by @d3zd3z is that DTLS-enabled protocols are/seems to have tighter coupling between layers. We may need to define a separate "interface" for UDP/DTLS.

As for RAW_SOCK, IIRC, they allow to send full IP packets (with headers, etc.). How that would map onto (D)TLS is unclear, I never heard about that. If/when someone formulates the requirements how that's supposed to work, it will be yet another, separate usecase.

[pfalcon]

Sorry, missed to answer this one:

And if that's the case, should this also be the new interface for all device drivers?

Conceptually and semantically yes, more specifically, I think we should follow POSIX I/O model, with read/write operations and their properties (like short read and short write semantics).

However, it apparently would be far-fetched to impose interface outlined here ("streams vtable") on all device drivers. At the very least, besides "stream of bytes" model, there's another model - "stream of records" with its own semantics differences.

Trying to devise "grand unified interface" (implementational interface) for everything will lead us to bloat, e.g. https://en.wikipedia.org/wiki/STREAMS . I hope we go in the opposite direction - reuse existing experiences and best practices to design an interface scaled down for a bare minimum required to implement a specific functionality adequately.

[pfalcon]

@GAnthony :

need to accommodate the secure socket offloading
All secure connection setup is done via setsockopt() calls

Right, so if TI API allows to create a plain socket and then convert it to TLS via setsockopt() calls, then this proposal covers such an implementation. We probably would need to go further C++ way, and add a bit of RTTI, in the shape of e.g. ->get_type(), to allow different stream types to differentiate each other (for full generality).

The flow this proposals entails is something like:

int sock_fd = socket(...);
stream *sock_stream = make_sock_stream(sock_fd);
stream *tls_stream = make_tls_stream(sock_stream, ...);

mqtt_client(tls_stream, ...);

So, as long as make_tls_stream() recognizes passed-in stream as backed by TLS-offloadable socket, it can configure it as need.

For that reason, I think the connection/configuration APIs should be included in the scope of this proposal, similar to what is done here:
https://www.openbsd.org/papers/linuxconfau2017-libtls/#slide-16

So, as I mentioned during today's discussion (on Linaro side for context), let's keep scope of the current work manageable. Let's remember that our immediate aim of this work is "how to make socket-based protocol implementations which can work with both plain and TLS sockets with as close to 100% code reuse as possible". So, we can and will "prototype" TLS connection/configuration API, but can't "standardize" on all aspects of it (in the timeframe posed, which is to have a working proof of concept in 1 month) - not without close and parallel involvement of security people, but even then I doubt we can "standardize" on all aspects of it (like e.g. offloading handling).

[GAnthony]

Right, so if TI API allows to create a plain socket and then convert it to TLS via setsockopt() calls,

http://www.ti.com/lit/an/swra509a/swra509a.pdf, section 2.2.6, hints that's possible, but the user must explicitly re-invoke accept() or connect() to upgrade security. It's not the normal mode of operation.

But why would that be necessary? Wouldn't the IP protocol normally already know that it intends to deal with a secure socket, and create such a socket from the outset? (I know there are exceptions to the rule, but why not design for the expected case?). It seems the client must know it will be a secure socket, because it's calling stream *tls_stream = make_tls_stream(sock_stream, ...);

So, the socket could be created with something like fd = socket(AF_INET, SOCK_STREAM, PROTOCOL_SEC_SOCKET), where 'PROTOCOL_SEC_SOCKET' could be our flag to prepare a secure socket, ready to be configured with subsequent setsockopt() calls.

Note: From the POSIX standard on socket(): "The protocols supported by the system are implementation-defined."

[pfalcon]

but the user must explicitly re-invoke accept() or connect() to upgrade security.

That's not my reading of that document:

The user must trigger the SSL/TLS handshake explicitly, irrespective of the socket connection (triggered by either sl_connect or sl_accept APIs).

"Irrespective" should give enough hints, but let's call it non-conclusive still. http://www.ti.com/lit/ug/swru455d/swru455d.pdf section 6.5.7 gives more specific information. It's still a bit confusing, because it start with talking about STARTTLS which is application protocol level feature, but following should clarify it:

Calling sl_SetSockOpt with the STARTTLS option triggers the NWP, in client mode, to send the client HELLO message, and in server mode to wait until the client HELLO message is received.

(Assuming "client HELLO" is TLS' ClientHello: https://en.wikipedia.org/wiki/Transport_Layer_Security#Basic_TLS_handshake)

So, TI SimpleLink should support normal socket vs TLS layering (while provides convenience API to do it all in one go).

[pfalcon]

Wouldn't the IP protocol normally already know that it intends to deal with a secure socket, and create such a socket from the outset?

Responding to that literally, IP protocol doesn't know anything about secure sockets, it just carries opaque data.

It seems the client must know it will be a secure socket, because it's calling stream *tls_stream = make_tls_stream(sock_stream, ...);

Right, so it's the higher(est) level which has (can have) the complete vision of what happens on the lower levels. And yes, we could have stream *tls_stream = make_ti_sl_tls_stream(...);, this proposal exactly allows that. But as mentioned above, such things outside of the primary scope (of the first phase) of this proposal. Because we won't be able to find fully general solution in a month ;-).

Note: From the POSIX standard on socket(): "The protocols supported by the system are implementation-defined."

Yes, but how sockets got popular is that there's common, portable (sub)set of protocol which work on (almost) every system. There's no PROTOCOL_SEC_SOCKET on other systems beyond TI SimpleLink, so it doesn't make sense to base the design on it. Or we'll really have a contradiction, saying "sockets is standard API, it allows for portability across systems", and as the next step add non-standard, non-portable thing to its implementation.

As I tried to elaborate above, I propose to clearly split layers: sockets don't belong to us, we just implement them as faithfully as reasonably possible with our constraints. Above sockets, the space belongs to us, and we implement things there to best suit our needs (which are apparently generality and minimal code size).

[GAnthony]

So, TI SimpleLink should support normal socket vs TLS layering (while provides convenience API to do it all in one go).

Looks like it might work. Proof will be in testing it out.
Agree, it will be good to avoid vendor specific flags in the socket calls.

[pfalcon]

Result of first iteration on implementation was posted as #5985 , which should give better look&feel of the proposed API.

What was not discussed in this ticket is allocation matters, but of course, they surface in the real usage. The basic is that space for a particular type of stream object is allocated by client, and there's an initializer which takes an allocated pointer. But for server use, we'd need to deal with array of objects. It's tempting to leave that to client still, but would need to see how cumbersome that looks when a server apps is converted, that's next in queue.

[d3zd3z]

The Wikipedia page on TLS has an interesting comment about why this API is hard to do:

TLS and SSL do not fit neatly into any single layer of the OSI model or the TCP/IP model.[7][8] TLS runs "on top of some reliable transport protocol (e.g., TCP),"[9] which would imply that it is above the transport layer. It serves encryption to higher layers, which is normally the function of the presentation layer. However, applications generally use TLS as if it were a transport layer,[7][8] even though applications using TLS must actively control initiating TLS handshakes and handling of exchanged authentication certificates.

In addition, Zephyr is likely to run into additional complexity because certificates and certificate chains are often too heavy to implement in a constrained device, and it is necessary to use other ciphersuites, such as PSK or bare pub/priv keys.

[d3zd3z]

As far as allocation, the underlying cipher, esp the asymmetric algorithms used for certificate management expect there to be an implementation of malloc/free. mbed TLS includes an implementation (that in my experience is significantly better than the tiny one included in Zephyr). Needless to say, though, that determining how much space to give to this is going to be an issue to setting up TLS/DTLS.

[laperie]

Per our recent discussion adding @pfl as an assignee

[nashif]

AFAIK this is already completed.