RFC: async/await + uselect + i2cslave

[notro]

I've had a longterm goal to try and use i2cslave with async/await since it looked like a simple way of gettting multitasking in CircuitPython.
Inspired by issue #1380 I set out to see what it could look like.

I haven't done much testing, the main purpose at this stage is to serve as an async/await example in the async discussion.

I was pleased to find that uselect had a flexible signalling mechanism for I/O. I think it can also be used for pin interrupts.

This scheduler/eventloop is based on an example in the curio docs:

import time
import uselect

def insort(a, x, lo=0, hi=None):
    lo = 0
    hi = len(a)
    while lo < hi:
        mid = (lo+hi)//2
        if x[0] < a[mid][0]: hi = mid
        else: lo = mid+1
    a.insert(lo, x)


def switch():
    yield ('switch',)

def sleep(seconds):
    yield ('sleep', seconds)

def poll(obj):
    yield ('poll', obj)

tasks = []
sleeping = []
polling = []

def run():
    poller = uselect.poll()
    while tasks or polling:
        if tasks:
            coro = tasks.pop(0)
            try:
                request, *args = coro.send(None)
                if request == 'switch':
                    tasks.append(coro)
                elif request == 'sleep':
                    seconds = args[0]
                    deadline = time.monotonic_ns() + seconds * 1000000000
                    insort(sleeping, (deadline, coro))
                elif request == 'poll':
                    obj = args[0]
                    poller.register(obj)
                    polling.append((obj, coro))
                else:
                    print('Unknown request:', request)
            except StopIteration as e:
                print('Task done:', coro)

        if tasks and not polling:
            continue

        if polling:
            if not tasks and sleeping:
                now = time.monotonic_ns()
                duration_ms = int((sleeping[0][0] - now) / 1000000)
                if duration_ms < 0:
                    duration_ms = 0
            elif not tasks:
                duration_ms = -1
            else:
                duration_ms = 0
            print('poll({}): {!r}'.format(duration_ms, [x[1] for x in polling]))
            ready = poller.poll(duration_ms)
            if ready:
                for obj, *args in ready:
                    poller.unregister(obj)
                    for x in polling:
                        if x[0] == obj:
                            polling.remove(x)
                            tasks.append(x[1])
                            break

        now = time.monotonic_ns()
        while sleeping:
            duration = sleeping[0][0] - now
            if duration > 0:
                if tasks:
                    break
                print('sleep({})'.format(duration / 1000000000))
                time.sleep(duration / 1000000000)
            _, coro = sleeping.pop(0)
            tasks.append(coro)

Some tasks:

async def countdown(n):
    while n > 0:
        print('T-minus', n)
        await sleep(2)
        n -= 1

async def countup(stop):
    n = 1
    while n <= stop:
        print('Up we go', n)
        await sleep(1)
        n += 1

async def blink(n):
    import board
    import digitalio
    with digitalio.DigitalInOut(board.D13) as red:
        red.switch_to_output(0)
        while n > 0:
            red.value = n & 1
            await sleep(0.1)
            n -= 1

async def i2c(address):
    import board
    import i2cslave
    regs = [0] * 16
    index = 0
    stop = False
    with i2cslave.I2CSlave(board.SCL, board.SDA, (address,)) as slave:
        while True:
            await poll(slave)
            r = slave.request()
            with r:
                if r.address == address:
                    if not r.is_read:
                        b = r.read(1)
                        if not b or b[0] > 15:
                            break
                        index = b[0]
                        b = r.read(1)
                        if b:
                            regs[index] = b[0]
                            print('i2c regs[{}] = {}'.format(index, regs[index]))
                            if index == 0 and b[0] == 1:
                                stop = True
                    elif r.is_restart:
                        n = r.write(bytes([regs[index]]))
                        print('i2c regs[{}] == {}'.format(index, regs[index]))
            if stop:
                break

import uio
class Stream(uio.IOBase):
    def __init__(self):
        self.buf = bytearray()

    def ioctl(self, request, arg):
        return 1 if self.buf else 0

    def write(self, b):
        self.buf += b

    def read(self, n=-1):
        if n == -1:
            n = len(self.buf)
        b = self.buf[:n]
        self.buf = self.buf[n:]
        return b

s = Stream()

async def reader():
    for _ in range(2):
        await poll(s)
        print('reader:', s.read())

async def writer():
    await sleep(2)
    s.write(b'Hello')
    print('writer:', s.buf)
    await sleep(1)
    s.write(b'World')
    print('writer:', s.buf)

tasks.append(countdown(10))
tasks.append(countup(15))
tasks.append(blink(50))
tasks.append(reader())
tasks.append(writer())
tasks.append(i2c(0x30))

run()

uio.IOBase gives access to the streaming protocol from python making it possible to hook python code into uselect.poll.

[notro]

Here's the output of a run: https://gist.github.com/notro/6e1028f000dd7d54de07a868d7519181
These are the i2c requests from that run:

$ i2cget -y 1 0x30 0x02
0x00
$ i2cset -y 1 0x30 0 1

[notro]

Example task using pulseio.Counter from PR #1423:

import pulseio
class Button:
    def __init__(self, pin, invert=True, debounce_ms=50):
        self.counter = pulseio.Counter(pin)
        self.debounce_ms = debounce_ms
        self.invert = invert
        self.last = False

    def __enter__(self):
        return self

    def __exit__(self, *args):
        self.counter.deinit()

    @property
    def value(self):
        if self.invert:
            return not self.counter.pinvalue
        else:
            return bool(self.counter.pinvalue)

    async def __await__(self):  # __await__ is not supported by MP
        while True:
            await poll(self.counter)
            if self.debounce_ms:
                await sleep(self.debounce_ms / 1000)
            count = self.counter.clear()
            if self.counter.count:  # Settled?
                continue
            value = self.value
            if value != self.last:
                self.last = value
                return value


async def button(pin):
    with Button(pin) as b:
        while True:
            if await b.__await__():
                print('BUTTON PRESSED')
            else:
                print('BUTTON RELEASED')
                if not tasks and not sleeping and not polling:
                    break

import board
tasks.append(button(board.D5))

[tannewt]

@notro Cool work! I have a few questions:

• What is the code size impact of this?
• How different is uselect from CPython's select?
• What would the typical process for adding async support to a hardware module look like?

Thanks for this example code!

[notro]

What is the code size impact of this?

SAMD51:

• async/await: 1272 bytes
• uselect: 1504 bytes
• uio.IOBase: 240 bytes

How different is uselect from CPython's select?

The main difference is that CPython polls on file descriptors:
poll.register(fd[, eventmask])

while MicroPython polls on objects:
poll.register(obj[, eventmask])

This object needs to support the stream protocol with its ioctl method.
It's also possible to implement the ioctl method from Python using uio.IOBase.

What would the typical process for adding async support to a hardware module look like?

Python supports an await method that makes objects awaitable (shown in the Button class in my pulseio.Counter example above). MicroPython does not currently support this.

The problem with implementing this method on library or C-code, is that it needs to be tied to the eventloop implementation.
Another challenge is that it would need a way for the object to signal that it's ready, or some generic way for the scheduler to poll the object.

However if we use uselect and implement the ioctl method, then we get a generic polling framework for free and it isn't tied to a particular eventloop implementation.

asyncio and curio use the selectors module which is a highlevel module on top of select.

Here's the proposed ioctl implementation for i2cslave: 51e2e83#diff-a66dc4755d023a92be9fe9c94db0f94f

Refs:

• PEP 492 -- Coroutines with async and await syntax
• Awaitable Objects
• uselect
• select
• selectors
• asyncio: Default eventloop

[tannewt]

Thanks for the info @notro. Do you think uselect and ioctl is the best way to go about it? The event mask bits seem a bit weird to me but I'm not sure what would be better.

[notro]

I think polling with uselect looks promising. It's the mechanism MicroPython has chosen for polling on I/O operations (in line with CPython) and is thus fairly future proof.
Even wiznet supports polling with uselect: socket_ioctl() -> wiznet5k_socket_ioctl().
I suspect that UART would work to if it changed to MP_STREAM_POLL in busio_uart_ioctl().

The eventmask is just to specify which operations you're waiting for (read and/or write).

I see that I need to be more precise with my i2cslave ioctl implementation and return MP_STREAM_POLL_RD or MP_STREAM_POLL_WR depending on the RD bit in the address.

Edit:
I want to add that I haven't used async/await before I did this PR so I'm a novice.

[notro]

Deep sleep
I see the possibility for the eventloop to go into deep sleep when there's nothing to do. Awaitable objects that wants to be polled can signal the scheduler whether or not its I/O resource supports wakeup. If all I/O objects support it, the scheduler can go to deep sleep.

[tannewt]

Deep sleep sounds awesome. You've done more async than I have now. :-)

[notro]

I was looking into how I could add support for the __await__ method and discovered that I could just support the __iter__ method. So awaitable objects are supported after all.

This is the small change necessary to my pulseio.Counter example:

 class Button:
+    __iter__ = __await__

 async def button(pin):
     with Button(pin) as b:
         while True:
-            if await b.__await__():
+            if await b:
                 print('BUTTON PRESSED')
             else:
                 print('BUTTON RELEASED')

[tannewt]

Closing this. New PRs can be made once a design is determined in #1380