[synthio] add a simple MidiTrack implementation

[tyomitch]

for QBasic-compatible music strings
(https://en.wikibooks.org/wiki/QBasic/Appendix#PLAY)

The music parser is WIP and comments are welcome
before the input validation and missing features
are added; in particular, would microbit syntax
(https://microbit-micropython.readthedocs.io/en/v1.0.1/tutorials/music.html#wolfgang-amadeus-microbit)
be preferable?

[jepler]

This looks neat!

I wonder whether we'll want to place it in its own separate module, like we did with audiomp3.

Good find on space savings :)

[tyomitch]

I wonder whether we'll want to place it in its own separate module, like we did with audiomp3.

Isn't it too simple for a module of its own? MP3Decoder (including mp3dec) has 2470 bytes of code, Music only 546

[tannewt]

It should definitely be in a module of it's own because 546 can be the difference to fitting things in. Why do this in C at all? Can't Python do it? The RTTTL library is very similar but in Python: https://github.com/adafruit/Adafruit_CircuitPython_RTTTL/

Perhaps there is a lower level API we need to add. I highly doubt we need to do any parsing in C though.

[tyomitch]

Why do this in C at all? Can't Python do it? The RTTTL library is very similar but in Python: https://github.com/adafruit/Adafruit_CircuitPython_RTTTL/

The main difference is that adafruit_rtttl cannot play things in background while other Python code runs.

Perhaps there is a lower level API we need to add. I highly doubt we need to do any parsing in C though.

It would be nice to be able to implement AudioSample classes in Python, but I thought there was a conscious decision to disallow any form of concurrency in Python code? And get_buffer() has to run concurrently with other Python code.

[tannewt]

Why do this in C at all? Can't Python do it? The RTTTL library is very similar but in Python: https://github.com/adafruit/Adafruit_CircuitPython_RTTTL/

The main difference is that adafruit_rtttl cannot play things in background while other Python code runs.

Yup! True.

Perhaps there is a lower level API we need to add. I highly doubt we need to do any parsing in C though.

It would be nice to be able to implement AudioSample classes in Python, but I thought there was a conscious decision to disallow any form of concurrency in Python code? And get_buffer() has to run concurrently with other Python code.

Ya, we don't really want to add concurrency because of all of it's pitfalls.

I wonder if a better way to do this would be to have a synthio module with a class that takes in a midi stream and generates audio samples. Or it could just generate control signals like eurorack. MIDI would be a good way to buffer changes into the sound generation code.

[tannewt]

Filed this issue: #4467

[tyomitch]

I wonder if a better way to do this would be to have a synthio module with a class that takes in a midi stream and generates audio samples.
MIDI would be a good way to buffer changes into the sound generation code.

There you go :) Essentially the same code, but taking MIDI stream as input.

[tannewt]

Thanks! That's awesome. One question about the API. I think we'll wait to wait until after 6.2.0 to merge this. That will give us time to refine the API if we need to.

[tannewt]

What do you think about not taking a buffer and having a write method instead that queues data up? That way you can pipe midi from BLE or USB directly into this object.

[tyomitch]

I see your point, but:

• that would make looping (or even passing the same AudioSample to play() repeatedly) impossible;
• the protocol for audiosample_reset_buffer_fun has no way to report failure if the AudioSample is not rewindable;
• if the queued MIDI stream depletes during playback, the playback will stop, instead of waiting for new input. For real-time input, when the MIDI events arrive at the same speed that they're played, this would happen after every event!

Perhaps a new protocol, e.g. AudioStream, should be defined alongside AudioSample, and Audio*Out implementations will need to be updated to handle the new kind of input -- e.g. a new method play_stream which wouldn't take a loop parameter. Then synthio can have MidiStream alongside MidiTrack, with much of the implementation shared. Similarly, there can then be RawStream alongside RawSample.

What do you think?

[tannewt]

Those are all really good points. I'd like to get @jepler's thoughts on it too since he has thought about MP3 streaming some.

[tannewt]

@tyomitch This all looks good. Would you mind putting your comment above in an issue so we can think about that for the future? I'll merge this now.

[tyomitch]

I think we'll wait to wait until after 6.2.0 to merge this. That will give us time to refine the API if we need to.

@tannewt @jepler ping?

[jepler]

I haven't had time to look into that with any detail. (or test it)

I think the main thing is putting it in the right place, module-wise. How to handle repeat=True I think we need to look at, but the time developing 7.0 may give us an opportunity -- and may end up changing how this works in the process.

[tyomitch]

I think the main thing is putting it in the right place, module-wise.

Sorry for forgetting to update the PR title; it no longer targets audiocore

done

[tannewt]

Thank you!

[tannewt]

Looks like it needs a translation merge. Sorry about that!

[tannewt]

Thanks for the update!

[tannewt]

Looks like three boards are out of space. The simplest fix is to disable the module for those builds.

[tyomitch]

Looks like three boards are out of space. The simplest fix is to disable the module for those builds.

Done! #4583 took care of two boards; the third one had to have synthio disabled even so.

[tannewt]

Awesome! Thank you for this!