mpflash: macOS specific ports - long scanning issues

[lukaskremla]

Describe the bug
This bug is related specifically to macOS and to its default /dev/cu.debug-console and bluetooth ports. This issue is present on my macbook M2 Pro running macOS Sequoia version 15.2 but I believe that it will most likely concern all macOS versions (as long as the old versions handle ports similarly).

The problem is that running ls /dev/tty.* it returns microcontroller serial ports but also ports of certain bluetooth devices and the default /dev/cu.debug-console port that is always present. Mpflash will attempt to scan these devices, the default debug-console port is usually skipped within 10-15 seconds, which makes it less annoying, however if I pair a bluetooth JBL speaker mpflash can hang on scanning it for over a minute.

FYI this is what ls /dev/tty.* will return on my computer:

/dev/tty.Bluetooth-Incoming-Port	/dev/tty.debug-console
/dev/tty.Hermes				/dev/tty.usbmodem2101

/dev/tty.Bluetooth-Incoming-Port gets skipped, the /dev/tty.debug-console for a few seconds, and /dev/tty.Hermes (which is what I renamed the JBL speaker to in my Mac's settings) hangs for 1-2 minutes. /dev/tty.usbmodem2101 gets correctly identified as an ESP32 S3 generic board, albeit it takes a while for it to timeout first on the previous non-microcontroller ports. Given that the bluetooth devices can be renamed simply looking for bluetooth in the port's name won't suffice.

I'm unsure if there is anyway to properly tell if a port is a bluetooth port on Mac. If not, a possible solution to narrow down the port listing it always expecting "usbmodem" or "usbserial" to be in the name, all microcontrollers I've worked with so far presented themselves with this in their name, however, I can't deny the possibility that certain microcontrollers won't show up as "usbmodem" or "usbserial".

Tested through iterm2 and PyCharm with mpflash version 1.24.2 and Python 3.10.11.

Hardware information

• MacBook M2 Pro
• MacOS Sequoia 15.2
• (This might concern all macOS computers)

[Josverl]

Thanks for the report
I do not have a Mac myself, so happy to work with you to improve the usability in Mac.
It would be helpful if you can share the output of: mpflash -VV list --json so we can track progress and timing.

• Filtering out /dev/tty.debug-console makes perfect sense and should be simple to add
• changing one textual filter for another im not too sure about yet , but agree that we should find a simple way to avoid mpflash from getting stuck on things like BT connected speakers.
  If 'simply looking for bluetooth in the port's name won't suffice' then I don't think filtering for "usbmodem" or "usbserial" is sufficiently robust either.

mpflash does already offer a way to ignore ports: --ignore -i SERIALPORT Serial port(s) (or globs) to ignore. Defaults to MPFLASH_IGNORE. , which also can be set from the environment. If there is a better default filet to be used on Mac OS that would be a good option.
I'm not sure what the UX rename possibilities are for ports and devices such as the Hermes BT speakers. Would it be possible , and acceptable to add some ignore or skip phrase in its name ?

In parallel Ill do some more research to find another method to filter-out BT devices via a Python API. Thus far I have mostly found cmdline utilities that sometimes require additional permissions.

[lukaskremla]

Hermes is just what I renamed a JBL (Horizon 3 I think?) speaker to. When connected, the below UI under Settings -> Bluetooth allows renaming bluetooth devices to fully custom names, these seem to carry over to port names that'll be listed.

Bluetooth device Management on Mac

Here is a video of executing the command you mentioned:

Video of executing `mpflash -VV list --json`

Screen.Recording.2024-12-16.at.10.37.05.mov

I'm not sure how viable it is to create custom exclusions based on the name - if JBL speakers are affected by this issue I can see a plethora of other devices affected as well. However, one thing to note is that I also have Airpods and bluetooth logitech mouse and keyboard, these don't appear here the same way the JBL speaker does.

[Josverl]

Thanks for the details, a screen cam speaks a thousand words :-)

I wonder if the /dev/cu. prefix is a good candidate for an ignore filter on MacOS.

from: https://pbxbook.com/other/mac-tty.html
You might notice that each serial device shows up twice in /dev, once as a tty.* and once as a cu.. So, what's the difference? Well, TTY devices are for calling into UNIX systems, whereas CU (Call-Up) devices are for calling out from them (eg, modems). We want to call-out from our Mac, so /dev/cu. is the correct device to use.
The technical difference is that /dev/tty.* devices will wait (or listen) for DCD (data-carrier-detect), eg, someone calling in, before responding. /dev/cu.* devices do not assert DCD, so they will always connect (respond or succeed) immediately.

Do you indeed see both /dev/cu.Hermes and /dev/tty.Hermes , or only one of the two ?
Also is there a /dev/cu.usbmodem2101 ?

WRT to checking on name

The name apparently will also depend on the driver needed for the usb-serial connection (also on other platforms)

https://stackoverflow.com/questions/64453154/jssc-not-recognizing-arduino-board/64472977#64472977
PORTNAMES_REGEXP = Pattern.compile("tty.((serial|usbserial|usbmodem).|wchusbserial)");

[lukaskremla]

I'm pretty sure that every device connected like this that I've ever worked with is always present as both cu. and tty.

Iterm screenshot

[Josverl]

Could you please run the following with aa few devices connected and share the text output here ?
( dependencies should be installed by mpflash)

import jsons
import serial.tools.list_ports as list_ports

comports = jsons.dumps(list_ports.comports())
print(comports)

That will allow me to to add more test and update the ignore code.

[lukaskremla]

The results are attached below. I don't have many bluetooth speakers. I tried to connect one other I had on hand, and I connected three ESP devkits I had lying around.

If need be I can get more devkits connected tomorrow when I'll have more time to get as many cables as possible to connect more at once, but I assume that the microcontrollers weren't the main subject of this test.

The other speaker I had on hand was a Jabra 510. And it doesn't pop up in the logs like the JBL Horizon 3 does.

One difference that I do see in the ui is that the JBL is recognized as a "speaker" in text specifically. Whereas the Jabra Device was recognized as a headset. Maybe these are treated differently? just like airpods, keyboards and mice aren't in the .cu device logs unlike the JBL speaker.

I'll try to see if I can find more bluetooth devices to try and connect to my computer tomorrow, though I'm not sure I'll find many.

Formatted result

[
{
  "description": "n/a",
  "device": "/dev/cu.debug-console",
  "hwid": "n/a",
  "interface": null,
  "location": null,
  "manufacturer": null,
  "name": "cu.debug-console",
  "pid": null,
  "product": null,
  "serial_number": null,
  "vid": null
},
{
  "description": "n/a",
  "device": "/dev/cu.Bluetooth-Incoming-Port",
  "hwid": "n/a",
  "interface": null,
  "location": null,
  "manufacturer": null,
  "name": "cu.Bluetooth-Incoming-Port",
  "pid": null,
  "product": null,
  "serial_number": null,
  "vid": null
},
{
  "description": "n/a",
  "device": "/dev/cu.Hermes",
  "hwid": "n/a",
  "interface": null,
  "location": null,
  "manufacturer": null,
  "name": "cu.Hermes",
  "pid": null,
  "product": null,
  "serial_number": null,
  "vid": null
},
{
  "description": "Espressif Device",
  "device": "/dev/cu.usbmodem2101",
  "hwid": "USB VID:PID=303A:4001 SER=983daee912f0cb3f LOCATION=2-1",
  "interface": null,
  "location": "2-1",
  "manufacturer": "Espressif Systems",
  "name": "cu.usbmodem2101",
  "pid": 16385,
  "product": "Espressif Device",
  "serial_number": "983daee912f0cb3f",
  "vid": 12346
},
{
  "description": "Espressif Device",
  "device": "/dev/cu.usbmodem1101",
  "hwid": "USB VID:PID=303A:4001 SER=68b6b3210350cb3f LOCATION=1-1",
  "interface": null,
  "location": "1-1",
  "manufacturer": "Espressif Systems",
  "name": "cu.usbmodem1101",
  "pid": 16385,
  "product": "Espressif Device",
  "serial_number": "68b6b3210350cb3f",
  "vid": 12346
},
{
  "description": "CP2102N USB to UART Bridge Controller",
  "device": "/dev/cu.usbserial-10",
  "hwid": "USB VID:PID=10C4:EA60 SER=c267ea1cda23ed118f088ae8f9a97352 LOCATION=0-1",
  "interface": null,
  "location": "0-1",
  "manufacturer": "Silicon Labs",
  "name": "cu.usbserial-10",
  "pid": 60000,
  "product": "CP2102N USB to UART Bridge Controller",
  "serial_number": "c267ea1cda23ed118f088ae8f9a97352",
  "vid": 4292
}
]

Unformatted result

[{"description": "n/a", "device": "/dev/cu.debug-console", "hwid": "n/a", "interface": null, "location": null, "manufacturer": null, "name": "cu.debug-console", "pid": null, "product": null, "serial_number": null, "vid": null}, {"description": "n/a", "device": "/dev/cu.Bluetooth-Incoming-Port", "hwid": "n/a", "interface": null, "location": null, "manufacturer": null, "name": "cu.Bluetooth-Incoming-Port", "pid": null, "product": null, "serial_number": null, "vid": null}, {"description": "n/a", "device": "/dev/cu.Hermes", "hwid": "n/a", "interface": null, "location": null, "manufacturer": null, "name": "cu.Hermes", "pid": null, "product": null, "serial_number": null, "vid": null}, {"description": "Espressif Device", "device": "/dev/cu.usbmodem2101", "hwid": "USB VID:PID=303A:4001 SER=983daee912f0cb3f LOCATION=2-1", "interface": null, "location": "2-1", "manufacturer": "Espressif Systems", "name": "cu.usbmodem2101", "pid": 16385, "product": "Espressif Device", "serial_number": "983daee912f0cb3f", "vid": 12346}, {"description": "Espressif Device", "device": "/dev/cu.usbmodem1101", "hwid": "USB VID:PID=303A:4001 SER=68b6b3210350cb3f LOCATION=1-1", "interface": null, "location": "1-1", "manufacturer": "Espressif Systems", "name": "cu.usbmodem1101", "pid": 16385, "product": "Espressif Device", "serial_number": "68b6b3210350cb3f", "vid": 12346}, {"description": "CP2102N USB to UART Bridge Controller", "device": "/dev/cu.usbserial-10", "hwid": "USB VID:PID=10C4:EA60 SER=c267ea1cda23ed118f088ae8f9a97352 LOCATION=0-1", "interface": null, "location": "0-1", "manufacturer": "Silicon Labs", "name": "cu.usbserial-10", "pid": 60000, "product": "CP2102N USB to UART Bridge Controller", "serial_number": "c267ea1cda23ed118f088ae8f9a97352", "vid": 4292}]

[lukaskremla]

My opinion right now is that this seems like a very rare problem, that only a small subset of users might experience, especially given it's mac specific, though when it happens it can make using mpflash list function quite annoying.

Perhaps setting aggressive timeouts, or bailing out early on the scanning if the device has as many null values like the first three non-microcontroller devices had when I ran the command you instructed me to run could help.

If it could be useful to you I can try to verify if ESP, Pi Pico and STM all boards follow the usbserial-123456789 or usbmodem123456789 port name format.

Those are all the microcontroller cpu manufacturers I have at home right now.

[Josverl]

Thanks for sharing the details,
Based that it appears that a filter on "hwid": "n/a", would remove the non-wanted devices, without any dependency on the name of the serial port or device driver.

Can you verify by installing this preview to see if that speeds things up for you ?

pip install "git+https://github.com/Josverl/micropython-stubber.git@mpflash-fix-macports#subdirectory=src/mpflash"

hopefully that resolves the annoyance, not only for you but other Mac users as well.

[lukaskremla]

Yup, looks good to me now!