Increased care with serial reads, retries and new passive mode for requesting data #8
Conversation
…buffer which should cause a relevant exception pimoroni#4
…moroni#4. Adding new FrameLengthError exception. Moving checksum calculation code to PMS5003Data constructor. Checking both the field length field and actual data length using a new classmethod in PMS5003Data. Tests awaiting FrameLengthError now look for it. 4 new tests for PMS5003Data construction.
…zero approach in https://github.com/adafruit/Adafruit_CircuitPython_PM25 pimoroni#5 Adding mode argument to PMS5003 class CHANGING the order of arguments too. Object instantiation and reset() now wait for a data frame to be sent from the PMS5003 by checking serial in_waiting property. PMS5003 setup() now works if reset and enable pins are not defined. New PMS5003CmdResponse to deal with PMS5003 repsonding to active/passive mode change commands. New PMS5003Response parent class for PMS5003CmdResponse and original PMS5003Data. Re-arranging buffer data addition in the tests to cater for the object needing the data frame response over serial to instantiate. Two new CircuitPython examples to test this new functionality on Feather board.
…are but observed case of an inopportune data frame sneaking in front of the response to command. Added three tests around that. pimoroni#5
…ctive mode. Adding test for data_available() to existing test_active_mode_read pimoroni#5
The PMS5003 now defaults to 5 retries for read() operations and only throws (the first) exception if all fail. Retaining existing behavior for some tests by setting retries to 0. Adding test_checksum_fail_withretries and test_checksum_retries_ok.
|
@dglaude This looks good if you wish to test on your FeatherS2 setup with Enviro+ FeatherWing and PMS5003. This goes with the standard Pimoroni setup described in https://www.instructables.com/Using-the-Pimoroni-Enviro-FeatherWing-With-the-Ada/ and pimoroni/EnviroPlus-FeatherWing#24 |
|
@dglaude FYI, the release notes for recent 6.1.0-rc.0 mention:
|
|
@Gadgetoid This together with pimoroni/EnviroPlus-FeatherWing#24 makes the Enviro+ FeatherWing more robust and more memory efficient, let me know what you think. This could also be patched to make an update for https://github.com/pimoroni/pms5003-python which is very similar to this library. Note: new use of |
|
Just fixed #5 (comment) which I had noted but missed. |
…oroni#5 Catching exceptions in the constructor where ReadTimeoutError can occur if PMS5003 not connected. Some minor reformatting.
… bytes is correct. pimoroni#6
|
This is really amazing stuff and I'll give it a proper look as soon as I can. I'm currently in house-move-limbo so I have no office and most of my kit in boxes. Bear with me, and thanks for putting in the time to make what appears to be a herculean changeset. |
|
Glad you like the look of it so far. I was trying to make all of this more robust as I gifted two of them this Christmas. I've kept the interface mostly the same but there's a fair amout of general change in there so it warrants a good review and testing by others. Am I right in thinking there are Enviro/Enviro+ users on Raspberry Pi than FeatherWing? It looks that way from Pimoroni Forum chatter. BTW, ESP32-S2-based FeatherS2 gives confusing gas sensor output due to its 2.6V capped ADC, all rather disappointing. You have two explicit recommendations in the product page for Feather boards but perhaps you also want to add a note on this otherwise nice little board being unsuitable or limited due to that issue? More detail in Feather ADC comparison including 2.6V limited ESP32-S2. |
|
I wrote a more detailed explanation, see Software Bugs in a Particulate Matter Sensor Library. |
… read() to improve exceptions. Removing superfluous conversion of serial data from bytes to bytearray pimoroni#4
|
I noticed if I didn't have a PMS5003 plugged in the exception was printed as That |
|
This- plus the linked article- are truly great stuff! My situation has changed to:
Let's see if I can make some of the latter. @dglaude are you still armed with a feather and PMS5003? I'd appreciate your input if you have the time/inclination to provide it. |
|
I am still armed with a many featherS (M4 and NRF52 and other) and a PMS5003, and also a Adafruit SCD-30. Maybe it is best for me to synchronise with him to check each PR and report so that you have some confidence that it is working. |
|
Wait... What is the best place to maintain such a driver for CircuitPython? |
|
The timing didn't seem right to move to the Adafruit version when I started this. Given that adafruit/Adafruit_CircuitPython_PM25#13 still hasn't been merged I don't think it is now either. I've duplicated the features from that PR. |
|
These fixes and features might also be applied to https://github.com/pimoroni/pms5003-python - from memory that was very similar and could almost be generated with a diff from this library. |
Well, I mean this hasn't been merged either 😆 If I'm honest, I had completely forgotten we'd produced a CircuitPython library for this (it's been a heck of a year and then some). Maybe we can get all this merged, updated and released and then compare notes later but I do tend to agree that leaving CircuitPython stuff in Adafruit's ballpark as much as we can helps because we don't necessarily have the people-power to maintain it. Though Pico CircuitPython might be changing the game a little. |
|
Okay so I've switched this repository from Travis to GitHub actions and merged this PR so we can move forward from a slightly less broken/old codebase. Cobbling together my Feather bits to see if I can't get this running! |
|
Now up and running on an Enviro+ Feather M4 Express combo on my desk. Will let it sit overnight and hope my USB doesn't explode again. |
|
3 days later... it's still running. Absolutely bulletproof. Exceptional work! 🎉 |
|
@kevinjwalters - I'm glad that PR I set up late last year has proven to be useful after all! Between moving home and taking care of my partner while she battles cancer I haven't had time to push that one over the finish line, let alone do the testing I've wanted to do with the stack of PMx00x UART sensors I've accumulated. Hoping to get back to it too, but this is solid progress in the right direction! |
Sheesh it's been a hard year without any extra complications- I hope you're both doing okay! 🤗 |
Some care is required with processing data from the serial connection on the Plantower PMS5003 particulate matter sensor. In its default mode it streams data at a variable rate which is not exactly 1Hz. Buffered data can cause a wide variety of error conditions. This change handles these internally and also adds new functionality based on @fivesixzero's work in adafruit/Adafruit_CircuitPython_PM25#13 for a
passivemode (Plantower's terminology). In passive mode data is requested via a transmitted command rather than streamed continuously. Theactivemode is the default and is retained as the default in the constructor to avoid impact to existing application code.passivemode to switchPMS5003into a mode where data isn't streamed but is requested by a command transmitted over serial connection. This eliminates any issue with matching read rate with the variable (200ms to 2.3s) and unpredictable streaming rate and associated potential for partial frames and falling behind with a full/large serial receieve buffer. Add request based reads for polling (AKA passive mode) #5PMS5003constructor to(serial=None, pin_reset=pimoroni_physical_feather_pins.pin23(), pin_enable=pimoroni_physical_feather_pins.pin22(), mode='active', retries=5, baudrate=9600. This is compatible with previous one if keyword args are used.serialcan now be passed and this is used in some tests. Specifyingbaudrateis presumably rare as it cannot be changed as it's dictated by sensor device??PMS5003construtor andreset()now wait for up to 20 seconds for the first data frame indicating presence and good health of the PMS5003. Exceptions aren't thrown in constructor unless newmodekwarg is bogus value Add request based reads for polling (AKA passive mode) #5data_available()method toPMS5003class for use in traditionalactivemode. Add request based reads for polling (AKA passive mode) #5deinit()method toPMS5003class for pins and serial. Add request based reads for polling (AKA passive mode) #5cmd_mode_passiveandcmd_mode_activefor switching between modes. Add request based reads for polling (AKA passive mode) #5PMS5003class to deal internally with various exceptions related to serial reads forread()method. These should read past any problems related to buffering or (presumably very rare at 9600) corruption. The default value of 5 coupled with timeout of 4 lengthens the time to detect a dying PMS5003 to 20 seconds. Add automatic retries for CheckSumMismatch and similar #6FrameLengthErrorexception to accurately report unexpected lengths but retry feature will normally consume/mask these. Now robust against bogus frame length which previously could try to allocate up to 64k of memory and potentially fail. This also eliminates exceptions fromstruct.unpack()parsing Frequent "PMS5003 Checksum Mismatch" #3 Various errors from not reading streaming data promptly #4PMS5003CmdResponseclass, not intended for application use. NewPMS5003Responseparent class. Add request based reads for polling (AKA passive mode) #5in_waitingproperty on serial object (NEW). Add request based reads for polling (AKA passive mode) #5 (Big Python's PySerial appears to have this so likely to be compatible with Raspberry Pi https://pyserial.readthedocs.io/en/latest/pyserial_api.html#serial.Serial.in_waiting)Nonereturn from UARTread()-SerialTimeoutErroris now raised as was intended Various errors from not reading streaming data promptly #4modetest.pyfor testing active/passive mode on device. Add request based reads for polling (AKA passive mode) #5resettest.pyto test and time PMS5003 resets on device Add request based reads for polling (AKA passive mode) #5pimoroni_physical_feather_pinsto mocked libraries. Followed existing practice and lumped these intotest_setup.py. Frequent "PMS5003 Checksum Mismatch" #3 Various errors from not reading streaming data promptly #4 Add request based reads for polling (AKA passive mode) #5 Add automatic retries for CheckSumMismatch and similar #60.0.6to facilitate synchronisation with Raspberry Pi equivalent library if desired.unit tests
modetest
Output from
modetestwith Feather nRF52840 Express running6.0.0.resettest
Output from
resettestwith Feather nRF52840 Express running6.0.0.(The capture doesn't quite run long enough to cover whole test.)
manual tests
PMS5003 disconnected test: