ADC1000 error with external interrupt #193

drmcnelson · 2023-05-04T22:52:28Z

spectrometer and system information

model: (i.e. USB2000, STS, etc...) ADC1000USB
operating system: (i.e. Windows 7 64bit) Linux, Fedora 37, Cinnamon
python version: (output of python --version) Python 3.11.2
python-seabreeze version: (i.e. 0.5.3 or 'current master') 2.2.0
installed-via: (conda, wheel, or python setup.py install) pip install

After setting interrupt mode 3, the call to self.dev.intensities() sometimes produces the following error. Other times it requires two pulses on the input before it will return one spectrum.

 File "/home/nelson/Projects/TeensyDataAcquistion/Python_Programs/./SeaBreezeInstrument.py", line 431, in readspectrum_
    self.ydata = self.dev.intensities()/self.dev.max_intensity
                 ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelson/.local/lib/python3.11/site-packages/seabreeze/spectrometers.py", line 212, in intensities
    out = self._dev.f.spectrometer.get_intensities()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelson/.local/lib/python3.11/site-packages/seabreeze/pyseabreeze/features/spectrometer.py", line 621, in get_intensities
    tmp = self._get_spectrum_raw()
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelson/.local/lib/python3.11/site-packages/seabreeze/pyseabreeze/features/spectrometer.py", line 171, in _get_spectrum_raw
    self.protocol.receive(
  File "/home/nelson/.local/lib/python3.11/site-packages/seabreeze/pyseabreeze/protocol.py", line 707, in receive
    return self.transport.read(
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/nelson/.local/lib/python3.11/site-packages/seabreeze/pyseabreeze/transport.py", line 203, in read
    ret: bytes = self._device.pyusb_device.read(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/usb/core.py", line 1029, in read
    ret = fn(
          ^^^
  File "/usr/lib/python3.11/site-packages/usb/backend/libusb1.py", line 846, in bulk_read
    return self.__read(self.lib.libusb_bulk_transfer,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/usb/backend/libusb1.py", line 954, in __read
    _check(retval)
  File "/usr/lib/python3.11/site-packages/usb/backend/libusb1.py", line 604, in _check
    raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 75] Overflow
S

current problem

detailed description of what doesn't work

steps to reproduce

Step by step instructions to reproduce the error. The more detailed the better, but please use some common sense:

run code example
...

minimal code example and error (very helpful if available)

copy of minimal code example and error messages

The text was updated successfully, but these errors were encountered:

ap-- · 2023-05-07T07:51:24Z

Hi @drmcnelson

Thank you for reporting the issue. I believe this is related to and should be addressed via #190

import json

import seabreeze
seabreeze.use("pyseabreeze")

from seabreeze.spectrometers import Spectrometer


spec = Spectrometer.from_first_available()


def endpoint_sizes(s):
    sbtp = s._dev._transport
    usbdev = s._dev._raw_device.pyusb_device

    ep = {}
    for i in usbdev.get_active_configuration():
        for e in i.endpoints():
            ep[repr((i, e))] = e.wMaxPacketSize

    return {
        "model": s.model,
        "default_read_size": sbtp._default_read_size,
        "default_read_endpoint": sbtp._default_read_endpoint,
        "default_read_spectrum_endpoint": sbtp._default_read_spectrum_endpoint,
        "read_endpoints": sbtp._read_endpoints,
        "endpoint_map": {
            k: f"0x{v:02x}"
            for k, v in vars(sbtp._endpoint_map).items()
        },
        "usb_endpoints": ep
    }

print(json.dumps(endpoint_sizes(spec), indent=2))

Please run the above script with your spectrometer connected, and report the output here.
If this confirms my suspicion that the hardcoded endpoint read size doesn't match the usb endpointsize we would know how to proceed to fix this issue.

Cheers,
Andreas 😃

pbeaucage · 2023-05-17T15:00:01Z

Hi Andreas,

I'm not sure if it is the same issue, but I have an overflow as well when trying to use an ADC1000 connected to a dual channel SD2000 spectrometer. Output of the endpoint map:

{
  "model": "ADC1000-USB",
  "default_read_size": {
    "low_speed": 64,
    "high_speed": 512,
    "high_speed_alt": 512
  },
  "default_read_endpoint": "low_speed",
  "default_read_spectrum_endpoint": "high_speed",
  "read_endpoints": {
    "low_speed": "lowspeed_in",
    "high_speed": "highspeed_in",
    "high_speed_alt": "highspeed_in2"
  },
  "endpoint_map": {
    "primary_out": "0x02",
    "ep_out": "0x02",
    "primary_in": "0x87",
    "lowspeed_in": "0x87",
    "secondary_out": "0x02",
    "secondary_in": "0x82",
    "highspeed_in": "0x82",
    "secondary_in2": "None",
    "highspeed_in2": "None"
  },
  "usb_endpoints": {
    "(<INTERFACE 0: Vendor Specific>, <ENDPOINT 0x2: Bulk OUT>)": 64,
    "(<INTERFACE 0: Vendor Specific>, <ENDPOINT 0x82: Bulk IN>)": 64,
    "(<INTERFACE 0: Vendor Specific>, <ENDPOINT 0x7: Bulk OUT>)": 64,
    "(<INTERFACE 0: Vendor Specific>, <ENDPOINT 0x87: Bulk IN>)": 64
  }
}

Hope this helps; let me know if I should open a separate issue for this. This is on a Mac using manually-forced pyseabreeze backend.

ap-- · 2023-05-18T03:05:26Z

Hi @pbeaucage

Thank you so much for reporting!
The output you posted confirms my suspicion that the actual endpoint read sizes differ from the hardcoded ones.

Could you try running the script that you normally use for measuring, but with the following modification:

import seabreeze
seabreeze.use("pyseabreeze")

from seabreeze.spectrometers import Spectrometer



def _update_default_endpoint_sizes(s):
    # haven't tested this code yet...
    t = s._dev._transport
    rs = t._default_read_size
    re = t._read_endpoints
    em = vars(t._endpoint_map)

    ud = s._dev._raw_device.pyusb_device
    pm = {
        e.bEndpointAddress: e.wMaxPacketSize
        for i in ud.get_active_configuration()
        for e in i.endpoints()
    }

    for e in rs:
        rs[e] = pm[em[re[e]]]



spec = Spectrometer.from_first_available()
_update_default_endpoint_sizes(spec)

Basically for your spectrometer this should set all values in spec._dev._transport._default_read_size to 64, after the spectrometer instance is created.

let me know if this prevents the overflow errors. If yes, I'll prepare a fix.

pbeaucage · 2023-05-18T12:30:27Z

Hey @ap--,
Thanks for the quick reply!
Yep, that fixes it, with a couple oddities that are, I think, separate issues.

On my setup, I have entries in the t._endpoint_map that map to None, specifically secondary_in2 and highspeed_in2. I had to manually patch these to a dummy value for the remapping to work, crudely:

   ...:     for k,v in em.items():
   ...:         if v is not None:
   ...:             new_em[k] = v
   ...:         else:
   ...:             new_em[k] = 130 # where 130 is a valid endpoint on my spectrometer
   ...:     em = new_em

Curiously, the first call to .intensities() still produces an overflow, but subsequent calls work fine and appear to return valid data.

drmcnelson · 2023-05-21T19:26:29Z

Why does the first call to intensities produce an overflow?

Are we sure which spectrum is returned in the second call to "intensities">

I gave up on the device when I saw this happening. I am triggering to get detailed time dependence, and I need some certainty in what I am reading from the instrument.

The one that I have is nice for the work I am doing because it has no slit, lots of signal at low excitation. But it has to be rock solid reliable.

drmcnelson · 2023-05-21T19:49:14Z

P/S I am going to try the patches now. if it works great. But I am thinking about designing a board, probably using an ARM7, to interface to the SD2000 directly, even both units, and connect to the host over USB. Simultaneous triggering could be pretty useful for example. If someone wants to sponsor the effort, it can probably happen quickly.

ap-- · 2023-05-21T21:11:58Z

Thanks for posting the results! The first overflow likely occurs because the hot fix I posted only fixes the endpoint size mismatch after the Spectrometer.init is executed.

I'll work on a PR and then the issue should be fixed.

Cheers,
Andreas 😊

drmcnelson · 2023-05-21T21:33:36Z

Traceback (most recent call last):
File "/home/nelson/Projects/TeensyDataAcquistion/Python_Programs/./SeaBreezeInstrument.py", line 1240, in
seabreeze = SeaBreezeInstrument( graphics=args.graph, debug=args.debug)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nelson/Projects/TeensyDataAcquistion/Python_Programs/./SeaBreezeInstrument.py", line 119, in init
print(json.dumps(endpoint_sizes(self.dev), indent=2))
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nelson/Projects/TeensyDataAcquistion/Python_Programs/./SeaBreezeInstrument.py", line 95, in endpoint_sizes
"endpoint_map": {
^
File "/home/nelson/Projects/TeensyDataAcquistion/Python_Programs/./SeaBreezeInstrument.py", line 96, in
k: f"0x{v:02x}"
^^^^^^^^^^^^
TypeError: unsupported format string passed to NoneType.format

drmcnelson · 2023-05-21T22:15:50Z

Okay, got it working, patch the line
k: f"0x{v:02x}"
to
k: f"0x{v:02x}" if v is not None else None

The output is:

{
"model": "ADC1000-USB",
"default_read_size": {
"low_speed": 64,
"high_speed": 512,
"high_speed_alt": 512
},
"default_read_endpoint": "low_speed",
"default_read_spectrum_endpoint": "high_speed",
"read_endpoints": {
"low_speed": "lowspeed_in",
"high_speed": "highspeed_in",
"high_speed_alt": "highspeed_in2"
},
"endpoint_map": {
"primary_out": "0x02",
"ep_out": "0x02",
"primary_in": "0x87",
"lowspeed_in": "0x87",
"secondary_out": "0x02",
"secondary_in": "0x82",
"highspeed_in": "0x82",
"secondary_in2": null,
"highspeed_in2": null
},
"usb_endpoints": {
"(<INTERFACE 0: Vendor Specific>, <ENDPOINT 0x2: Bulk OUT>)": 64,
"(<INTERFACE 0: Vendor Specific>, <ENDPOINT 0x82: Bulk IN>)": 64,
"(<INTERFACE 0: Vendor Specific>, <ENDPOINT 0x7: Bulk OUT>)": 64,
"(<INTERFACE 0: Vendor Specific>, <ENDPOINT 0x87: Bulk IN>)": 64
}
}

drmcnelson · 2023-05-21T22:32:04Z

The patch also has an error, at

 rs[e] = pm[em[re[e]]]

So, we modify that to

for e in rs:
    try:
        rs[e] = pm[em[re[e]]]
    except Exception as exc:
        print( exc)
        if e is not None:
            print( 'e', e )
            if e in re:
                print( 're[e]', re[e] )
                if re[e] in em:
                    print( 'em[re[e]]', em[re[e]] )
                    if em[re[e]] in pm:
                        print( 'pm[em[re[e]]]', pm[em[re[e]]] )

And then the output before and after running the patch is
1a2,5

None
e high_speed_alt
re[e] highspeed_in2
em[re[e]] None
6c10
< "high_speed": 512,
"high_speed": 64,

In other words, the patch reduced the high_speed setting, whatever it is, from 512 to 64.

Is that what was intended?

ap-- mentioned this issue May 9, 2023

dark pixel indices for ST #194

Merged

ap-- added the waiting for information label May 9, 2023

ap-- mentioned this issue Jun 11, 2023

Dynamically limit max ep size #198

Merged

ap-- closed this as completed in #198 Jun 11, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ADC1000 error with external interrupt #193

ADC1000 error with external interrupt #193

drmcnelson commented May 4, 2023 •

edited by ap--

Loading

ap-- commented May 7, 2023

pbeaucage commented May 17, 2023

ap-- commented May 18, 2023

pbeaucage commented May 18, 2023

drmcnelson commented May 21, 2023

drmcnelson commented May 21, 2023

ap-- commented May 21, 2023

drmcnelson commented May 21, 2023

drmcnelson commented May 21, 2023 •

edited

Loading

drmcnelson commented May 21, 2023 •

edited

Loading

ADC1000 error with external interrupt #193

ADC1000 error with external interrupt #193

Comments

drmcnelson commented May 4, 2023 • edited by ap-- Loading

spectrometer and system information

current problem

steps to reproduce

minimal code example and error (very helpful if available)

ap-- commented May 7, 2023

pbeaucage commented May 17, 2023

ap-- commented May 18, 2023

pbeaucage commented May 18, 2023

drmcnelson commented May 21, 2023

drmcnelson commented May 21, 2023

ap-- commented May 21, 2023

drmcnelson commented May 21, 2023

drmcnelson commented May 21, 2023 • edited Loading

drmcnelson commented May 21, 2023 • edited Loading

drmcnelson commented May 4, 2023 •

edited by ap--

Loading

drmcnelson commented May 21, 2023 •

edited

Loading

drmcnelson commented May 21, 2023 •

edited

Loading