Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

5.0.9-0 & 5.0.10-0 lsblk whitespace only values - not enough values to unpack #2853

Closed
2 tasks done
phillxnet opened this issue Jun 20, 2024 · 4 comments
Closed
2 tasks done
Assignees

Comments

@phillxnet
Copy link
Member

phillxnet commented Jun 20, 2024

Thanks to forum member coffax for highlighting this issue, and McFaul for confirming. No lsblk serial (SERIAL="") fails to parse with the following error:

File “/opt/rockstor/src/rockstor/rest_framework_custom/generic_view.py”, line 41, in _handle_exception
yield
File “/opt/rockstor/src/rockstor/storageadmin/views/disk.py”, line 418, in post
return self._update_disk_state()
^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib64/python3.11/contextlib.py”, line 81, in inner
return func(*args, **kwds)
^^^^^^^^^^^^^^^^^^^
File “/opt/rockstor/src/rockstor/storageadmin/views/disk.py”, line 115, in _update_disk_state
attached_disks = scan_disks(MIN_DISK_SIZE)
^^^^^^^^^^^^^^^^^^^^^^^^^
File “/opt/rockstor/src/rockstor/system/osi.py”, line 330, in scan_disks
blk_dev_properties: dict = {
^
File “/opt/rockstor/src/rockstor/system/osi.py”, line 336, in
for key, value in (
^^^^^^^^^^
ValueError: not enough values to unpack (expected 2, got 1)

Blocking our prior fail-over too udev for serial retrieval. And given this is a low level failure, uncaught, we end up with no drive info.

Requirements

  • Reproducer test in src/rockstor/system/tests/test_osi.py as reproducer - and to help avoid regression in the future.
  • Parsing fix/work-around proven against the above test.

Forum reference: https://forum.rockstor.com/t/houston-weve-had-a-problem-not-enough-values-to-unpack-expected-2-got-1/9534

@phillxnet phillxnet self-assigned this Jun 20, 2024
@phillxnet
Copy link
Member Author

Noting for reference the last major change in this testing branch/channel code:
"Un special-case system drive btrfs-in-partition treatment #2824" #2835

@phillxnet phillxnet added this to the 5.1.X-X Stable release milestone Jun 20, 2024
@phillxnet
Copy link
Member Author

Python console reproducer:

Our failing line parse looks to be:

>>> line = 'NAME="/dev/sda" MODEL="ST18000NM000J-2T" SERIAL="ZR54RJQ2" SIZE="16.4T" TRAN="" VENDOR="        " HCTL="1:0:0:0" TYPE="disk" FSTYPE="btrfs" LABEL="Pool" UUID="f15ef88e-c734-49a9-9f5a-3bfaf40656d0"'
>>> blk_dev_properties: dict = {
...             key.lower()
...             if key != "TRAN"
...             else "transport": value.replace('"', "").strip()
...             if value.replace('"', "").strip() != ""
...             else None
...             for key, value in (
...                 key_value.split("=") for key_value in line.strip().split('" ')
...             )
...         }
Traceback (most recent call last):
...
  File "<input>", line 1, in <module>
  File "<input>", line 7, in <dictcomp>
ValueError: not enough values to unpack (expected 2, got 1)

Working line example:

Where-as if we do the same on a working line we have:

>>> line = 'NAME="/dev/sdb" MODEL="Virtual Disk" SERIAL="6002248008e48f45909f67474b658d2a" SIZE="32G" TRAN="" VENDOR="Msft    " HCTL="0:0:0:0" TYPE="disk" FSTYPE="" LABEL="" UUID=""'
>>> blk_dev_properties: dict = {
...             key.lower()
...             if key != "TRAN"
...             else "transport": value.replace('"', "").strip()
...             if value.replace('"', "").strip() != ""
...             else None
...             for key, value in (
...                 key_value.split("=") for key_value in line.strip().split('" ')
...             )
...         }
>>> print(blk_dev_properties)
{'name': '/dev/sdb', 'model': 'Virtual Disk', 'serial': '6002248008e48f45909f67474b658d2a', 'size': '32G', 'transport': None, 'vendor': 'Msft', 'hctl': '0:0:0:0', 'type': 'disk', 'fstype': None, 'label': None, 'uuid': None}

Reproducer lsblk output taken from McFaul's forum feedback using a VM instance of Rockstor with a drive passthrough.

Initial suspicion is that this is VM only and relates to the following lsblk element provided by the hypervisor:

VENDOR=" "
throwing our parsing .

We now have a unit test reproducer for this issue as reported.

@phillxnet
Copy link
Member Author

Suggestion reinforcing suspicion re whitespace VENDOR as provided by hypervisor and we have:

line = 'NAME="/dev/sda" MODEL="ST18000NM000J-2T" SERIAL="ZR54RJQ2" SIZE="16.4T" TRAN="" VENDOR="" HCTL="1:0:0:0" TYPE="disk" FSTYPE="btrfs" LABEL="Pool" UUID="f15ef88e-c734-49a9-9f5a-3bfaf40656d0"'
blk_dev_properties: dict = {
            key.lower()
            if key != "TRAN"
            else "transport": value.replace('"', "").strip()
            if value.replace('"', "").strip() != ""
            else None
            for key, value in (
                key_value.split("=") for key_value in line.strip().split('" ')
            )
        }
print(blk_dev_properties)
{'name': '/dev/sda', 'model': 'ST18000NM000J-2T', 'serial': 'ZR54RJQ2', 'size': '16.4T', 'transport': None, 'vendor': None, 'hctl': '1:0:0:0', 'type': 'disk', 'fstype': 'btrfs', 'label': 'Pool', 'uuid': 'f15ef88e-c734-49a9-9f5a-3bfaf40656d0'}

I.e. previously failing line parse works as expected.

We are dealing with an unsupported platform here, but I'll look for a bit at viable work-arounds if they do not overly complicate all other supported instances for the same of buggy hypervisor communications of drive VENDOR as indicated.

@phillxnet phillxnet changed the title 5.0.9-0 & 5.0.10-0 lsblk no serial fails with - not enough values to unpack 5.0.9-0 & 5.0.10-0 lsblk white space VENDOR (VM) - not enough values to unpack Jun 20, 2024
@phillxnet phillxnet changed the title 5.0.9-0 & 5.0.10-0 lsblk white space VENDOR (VM) - not enough values to unpack 5.0.9-0 & 5.0.10-0 lsblk whitespace only values - not enough values to unpack Jun 20, 2024
phillxnet added a commit to phillxnet/rockstor-core that referenced this issue Jun 20, 2024
…o unpack rockstor#2853

VM instance report had all whitespace VENDOR value returned by lsblk.
Add re.sub() pre-processor to each lsblk line prior to main parsing.

Includes
- Remarked out (noisy) debug line to easy future investigations.
- Additional unit-test reproducer to prove fix and guard against
regression.
phillxnet added a commit that referenced this issue Jun 22, 2024
…-whitespace-only-values---not-enough-values-to-unpack

5.0.9-0 & 5.0.10-0 lsblk whitespace only values - not enough values to unpack #2853
@phillxnet
Copy link
Member Author

Closing as:
Fixed by #2855

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant