Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance scan_disks to handle md block devices #1063

Closed
phillxnet opened this issue Dec 22, 2015 · 7 comments
Closed

enhance scan_disks to handle md block devices #1063

phillxnet opened this issue Dec 22, 2015 · 7 comments

Comments

@phillxnet
Copy link
Member

Although the build in fail over to use udev rather than lsblk to find an md block device's serial number succeeds the fact that it appears twice in lsblk's output (in the case of a 2 device md) then triggers the duplication removal logic and the real udev rule applied serial number successfully retrieved on the first pass is overwritten with an auto generated fake-serial on the second. This in turn causes the "Disk Serial number not legitimate" warning even when a valid serial is available (applied via udev rule).
N.B. this has only been proved on non root md devices but is strongly suspected to apply to the system device also which is the only likely supported use for md devices.

Please update the following forum post on progress with this issue:-
http://forum.rockstor.com/t/issues-cant-wipe-some-drives-no-smart-diagnostics-disk-serial-numbers-not-legitamate/820

phillxnet added a commit to phillxnet/rockstor-core that referenced this issue Dec 22, 2015
@phillxnet
Copy link
Member Author

the attached logging patch indicates the issue.

[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:841] ########### process next block device ############
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:842] dmap['serial'] = WD-WMAET2305538 
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:843] always_use_udev_serial = False 
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:841] ########### process next block device ############
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:842] dmap['serial'] =  
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:843] always_use_udev_serial = False 
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:845] USING the UDEV based get_disk_serial SYSTEM
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:850] get_disk_serial returned = 235711
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:841] ########### process next block device ############
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:842] dmap['serial'] = 6RW17C9A 
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:843] always_use_udev_serial = False 
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:841] ########### process next block device ############
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:842] dmap['serial'] =  
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:843] always_use_udev_serial = False 
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:845] USING the UDEV based get_disk_serial SYSTEM
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:850] get_disk_serial returned = 235711
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:852] the contents of serials = ['WD-WMAET2305538', '235711', '6RW17C9A']
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:841] ########### process next block device ############
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:842] dmap['serial'] = C109020428 
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:843] always_use_udev_serial = False 
[22/Dec/2015 20:31:49] DEBUG [fs.btrfs:873] scan_disks returns the following as disks [Disk(name='md0', model=None, serial='fake-serial-dd5d6e7d-2d55-41db-9b27-bf307954412b', size=78118912, transport=None, vendor=None, hctl=None, type='raid1', fstype=None, label=None, btrfs_uuid=None, parted=False, root=False), Disk(name='sdc3', model=None, serial='C109020428', size=27682406, transport=None, vendor=None, hctl=None, type='part', fstype='btrfs', label='rockstor_rockstor', btrfs_uuid='8a33134f-e461-4367-818d-6eca8ed1faa6', parted=False, root=True), Disk(name='sda', model='WDC WD800JD-40GB', serial='WD-WMAET2305538', size=78118912, transport='sata', vendor='ATA', hctl='1:0:0:0', type='disk', fstype=None, label=None, btrfs_uuid=None, parted=True, root=False), Disk(name='sdb', model='ST380815AS', serial='6RW17C9A', size=78118912, transport='sata', vendor='ATA', hctl='2:0:0:0', type='disk', fstype=None, label=None, btrfs_uuid=None, parted=True, root=False)]

phillxnet added a commit to phillxnet/rockstor-core that referenced this issue Dec 26, 2015
md* (multi device) block devices such as software raid
and some hardware raid devices are listed multiple times
in lsblk output, this causes them to be labeled as duplicate
devices due to shared serial number and any serial number
retrieved on the first occurrence is over-ridden on the second
occurrence by a fake to denote serial, and hence device,
duplication. This is incorrect in the case of these md devices.
@phillxnet
Copy link
Member Author

logging after above attached patch to skip duplicate kernel dev names in scan_disks:-

[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:812] NAME found from lsblk parse was = sda 
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:822] contents of device_names_seen = ['sda']
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:854] ######## process block device - sda - #########
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:855] dmap['serial'] = WD-WMAET2305538 
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:856] always_use_udev_serial = False 
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:812] NAME found from lsblk parse was = sda1 
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:822] contents of device_names_seen = ['sda', 'sda1']
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:812] NAME found from lsblk parse was = md0 
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:822] contents of device_names_seen = ['sda', 'sda1', 'md0']
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:854] ######## process block device - md0 - #########
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:855] dmap['serial'] =  
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:856] always_use_udev_serial = False 
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:858] USING the UDEV based get_disk_serial SYSTEM
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:863] get_disk_serial returned = 235711
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:812] NAME found from lsblk parse was = sdb 
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:822] contents of device_names_seen = ['sda', 'sda1', 'md0', 'sdb']
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:854] ######## process block device - sdb - #########
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:855] dmap['serial'] = 6RW17C9A 
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:856] always_use_udev_serial = False 
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:812] NAME found from lsblk parse was = sdb1 
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:822] contents of device_names_seen = ['sda', 'sda1', 'md0', 'sdb', 'sdb1']
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:812] NAME found from lsblk parse was = md0 
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:819] found DUPLICATE device name in lsblk so skipping
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:812] NAME found from lsblk parse was = sdc 
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:822] contents of device_names_seen = ['sda', 'sda1', 'md0', 'sdb', 'sdb1', 'sdc']
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:812] NAME found from lsblk parse was = sdc1 
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:822] contents of device_names_seen = ['sda', 'sda1', 'md0', 'sdb', 'sdb1', 'sdc', 'sdc1']
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:812] NAME found from lsblk parse was = sdc2 
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:822] contents of device_names_seen = ['sda', 'sda1', 'md0', 'sdb', 'sdb1', 'sdc', 'sdc1', 'sdc2']
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:812] NAME found from lsblk parse was = sdc3 
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:822] contents of device_names_seen = ['sda', 'sda1', 'md0', 'sdb', 'sdb1', 'sdc', 'sdc1', 'sdc2', 'sdc3']
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:854] ######## process block device - sdc3 - #########
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:855] dmap['serial'] = C109020428 
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:856] always_use_udev_serial = False 
[26/Dec/2015 18:00:30] DEBUG [fs.btrfs:886] scan_disks returns the following as disks [Disk(name='md0', model=None, serial='235711', size=78118912, transport=None, vendor=None, hctl=None, type='raid1', fstype=None, label=None, btrfs_uuid=None, parted=False, root=False), Disk(name='sdc3', model=None, serial='C109020428', size=27682406, transport=None, vendor=None, hctl=None, type='part', fstype='btrfs', label='rockstor_rockstor', btrfs_uuid='8a33134f-e461-4367-818d-6eca8ed1faa6', parted=False, root=True), Disk(name='sda', model='WDC WD800JD-40GB', serial='WD-WMAET2305538', size=78118912, transport='sata', vendor='ATA', hctl='1:0:0:0', type='disk', fstype=None, label=None, btrfs_uuid=None, parted=True, root=False), Disk(name='sdb', model='ST380815AS', serial='6RW17C9A', size=78118912, transport='sata', vendor='ATA', hctl='2:0:0:0', type='disk', fstype=None, label=None, btrfs_uuid=None, parted=True, root=False)]

@phillxnet
Copy link
Member Author

So far only tested as working with non system, ie data, mdadmin md device and udev assigned serial of "235711" (patch applied to testing channel 3.8-10.07):-
Before:-
before-duplicate-dev-name-removal

After:-
after-duplicate-dev-name-removal

@phillxnet
Copy link
Member Author

Note that md0 is made from sda (sda1) + sdb (sdb1) in Raid1 (mdadmin) as a test setup for debug purposes to replicate multiple lsblk entries for the same device which was observed on forum reporters hardware raid system block device as well.
Also note if this was to be a supported configuration the cog could warn of possible raid membership as a quick workaround for now and the delete dialog as well. Better to also list raid device members in md device entry and possibly also the reverse ie list membership of md0 in sda's and sdb's disk entries. However this complicates the interface and make it more confusing potentially.

@schakrava I am not proposing that we support md devices due to the above confusion element only that we deal more elegantly with them and handle their use in a limited way by facilitating the udev serial attribution to achieve the above "After" effect.

@phillxnet phillxnet changed the title enhance scan_disks to handle md block devices on system drive enhance scan_disks to handle md block devices Dec 29, 2015
@phillxnet
Copy link
Member Author

Changed title to reflect tested context, see images earlier in this thread. The patch as is simply facilitate the use of udev to assign serial numbers to md devices and not have them be overridded by being considered duplicate devices, as per their lsblk multiple entries.

phillxnet added a commit to phillxnet/rockstor-core that referenced this issue Dec 29, 2015
@phillxnet
Copy link
Member Author

I have tested with multiple md devices made from paired partitions on 2 drives to further check functionality each md device had a udev rule to provide the serial number:-
multiple-md-devices-with-udev-assigned-serials

schakrava added a commit that referenced this issue Dec 30, 2015
@schakrava
Copy link
Member

Fixed by #1071

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants