Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hang on USB disconnect #3817

Closed
vozhyk- opened this issue Sep 22, 2015 · 4 comments
Closed

Hang on USB disconnect #3817

vozhyk- opened this issue Sep 22, 2015 · 4 comments
Milestone

Comments

@vozhyk-
Copy link
Contributor

vozhyk- commented Sep 22, 2015

A single USB pendrive pool. When the pendrive is disconnected, ZFS doesn't notice it (shows it as ONLINE despite the device not existing in /dev), and when something tries to access it, the system hangs.
Backtraces and dmesg after triggering a panic with SysRq (the pendrive is F0394B5D):
https://bpaste.net/show/47a9268adc7b (3) - the drive was deliberately pulled out, and the hang happened when MPD tried to access the next song (a bit different than the others, probably because I used sysrq-u immediately before sysrq-c)
https://bpaste.net/show/504b0e6caf37 (2)
https://bpaste.net/show/0ea0fabc4a3b (1)
In (1) and (2) a disconnect and reconnect is visible some time before the hang.
In all 3 instances the ZFS processes hang when trying to access the pool.

Funtoo, linux 4.0.9-aufs with BFS, BFQ, UKSM and TuxOnIce patches.

@vozhyk-
Copy link
Contributor Author

vozhyk- commented Sep 22, 2015

ZFS/SPL versions are close to 0.6.5 -- ZFS 0.6.5 - 1 commit .. 0.6.5.1

@kernelOfTruth
Copy link
Contributor

Perhaps somewhat related to #3785 system freezes when zfs is waiting for disks to spin up

@behlendorf behlendorf added this to the 0.6.5.2 milestone Sep 23, 2015
behlendorf added a commit that referenced this issue Sep 25, 2015
Commit b39c22b set the READ_SYNC and WRITE_SYNC flags for a bio
based on the ZIO_PRIORITY_* flag passed in.  This had the unnoticed
side-effect of making the vdev_disk_io_start() synchronous for
certain I/Os.

This in turn resulted in vdev_disk_io_start() being able to
re-dispatch zio's which would result in a RCU stalls when a disk
was removed from the system.  Additionally, this could negatively
impact performance and explains the performance regressions reported
in both #3829 and #3780.

This patch resolves the issue by making the blocking behavior
dependent on a 'wait' flag being passed rather than overloading
the passed bio flags.

Finally, the WRITE_SYNC and READ_SYNC behavior is restricted to
non-rotational devices where there is no benefit to queuing to
aggregate the I/O.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #3652
Issue #3780
Issue #3785
Issue #3817
Issue #3821
Issue #3829
Issue #3832
Issue #3870
@behlendorf
Copy link
Contributor

Resolved by 5592404 which will be cherry-picked in to 0.6.5.2 release.

behlendorf added a commit that referenced this issue Sep 30, 2015
Commit b39c22b set the READ_SYNC and WRITE_SYNC flags for a bio
based on the ZIO_PRIORITY_* flag passed in.  This had the unnoticed
side-effect of making the vdev_disk_io_start() synchronous for
certain I/Os.

This in turn resulted in vdev_disk_io_start() being able to
re-dispatch zio's which would result in a RCU stalls when a disk
was removed from the system.  Additionally, this could negatively
impact performance and explains the performance regressions reported
in both #3829 and #3780.

This patch resolves the issue by making the blocking behavior
dependent on a 'wait' flag being passed rather than overloading
the passed bio flags.

Finally, the WRITE_SYNC and READ_SYNC behavior is restricted to
non-rotational devices where there is no benefit to queuing to
aggregate the I/O.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #3652
Issue #3780
Issue #3785
Issue #3817
Issue #3821
Issue #3829
Issue #3832
Issue #3870
@pcjc2
Copy link

pcjc2 commented Nov 14, 2017

This symptom (hang on USB removal) appears to be back in 0.7.1. Again, zpool showed the device as ONLINE.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants