Very slow zfs destroy #11933

rnz · 2021-04-23T13:08:22Z

After shutdown with multiple containers (~1000) on docker start run typical operations:

...
root      118183    8047  0 16:01 ?        00:00:00     zfs destroy -r pool-docker/ecd5083198a69056a25d4074e5aedc7498751c9095b2ade1f190344e75653a18-init
root      118184    8047  0 16:01 ?        00:00:00     zfs destroy -r pool-docker/7d2a1f47e436c731bb0262747be2cc7149f6baec891cdc811daf0de2453dc0f7-init
root      118185    8047  0 16:01 ?        00:00:00     zfs destroy -r pool-docker/9535d8a1c36d0fae783f43e3198d6232540734a734d45c658515014766392b69-init
root      118186    8047  0 16:01 ?        00:00:00     zfs destroy -r pool-docker/c298378cc66393b5df9701db92c2014dc4bfa3c1f46820986f376017c8e1055e-init
root      118187    8047  0 16:01 ?        00:00:00     zfs destroy -r pool-docker/c26b66ee106a8da89818e849353b74408c129bf7ae0bdf0708f630ecea88d8ee-init
root      118188    8047  0 16:01 ?        00:00:00     zfs destroy -r pool-docker/5690304337c0d5e5d7d5f28e61db7ce68c6b238280653526373d50088ee6d603-init
root      123164    8047  0 16:01 ?        00:00:00     zfs destroy -r pool-docker/8aea1a0fe1902d589ce4db6a520336deb1cced2f5ae67a19038c29a82de4e916
root      123711    8047  0 16:01 ?        00:00:00     zfs destroy -r pool-docker/9951cb7df52c82d88a79fe7d9a11a4f54bb6357c0fd650cf7f52e38f77f73936
root      123948    8047  0 16:01 ?        00:00:00     zfs destroy -r pool-docker/03934143ce2efe935ad1a2c1b15b6875939099c2dcd8cb855ab4258c25ddd7a8
root      124247    8047  0 16:01 ?        00:00:00     zfs destroy -r pool-docker/7cafd10677c5505241dd71a9a885a7ecf0c0864d456815a7b02d54ef78fe1d4d
root      124342    8047  0 16:01 ?        00:00:00     zfs destroy -r pool-docker/2239e9853fddb2a609199f435169caaa895365c2fb4a3d59cc507a37b3c68ae7
root      125012    8047  0 16:01 ?        00:00:00     zfs destroy -r pool-docker/44608777834f5a6131e548fbe071dc1d6a9b763db63749ccfad3eb8b4f5db684
root      145987    8047  0 16:02 ?        00:00:00     zfs destroy -r pool-docker/c9b95afc1ea09b3d7b1786abd029214462ab724dcd082df9fefb4a28e9de63b1
...

# pgrep -fc 'zfs destroy'
1162

This forces to abandon use ZFS as storage backend for docker... ((

System information

Type	Version/Name
Distribution Name	Debian
Distribution Version	10
Linux Kernel	4.19.98
Architecture	amd64
ZFS Version	2.0.3-1~bpo10+1
SPL Version	2.0.3-1~bpo10+1

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0,01    0,00    0,70   98,65    0,01    0,63

Device            r/s     w/s     rkB/s     wkB/s   rrqm/s   wrqm/s  %rrqm  %wrqm r_await w_await aqu-sz rareq-sz wareq-sz  svctm  %util
sda              0,00    0,00      0,00      0,00     0,00     0,00   0,00   0,00    0,00    0,00   0,00     0,00     0,00   0,00   0,00
sdc              0,00    0,00      0,00      0,00     0,00     0,00   0,00   0,00    0,00    0,00   0,00     0,00     0,00   0,00   0,00
sdb              0,00    0,00      0,00      0,00     0,00     0,00   0,00   0,00    0,00    0,00   0,00     0,00     0,00   0,00   0,00
zram0            0,00    0,00      0,00      0,00     0,00     0,00   0,00   0,00    0,00    0,00   0,00     0,00     0,00   0,00   0,00

# zpool iostat -v 1
                                          capacity     operations     bandwidth 
pool                                    alloc   free   read  write   read  write
--------------------------------------  -----  -----  -----  -----  -----  -----
pool-data                                311G   189G     26     67   290K  2.09M
  scsi-0QEMU_QEMU_HARDDISK_drive-scsi1   311G   189G     26     67   290K  2.09M
--------------------------------------  -----  -----  -----  -----  -----  -----
pool-docker                             47.9G   102G    100     51  1.98M  1.77M
  scsi-0QEMU_QEMU_HARDDISK_drive-scsi2  47.9G   102G    100     51  1.98M  1.77M
--------------------------------------  -----  -----  -----  -----  -----  -----
                                          capacity     operations     bandwidth 
pool                                    alloc   free   read  write   read  write
--------------------------------------  -----  -----  -----  -----  -----  -----
pool-data                                311G   189G      0      0      0      0
  scsi-0QEMU_QEMU_HARDDISK_drive-scsi1   311G   189G      0      0      0      0
--------------------------------------  -----  -----  -----  -----  -----  -----
pool-docker                             47.9G   102G      0      0      0      0
  scsi-0QEMU_QEMU_HARDDISK_drive-scsi2  47.9G   102G      0      0      0      0
--------------------------------------  -----  -----  -----  -----  -----  -----

zfs destroy only wait something...

The text was updated successfully, but these errors were encountered:

rincebrain · 2021-04-23T13:22:21Z

You seem to have deleted the entire form of requested information for bug reports; please don't do that, and provide the requested information so people can try to do something useful with your report.

ahrens · 2021-04-23T16:59:13Z

What are the things you are zfs destroy-ing? Filesystems? Clones? How big are they? (referenced and used) How long does it take to destroy them? How long do you expect it to take? How fast is the storage? How many IOPS are we doing to the storage during zfs destroy and how many should the storage hardware be able to deliver?

rnz · 2021-04-23T18:28:43Z

@ahrens - https://docs.docker.com/storage/storagedriver/zfs-driver/

clones (snapshots)
referenced: 533 M, used: 0-144 B
How long does it take to destroy them? - infinite
How long do you expect it to take? - 5-10 seconds per snapshot
How fast is the storage? - >2000 iops, >300 M/s
How many IOPS are we doing to the storage during zfs destroy and how many should the storage hardware be able to deliver? - zfs destroy can almost hang already at ~100 operations

ahrens · 2021-04-23T18:40:20Z

Interesting, so it seems like the zfs destroys are not making progress at all? You have the same number of zfs destroy processes for a long time (dozens of minutes)? That sounds like something has hung/deadlocked. Do you have anything interesting in dmesg?

rnz · 2021-04-23T18:59:59Z

@ahrens no, it making progress but very slow. I think that is becouse system has accumulated many clones and snaps:

# zfs list -t snapshot -o name,referenced,used
NAME                                                                                            REFER   USED
pool-docker/000114ddd50a5c96605014b5ca22ca16182068aecdca0f2266885e267cca2d61-init@225235166      533M     0B
pool-docker/002c2796494f92236c3204be73987a7fea3c70f4d5f97adb5453983114da35d7-init@895160915      533M     0B
pool-docker/00317ae6dbcd7bf6b485892ec5ac90195037c6f239e94787e15bd96f57f3315f-init@904796009      533M   144K
pool-docker/004002851bf34d0c3a2118ef06235574fae0a87eb584e5d8b25f43c969dc4652-init@896754419      533M   144K
pool-docker/0049c3d726c0db75f672e2106265a1aac58c91b3c2716b7b76a852b65e64c6e3-init@696579274      533M     0B
pool-docker/004a7894a6d1b8a38aa452f727d3e20736c697e16ff81df29ad2c7bb418e4469@997489309           114M    16K
pool-docker/004d3df8b8507b7d8dd1cb2976399a98d9377181e5284147aaccc011dc943d21-init@835106641      533M     0B
pool-docker/0056d0e8e6558e6a21528d7fb5b7e4dc0352441e2497ca260d39a6e5c4ea0cb4-init@147902081      533M     0B
pool-docker/005a3b8abee8775dcd943fac8d7dcc7f74cb599584035af121d2d728ee74ecec-init@744850428      533M   144K
pool-docker/006231efbb5a9bf2c9240886c2fce1cd74e1e1f9e5c7981c0414b57eb87046be-init@572340213      533M   144K
pool-docker/00a2772396565301ef073eac9c6bb6002aa6c349a39cc28db8dad7bc01f9582e-init@720248583      533M     0B
pool-docker/00b277c0eddf61f34fdb2c29f55e8068f75b2f250be878915bf574153e2e81fc-init@719983133      533M   144K
pool-docker/00cb0b3870723a4c30dc70f4e14ca876ee9b41c73d60ff880d0531a7e2ecdff4-init@420358744      533M     0B
pool-docker/00d292982ca3083cc67141b3ff2b8f254e6b034360d54f1e58f11ece36cadec4-init@479763402      533M   144K
pool-docker/0120242c0cad4b0bce6adf7280c5437f8023851f47e238f854b410e05b223fab-init@261879288      533M     0B
...

$ xsel -b | grep 533M -c
13820

You have the same number of zfs destroy processes for a long time (dozens of minutes)? - not minutes - hours. I play with that all day.
That sounds like something has hung/deadlocked. - yes, but progress is present but very slow
Do you have anything interesting in dmesg? - no.

rincebrain · 2021-04-23T20:53:16Z

Huh, so they're a bunch of clones. Do you have feature@livelist enabled on the pool? (zpool get feature@livelist) I think I expect that to help with this scenario, in theory.

edit: I would assume that like most things with ZFS, it won't retroactively make things better for existing clones when you turn it on, though, just improve the behavior for clones made after you do.

rnz · 2021-04-23T21:02:46Z

@rincebrain

# zpool get feature@livelist
NAME         PROPERTY          VALUE             SOURCE
pool-data    feature@livelist  disabled          local
pool-docker  feature@livelist  active            local

But that was enabled recently.

rnz · 2021-04-24T20:37:28Z

Yes it is deadlock:

Next commands was not take any effects:

# pkill -9 'zfs clone'
# pkill -9 'zfs snapshot'
# pkill -9 'zfs destroy'

z_livelist_dest is blocked (dmesg):

[Sat Apr 24 23:00:42 2021] INFO: task z_livelist_dest:38074 blocked for more than 120 seconds.
[Sat Apr 24 23:00:42 2021]       Tainted: P           OE     4.19.98-custom #1
[Sat Apr 24 23:00:42 2021] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Sat Apr 24 23:00:42 2021] z_livelist_dest D    0 38074      2 0x80000000
[Sat Apr 24 23:00:42 2021] Call Trace:
[Sat Apr 24 23:00:42 2021]  ? __schedule+0x2a2/0x870
[Sat Apr 24 23:00:42 2021]  schedule+0x28/0x80
[Sat Apr 24 23:00:42 2021]  io_schedule+0x12/0x40
[Sat Apr 24 23:00:42 2021]  cv_wait_common+0xaf/0x130 [spl]
[Sat Apr 24 23:00:42 2021]  ? finish_wait+0x80/0x80
[Sat Apr 24 23:00:42 2021]  txg_wait_synced_impl+0xc2/0x110 [zfs]
[Sat Apr 24 23:00:42 2021]  txg_wait_synced+0xc/0x40 [zfs]
[Sat Apr 24 23:00:42 2021]  dsl_sync_task_common+0x1b5/0x290 [zfs]
[Sat Apr 24 23:00:42 2021]  ? spa_spawn_aux_threads+0xb0/0xb0 [zfs]
[Sat Apr 24 23:00:42 2021]  ? dsl_scan_assess_vdev+0xb0/0xb0 [zfs]
[Sat Apr 24 23:00:42 2021]  ? dsl_scan_assess_vdev+0xb0/0xb0 [zfs]
[Sat Apr 24 23:00:42 2021]  ? spa_spawn_aux_threads+0xb0/0xb0 [zfs]
[Sat Apr 24 23:00:42 2021]  ? __thread_exit+0x20/0x20 [spl]
[Sat Apr 24 23:00:42 2021]  dsl_sync_task+0x16/0x20 [zfs]
[Sat Apr 24 23:00:42 2021]  spa_livelist_delete_cb+0x1e8/0x300 [zfs]
[Sat Apr 24 23:00:42 2021]  zthr_procedure+0x13c/0x160 [zfs]
[Sat Apr 24 23:00:42 2021]  ? zrl_is_locked+0x20/0x20 [zfs]
[Sat Apr 24 23:00:42 2021]  thread_generic_wrapper+0x6f/0x80 [spl]
[Sat Apr 24 23:00:42 2021]  kthread+0x112/0x130
[Sat Apr 24 23:00:42 2021]  ? kthread_bind+0x30/0x30
[Sat Apr 24 23:00:42 2021]  ret_from_fork+0x22/0x40
[Sat Apr 24 23:00:42 2021] INFO: task dockerd:45848 blocked for more than 120 seconds.
[Sat Apr 24 23:00:42 2021]       Tainted: P           OE     4.19.98-custom #1
[Sat Apr 24 23:00:42 2021] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Sat Apr 24 23:00:42 2021] dockerd         D    0 45848      1 0x00000000
[Sat Apr 24 23:00:42 2021] Call Trace:
[Sat Apr 24 23:00:42 2021]  ? __schedule+0x2a2/0x870
[Sat Apr 24 23:00:42 2021]  schedule+0x28/0x80
[Sat Apr 24 23:00:42 2021]  io_schedule+0x12/0x40
[Sat Apr 24 23:00:42 2021]  cv_wait_common+0xaf/0x130 [spl]
[Sat Apr 24 23:00:42 2021]  ? finish_wait+0x80/0x80
[Sat Apr 24 23:00:42 2021]  txg_wait_synced_impl+0xc2/0x110 [zfs]
[Sat Apr 24 23:00:42 2021]  txg_wait_synced+0xc/0x40 [zfs]
[Sat Apr 24 23:00:42 2021]  zfsvfs_teardown+0x2fb/0x320 [zfs]
[Sat Apr 24 23:00:42 2021]  zfs_umount+0x30/0xe0 [zfs]
[Sat Apr 24 23:00:42 2021]  zpl_put_super+0x25/0x40 [zfs]
[Sat Apr 24 23:00:42 2021]  generic_shutdown_super+0x6c/0x100
[Sat Apr 24 23:00:42 2021]  kill_anon_super+0x14/0x30
[Sat Apr 24 23:00:42 2021]  deactivate_locked_super+0x2f/0x70
[Sat Apr 24 23:00:42 2021]  cleanup_mnt+0x3f/0x70
[Sat Apr 24 23:00:42 2021]  task_work_run+0x8a/0xb0
[Sat Apr 24 23:00:42 2021]  exit_to_usermode_loop+0xeb/0xf0
[Sat Apr 24 23:00:42 2021]  do_syscall_64+0xe1/0xf0
[Sat Apr 24 23:00:42 2021]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[Sat Apr 24 23:00:42 2021] RIP: 0033:0x56086d7f3f80
[Sat Apr 24 23:00:42 2021] Code: Bad RIP value.
[Sat Apr 24 23:00:42 2021] RSP: 002b:000000c05bec8700 EFLAGS: 00000202 ORIG_RAX: 00000000000000a6
[Sat Apr 24 23:00:42 2021] RAX: 0000000000000000 RBX: 000000c00005c000 RCX: 000056086d7f3f80
[Sat Apr 24 23:00:42 2021] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 000000c016e30c00
[Sat Apr 24 23:00:42 2021] RBP: 000000c05bec8758 R08: 0000000000000000 R09: 0000000000000000
[Sat Apr 24 23:00:42 2021] R10: 0000000000000000 R11: 0000000000000202 R12: ffffffffffffffff
[Sat Apr 24 23:00:42 2021] R13: 0000000000000021 R14: 0000000000000020 R15: 0000000000000055

rnz · 2021-04-24T21:17:41Z

May be I have damaged snap/clone, I can't destroy specific clones of pool-docker/2884eec69f65945cc341d87af691a57f79970f10b3cc3bb3d621688d6dedfb62@<any>

destroy operations is hung:

[Sun Apr 25 00:10:41 2021] INFO: task zfs:98572 blocked for more than 120 seconds.
[Sun Apr 25 00:10:41 2021]       Tainted: P           OE     4.19.98-custom #1
[Sun Apr 25 00:10:41 2021] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Sun Apr 25 00:10:41 2021] zfs             D    0 98572      1 0x00000000
[Sun Apr 25 00:10:41 2021] Call Trace:
[Sun Apr 25 00:10:41 2021]  ? __schedule+0x2a2/0x870
[Sun Apr 25 00:10:41 2021]  schedule+0x28/0x80
[Sun Apr 25 00:10:41 2021]  io_schedule+0x12/0x40
[Sun Apr 25 00:10:41 2021]  cv_wait_common+0xaf/0x130 [spl]
[Sun Apr 25 00:10:41 2021]  ? finish_wait+0x80/0x80
[Sun Apr 25 00:10:41 2021]  txg_wait_synced_impl+0xc2/0x110 [zfs]
[Sun Apr 25 00:10:41 2021]  txg_wait_synced+0xc/0x40 [zfs]
[Sun Apr 25 00:10:41 2021]  dsl_sync_task_common+0x1b5/0x290 [zfs]
[Sun Apr 25 00:10:41 2021]  ? dsl_destroy_head_sync_impl+0xde0/0xde0 [zfs]
[Sun Apr 25 00:10:41 2021]  ? dsl_destroy_head_check_impl+0x1d0/0x1d0 [zfs]
[Sun Apr 25 00:10:41 2021]  ? dsl_destroy_head_check_impl+0x1d0/0x1d0 [zfs]
[Sun Apr 25 00:10:41 2021]  ? dsl_destroy_head_sync_impl+0xde0/0xde0 [zfs]
[Sun Apr 25 00:10:41 2021]  dsl_sync_task+0x16/0x20 [zfs]
[Sun Apr 25 00:10:41 2021]  dsl_destroy_head+0xf0/0x180 [zfs]
[Sun Apr 25 00:10:41 2021]  zfs_ioc_destroy+0xa6/0x160 [zfs]
[Sun Apr 25 00:10:41 2021]  ? spa_name_compare+0xa/0x20 [zfs]
[Sun Apr 25 00:10:41 2021]  ? avl_find+0x58/0x90 [zavl]
[Sun Apr 25 00:10:41 2021]  ? refcount_inc_checked+0x5/0x30
[Sun Apr 25 00:10:41 2021]  ? apparmor_capable+0x6b/0xc0
[Sun Apr 25 00:10:41 2021]  ? refcount_inc_checked+0x5/0x30
[Sun Apr 25 00:10:41 2021]  ? apparmor_capable+0x6b/0xc0
[Sun Apr 25 00:10:41 2021]  ? security_capable+0x38/0x50
[Sun Apr 25 00:10:41 2021]  ? _cond_resched+0x15/0x30
[Sun Apr 25 00:10:41 2021]  ? __kmalloc_node+0x1e1/0x2b0
[Sun Apr 25 00:10:41 2021]  ? spl_kmem_alloc+0xc6/0x110 [spl]
[Sun Apr 25 00:10:41 2021]  ? spl_kmem_alloc+0xc6/0x110 [spl]
[Sun Apr 25 00:10:41 2021]  ? strlcpy+0x2d/0x40
[Sun Apr 25 00:10:41 2021]  zfsdev_ioctl_common+0x5b5/0x830 [zfs]
[Sun Apr 25 00:10:41 2021]  ? kmalloc_large_node+0x37/0x60
[Sun Apr 25 00:10:41 2021]  ? __kmalloc_node+0x20a/0x2b0
[Sun Apr 25 00:10:41 2021]  zfsdev_ioctl+0x4f/0xe0 [zfs]
[Sun Apr 25 00:10:41 2021]  do_vfs_ioctl+0xa4/0x630
[Sun Apr 25 00:10:41 2021]  ? do_munmap+0x33c/0x430
[Sun Apr 25 00:10:41 2021]  ksys_ioctl+0x60/0x90
[Sun Apr 25 00:10:41 2021]  __x64_sys_ioctl+0x16/0x20
[Sun Apr 25 00:10:41 2021]  do_syscall_64+0x50/0xf0
[Sun Apr 25 00:10:41 2021]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[Sun Apr 25 00:10:41 2021] RIP: 0033:0x7f3d531e5427
[Sun Apr 25 00:10:41 2021] Code: Bad RIP value.
[Sun Apr 25 00:10:41 2021] RSP: 002b:00007ffeff096488 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[Sun Apr 25 00:10:41 2021] RAX: ffffffffffffffda RBX: 00007ffeff0964b0 RCX: 00007f3d531e5427
[Sun Apr 25 00:10:41 2021] RDX: 00007ffeff0964b0 RSI: 0000000000005a18 RDI: 0000000000000005
[Sun Apr 25 00:10:41 2021] RBP: 00007ffeff099aa0 R08: 00005653f193b238 R09: 00005653f193b230
[Sun Apr 25 00:10:41 2021] R10: 0000000000000007 R11: 0000000000000246 R12: 0000000000000000
[Sun Apr 25 00:10:41 2021] R13: 0000000000005a18 R14: 0000000000005a18 R15: 00005653f193b230

I was run zfs scrub - it was work 30min and increase hung_task_timeout_secs to 1200 - now zfs destroy -R is work (I launched it in a sequence to remove all wrong snaps) but that is very slow

rnz · 2021-04-24T22:55:42Z

zfs destroy -R is taken more then 2 minutes:

pool-docker/2884eec69f65945cc341d87af691a57f79970f10b3cc3bb3d621688d6dedfb62@524842236

real    2m23,329s
user    0m0,007s
sys     0m0,008s
pool-docker/2884eec69f65945cc341d87af691a57f79970f10b3cc3bb3d621688d6dedfb62@811039990

real    2m22,355s
user    0m0,005s
sys     0m0,011s
pool-docker/2884eec69f65945cc341d87af691a57f79970f10b3cc3bb3d621688d6dedfb62@710095942

real    2m17,401s
user    0m0,004s
sys     0m0,021s
pool-docker/2884eec69f65945cc341d87af691a57f79970f10b3cc3bb3d621688d6dedfb62@749651645

real    2m22,785s
user    0m0,004s
sys     0m0,014s

and

# time zfs list -t all -o name,origin,clones is taken 5 min:

real	4m57,811s
user	0m1,808s
sys	0m57,851s

Is zfs destroy try to get full list of snaps on each run?

added: I change storage (move to all flash storage connected by FC 16G) - same slow zfs destroy -R :

pool-docker/2884eec69f65945cc341d87af691a57f79970f10b3cc3bb3d621688d6dedfb62@909902530

real    2m42,024s
user    0m0,004s
sys     0m0,010s

I suppose that the fact is that every time zfs destroy is launched, the entire list of snaps/clones on the pool is read and when there are many of them, we have a problem.

bghira · 2021-04-25T19:12:20Z

curious if you are using dedup

rnz · 2021-04-25T23:59:11Z

@bghira

curious if you are using dedup

no

rnz · 2021-04-26T11:19:38Z

Few details:

# docker ps -a | grep -EIho 'Exit|Dead|Created' | sort | uniq -c

    231 Created
   6394 Dead
   3818 Exit

Docker try to remove 3818 Exit containers on start if previous run is hang or failed. That was forked zfs destroy by each exit container (3818 processes of zfs destroy -r) - and I/O is grow up to top. Also if docker can't over some operations, it try to restart containers (under service, docker swarm) and again forked many zfs snapshot and destroy:

And each fail grow up created, dead and exit containers with zfs snapshots.
Since the zfs destroy -r be carried out very slowly, it makes no sense to wait the completion of thousands of such processes

stooone · 2021-05-01T13:06:04Z

Maybe related: #11480

KyleSanderson · 2022-03-13T04:50:57Z

I'm seeing the exact same behaviour. It's convinced me that ZFS is completely unusable when zfs list takes minutes to complete and zsysd has a hard coded giveup after 30 seconds. I'm on Ubuntu 20.04, using containerd which seems to make snapshots on snapshots on snapshots leaving around thousands of orphans.

rincebrain · 2022-03-13T05:16:44Z

I believe the "livelist" feature in 2.0.X should improve your performance markedly, though 20.04 is shipping 0.8.X, so you'd have to go out of your way to try it. (Assuming it's a lot of clones that you're destroying, which seems safe, from my understanding of what containerd does.)

(It also, I assume, wouldn't help for existing clones, only ones made after enabling the feature, because of how it works, but that's just an assumption on my part, I haven't looked to see if it has any idea of how to backfill that information if activated later.)

ryao · 2023-03-13T04:24:54Z

#14603 might help here.

stale · 2024-03-13T10:26:50Z

This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.

KyleSanderson · 2024-03-14T03:35:15Z

.

Neurrone · 2025-01-03T09:57:50Z

This triggers for me if I recursively delete a dataset. One of the snapshots is causing a panic, see #15030 (comment)

rnz added Status: Triage Needed New issue which needs to be triaged Type: Defect Incorrect behavior (e.g. crash, hang) labels Apr 23, 2021

rnz changed the title ~~Very slow zfs destroy~~ Very slow zfs destroy and snapshot Apr 23, 2021

rnz changed the title ~~Very slow zfs destroy and snapshot~~ Very slow zfs destroy Apr 23, 2021

aivanise mentioned this issue Dec 6, 2022

wish: implement a retry policy Patryk27/lxd-snapper#12

Open

stale bot added the Status: Stale No recent activity for issue label Mar 13, 2024

stale bot removed the Status: Stale No recent activity for issue label Mar 14, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Very slow zfs destroy #11933

Very slow zfs destroy #11933

rnz commented Apr 23, 2021 •

edited

Loading

rincebrain commented Apr 23, 2021

ahrens commented Apr 23, 2021

rnz commented Apr 23, 2021 •

edited

Loading

ahrens commented Apr 23, 2021

rnz commented Apr 23, 2021 •

edited

Loading

rincebrain commented Apr 23, 2021 •

edited

Loading

rnz commented Apr 23, 2021 •

edited

Loading

rnz commented Apr 24, 2021 •

edited

Loading

rnz commented Apr 24, 2021 •

edited

Loading

rnz commented Apr 24, 2021 •

edited

Loading

bghira commented Apr 25, 2021

rnz commented Apr 25, 2021

rnz commented Apr 26, 2021 •

edited

Loading

stooone commented May 1, 2021

KyleSanderson commented Mar 13, 2022

rincebrain commented Mar 13, 2022 •

edited

Loading

ryao commented Mar 13, 2023

stale bot commented Mar 13, 2024

KyleSanderson commented Mar 14, 2024

Neurrone commented Jan 3, 2025

Very slow zfs destroy #11933

Very slow zfs destroy #11933

Comments

rnz commented Apr 23, 2021 • edited Loading

System information

rincebrain commented Apr 23, 2021

ahrens commented Apr 23, 2021

rnz commented Apr 23, 2021 • edited Loading

ahrens commented Apr 23, 2021

rnz commented Apr 23, 2021 • edited Loading

rincebrain commented Apr 23, 2021 • edited Loading

rnz commented Apr 23, 2021 • edited Loading

rnz commented Apr 24, 2021 • edited Loading

rnz commented Apr 24, 2021 • edited Loading

rnz commented Apr 24, 2021 • edited Loading

bghira commented Apr 25, 2021

rnz commented Apr 25, 2021

rnz commented Apr 26, 2021 • edited Loading

stooone commented May 1, 2021

KyleSanderson commented Mar 13, 2022

rincebrain commented Mar 13, 2022 • edited Loading

ryao commented Mar 13, 2023

stale bot commented Mar 13, 2024

KyleSanderson commented Mar 14, 2024

Neurrone commented Jan 3, 2025

rnz commented Apr 23, 2021 •

edited

Loading

rnz commented Apr 23, 2021 •

edited

Loading

rnz commented Apr 23, 2021 •

edited

Loading

rincebrain commented Apr 23, 2021 •

edited

Loading

rnz commented Apr 23, 2021 •

edited

Loading

rnz commented Apr 24, 2021 •

edited

Loading

rnz commented Apr 24, 2021 •

edited

Loading

rnz commented Apr 24, 2021 •

edited

Loading

rnz commented Apr 26, 2021 •

edited

Loading

rincebrain commented Mar 13, 2022 •

edited

Loading