OpenZFS 6569 - large file delete can starve out write ops

[gmelikov]

Authored by: Alek Pinchuk alek@nexenta.com
Reviewed by: Matt Ahrens mahrens@delphix.com
Reviewed by: Sanjay Nadkarni sanjay.nadkarni@nexenta.com
Reviewed by: Pavel Zakharov pavel.zakharov@delphix.com
Ported-by: George Melikov mail@gmelikov.ru

OpenZFS-issue: https://www.illumos.org/issues/6569
OpenZFS-commit: openzfs/openzfs@1bf4b6f

[mention-bot]

@gmelikov, thanks for your PR! By analyzing the history of the files in this pull request, we identified @behlendorf, @ahrens and @grwilson to be potential reviewers.

[gmelikov]

This is a ready to commit version of #5449, i ported openzfs/openzfs@1bf4b6f again.

@behlendorf i had a compiler message (already fixed in code):

../../module/zfs/dmu.c: In function 'dmu_free_long_range_impl':
../../module/zfs/dmu.c:777:3: error: 'for' loop initial declarations are only allowed in C99 or C11 mode
   for (int t = 0; t < TXG_SIZE; t++) {
   ^
../../module/zfs/dmu.c:777:3: note: use option -std=c99, -std=gnu99, -std=c11 or -std=gnu11 to compile your code

thought that it's behaviour was changed in #5686. Can we disable this warning too?

[behlendorf]

@gmelikov thanks for porting this. This is a slightly different warning. I'm not aware of a safe gcc option to suppress it for C90 builds. There may be one but it'll require more investigation. For the moment, you'll need to address it as you've done.

The other issue here will be the need for a new free__long__range dtrace probe. You'll need to add it to include/sys/trace_dmu.h and it can be modeled after the existing ones.

fs/zfs/zfs/dmu.c: In function ‘dmu_free_long_range_impl’:
fs/zfs/zfs/dmu.c:813:3: error: implicit declaration of function ‘trace_zfs_free__long__range’ [-Werror=implicit-function-declaration]
   DTRACE_PROBE3(free__long__range,
   ^
cc1: some warnings being treated as errors

[gmelikov]

@behlendorf thank you, added the free__long__range dtrace probe.

[kernelOfTruth]

Been using it in production with an rt-kernel ever since #5449 was available, no issues or regressions observed so far

[behlendorf]

LGTM. Thanks, and nice job on the trace point, that isn't the easiest code to grok.