Put in a warning about dedup requiring lots and lots of memory.

[FransUrbo]

Figues might differ and depend on usage and data stored, but a firm
warning is required - people are trying to use this on machines
with 8GB or even 4GB memory!

[rottegift]

While you’re there you might want to also point out the impact of large numbers of non-unique blocks on resilver times.
(See the comment above dsl_scan_ddt() in dsl_scan.c). Every non-unique block involves highly random IO, so for example large amounts of lightly-deduplicated (e.g. refcount=2) blocks on a pool with a single large rotating-disk raidz1 storage vdev will make the early part of a resilver take a lonnnng time even if you have lots of system RAM. =

[FransUrbo]

I've revised it twice, and decided in the end to remove any speculation and settle with "don't use unless you know what you're doing". This should caution users and make them investigate the issue further and not just "enable it because it sounds cool/nice".

[behlendorf]

That's definitely a scary warning. Dedup might cause you significant performance problems but it shouldn't ever result in pool loss. Are you thinking about cases where it takes many days to import the pool due to the pool performance?

[FransUrbo]

Are you thinking about cases where it takes many days to import the pool due to the pool performance?

Yes and no. I've seen at least one having problem importing the pool on the list. It took days,
but finally crashed because ZoL ate all the memory. So "in practice" the pool was "lost".
More memory could [probably] help, but in that case there was no physical possibility to
add more..

It wasn't lost in the sense that it could never, ever be imported again. It was "lost" in the
sense that the user simply wasn't able to import it on any of the hardware he had (nor was

able/could afford to buy)...

Ehhhhm - The battle cry of the cronical masturbater.

• Charlie Harper

[behlendorf]

I have seen reports of problems importing dedup'ed pools due to memory exhaustion. I've also seen people reporting horrible performance problems. But at least in my mind those are both very different things than pool loss. They're bugs which need to be reported so we can address them.

I have no objection what so ever to warning people about enabling deduplication. But let's update the warning in the man page to accurately reflect the known risks.

[FransUrbo]

Fair enough. How about this:

       WARNING: Please don't enable this unless you know exactly what you're doing! It can be very
       dangerous and can result in the pool being unimportable because of memory exhaustions if you
       don't have enough memory.

[behlendorf]

I can get on board with something like that.

[behlendorf]

How about something like this?

       Note  that enabling deduplication greatly increases the memory require-
       ments.  If your system does not have enough memory this can be  danger-
       ous  because  it  leads to memory exhaustion and extremely poor perfor-
       mance.  Please be careful and only enable this functionality on a prop-
       erly provisioned system.

[FransUrbo]

Don't feel that it's strong enough... leads to memory exhaustion and extremely poor performance. . Hell, if that was the truth, the whole truth and nothing but the truth, I'd enable it right now! :).

I think users needs to get a good kick in the teeth. If they survive that, then by all means enable dedup... :)

[behlendorf]

Maybe extremely poor performance should be quantified more. But dedup should never eat your pool and if it does that's a bug we need to fix.

[dswartz]

Users MUST understand the full scope of consequences for enabling dedup.
Even if it's a design flaw that cannot be overcome, OOM and inability to
import pool is not a good situation to be in. Are you going to loan
machines with 512GB RAM to users who enable the feature and can't disable
it?

Agreed. If I got bit by this and had to choose between waiting for an
unknown amount of time (and it's been 2 days already) vs destroying the
pool and restoring from backups, that's 'data loss' to me...

[behlendorf]

OK, can someone please suggest some stern but polite text which explains this.

[behlendorf]

@kpande the fact that we don't know is clearly part of the problem...

[FransUrbo]

OK, can someone please suggest some stern but polite text which explains this.

Anything not polite with my last attempt (with some addition of performance issues)?

WARNING: Please don't enable this unless you know exactly what you're doing! It can be very
dangerous, can result in extreme performance issues as well as result in the pool being
unimportable because of memory exhaustions if you don't have enough memory and have planned
your hardware appropriately.
```=

[Sachiru]

Minor tweak to your text, FransUrbo:

WARNING: DO NOT ENABLE DEDUPLICATION UNLESS YOU NEED IT AND KNOW EXACTLY WHAT YOU ARE DOING!
Enabling deduplication on an improperly-designed system will result in extreme performance issues and can lead to data loss (i.e. unimportable pool due to memory exhaustion, extremely slow snapshot deletion, etc.).
Before creating a pool with deduplication enabled, ensure that you have planned your hardware requirements appropriately. If you enable deduplication on your pool, you accept the risks and responsibilities that are part of deduplication, and you acknowledge that you need to implement deduplication with the appropriate data and software recovery practices (offsite backups, etc.) in place.

[FransUrbo]

Minor tweak to your text, FransUrbo:

Like it. Might be a little "chatty" though :)

[Sachiru]

Minifying:

-- Dedup Warning Message -
WARNING: DO NOT ENABLE DEDUPLICATION UNLESS YOU NEED IT AND KNOW EXACTLY WHAT YOU ARE DOING!
Enabling deduplication on an improperly-designed system will result in extreme performance issues and can lead to data loss (i.e. unimportable pool due to memory exhaustion, extremely slow snapshot deletion, etc.).
Before creating a pool with deduplication enabled, ensure that you have planned your hardware requirements appropriately and implement appropriate recovery practices such as regular offsite backups.
-- End Dedup Warning Message --

Perhaps we can also alter the man page for zfs create so that it adds this text to the dedup options pane:

-- Begin MAN page --
Deduplication

Deduplication is the process for removing redundant data at the block-level, reducing the total amount of data stored. If a file system has the dedup property enabled, duplicate data blocks are removed synchronously. The result is that only unique data is stored and common components are shared among files.
Unless it is absolutely necessary and you are completely sure that your hardware can support it, do NOT enable deduplication. Consider using compression=lz4 instead for a less resource-intensive alternative.

---

dedup=on | off | verify | sha256[,verify]
Controls whether deduplication is in effect for a dataset. The default value is off. The default checksum used for deduplication is sha256 (subject to change). When dedup is enabled, the dedup checksum algorithm overrides the checksum property. Setting the value to verify is equivalent to specifying sha256,verify.
If the property is set to verify, then, whenever two blocks have the same signature, ZFS will do a byte-for-byte comparison with the existing block to ensure that the contents are identical.
WARNING: Enabling deduplication can result in data loss. Deduplicating data is a very resource-intensive operation. It is generally recommended that you have 1.25 GB of RAM per 1 TB of storage when you enable deduplication. Deduplication on systems with multiple snapshots can cause memory exhaustion, preventing the pool from being imported. Destroying deduplicated pools, datasets, filesystems or snapshots of the above can cause your system to stop responding.
Unless necessary, deduplication should NOT be enabled on a system. Using compression=lz4 instead will provide similar benefits with less resource usage and a lesser chance of losing data.

-- End MAN page --

[FransUrbo]

Thanx @Sachiru for your comments. I think your last one was a little repetitive and "wordy", but over all very good.

I've taken a lot of that and pushed a new version. Please tell me what you think.

[behlendorf]

I like this a lot better. I should be able to get it merged today unless someone wants to suggest additional updates.