Documentation for using Compress/Decompress?

[gaby]

I noticed that fsspec has fsspec.compression but it's not documented anywhere how to use it. Even a simple example would help on how to create a fs using fsspec and write files compressed to it.

The other thing I read is that fsspec doesn't have compression dependencies, if I want to use zstd, which package am I suppose to add for that to work with fsspec ?

[martindurant]

The code is reached via the compression= kwargs in fsspec.open:

    compression: string or None
        If given, open file using compression codec. Can either be a compression
        name (a key in ``fsspec.compression.compr``) or "infer" to guess the
        compression from the filename suffix.

For Zstd, the import is zstandard, but you are right, documentation on what these are and what to do when requesting a compression that is known but not installed would be very useful. Would you like to work on this?

[gaby]

That means the filesystem perse is compressed? How about individual files is that something filesystem_spec supports or should I use something else to compress the file then add it to a fsspec/filesystem_spec ?

[martindurant]

Yes, this is for opening individual files.

the filesystem perse is compressed

Sorry, I didn't understand this.

[gaby]

Yes, this is for opening individual files.

the filesystem perse is compressed

Sorry, I didn't understand this.

I have no idea what I wrote. Haha. I will let someone else document this, there's a lot of sections missing docs, so I can't really track what to put where.