Can't get fastmultigather to write to a specific output file

[AnneliektH]

Running fastmultigather on a set of metagenome reads x a set of zip files and i want the output to be named as metagenome.x.zip.csv

Running as:
sourmash scripts fastmultigather ../../sourmash_sketches/sketch_reads/ERR1135178.k21.zip ../../sourmash_sketches/split_sig/ERR11351_1.zip --output ERR1135178xERR11351_1.csv -k 21 -t 1000 -s 100 -c 1

Gives me 2 output files: ERR1135178.gather.csv and ERR1135178.gather.csv, which is not what I want

[ctb]

right, see #239 - --output only works with a rocksdb. We have two quite different modes of operation of fastmultigather that need to converge at some point :).

[AnneliektH]

Ahh too bad. Would there be a way to work around this using zip files? as i currently cannot get rockdbs to work

[bluegenes]

@ctb are you ok with switching to single-file output for all queries? Or should we aim to support both?

[ctb]

@ctb are you ok with switching to single-file output for all queries? Or should we aim to support both?

Two hot takes -

I'm kind of against switching things up wilhe-nilhe just because? The branchwater plugin is becoming a bit of a UX nightmare already and changing up what we do in the middle of a point release seems suboptimal. We've done it before, we'll do it again, but I question the necessity here ;). Can't we just give @amterhorst a csvtk command that aggregates all of the CSV files into one or something??

I would be more supportive of making it so that when -o is give, it aggregates all the output into a single file, but when it is not given, it sticks with the current behavior. I fear that will lead to a lot of code nastiness, which I dislike more than confusing users for this particular package.

This is all in the background of a complicated set of design decisions discussed over in sourmash-bio/sourmash#2328, for which I have no answer.

So, umm, after all of this - yeah, sure, switch to -o aggregating all output into a single file. It'll be a required parameter now, right?

[AnneliektH]

Ok: rocksdb not working was my fault, as i tried to create directly from a sig.gz file, not a list of filepaths.
So I can create output files with a specific name now

As for a csvtk command, that be great, but my issue is that the output file gets overwritten.
Im trying to run the same metagenome against different sets of databases, and output files are named after the metag.

Doing this because if using one db, it doesn't finish.

Anyway: Rocksdb fixed my problem.

[ctb]

As for a csvtk command, that be great, but my issue is that the output file gets overwritten.
Im trying to run the same metagenome against different sets of databases, and output files are named after the metag.

True!

Another workaround is to run it in a separate directory.