Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support --ungroup option like GNU-parallel #21

Open
zakstucke opened this issue May 14, 2024 · 2 comments
Open

support --ungroup option like GNU-parallel #21

zakstucke opened this issue May 14, 2024 · 2 comments

Comments

@zakstucke
Copy link

zakstucke commented May 14, 2024

Love the look of the library.

Looking to replace GNU-parallel with this, but seems like --ungroup isn't an option.
In my case, I do want output interleaving as I'm running "infinite" processes, with the current --group behaviour I'll never see those logs come out.

Thanks!

P.s I've just seen in 1.16.0 the behaviour of --exit-on-error was changed to wait for in-process tasks to finish before exiting. Given my intended use case I'd also need the previous behaviour. Maybe 3 separate options needed? I.e.

  • --exit-on-error -> current behaviour
  • --kill-on-error-hard -> SIGTERM+SIGHUP all ongoing processes if any error
  • --kill-on-error-soft -> SIGKILL all ongoing processes if any error
@aaronriekenberg
Copy link
Owner

aaronriekenberg commented May 15, 2024

@zakstucke you are right the --ungroup option does not exist now. Outputs only happen after each process exits which is similar to the --group behavior in GNU Parallel.

Could you please give an example command line where you want to use --ungroup? Would like to understand more on use case for this.

On the --exit-on-error behavior - you are correct there was a change in version 1.16.0:

  • In 1.15.0 if --exit-on-error was specified rust-parallel would immediately exit with status 1 after first command fails
  • In 1.16.0 this behavior changed so on errors rust-parallel waits for in progress commands to finish before exiting with status 1

Could you please write a second issue for more exit-on-error options?

Thanks!

@zakstucke
Copy link
Author

zakstucke commented May 16, 2024

@aaronriekenberg my specific use case:

  • Running multiple processes from a single docker instance, e.g. a webserver + background task processor.
  • I used to do this using GNU parallel, but for various reasons it's caused me problems, and I wanted a simple binary parallel runner instead, so rust-parallel sounded like a good fit.
  • In regards to --ungroup, logging is done separately, but some command line stdout/stderr exists and needs to be seen. Given all processes are ideally "endless", this needs to be printed to console straight away with no buffer, otherwise it will be i'm assuming endlessly buffered inside rust-parallel.
  • In regards to --exit-on-error, as I mentioned above, I need the container to exit if any process dies, but given each process is infinite, unless a termination is sent to the other processes, the docker container will not exit. The option for different kill signals being sent to the healthy processes allows the user to "softly" kill processes with e.g. SIGTERM, or instantly with SIGINT.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants