proposing new --format=<format> --out-file=<file.ext> feature #17507

memsharded · 2024-12-19T00:15:08Z

Changelog: Feature: Allow defining --out-file=file.ext instead of --format=ext > file.ext to write to files directly and avoid issues with redirects.
Docs: conan-io/docs#3966

This would solve issues with stdout/stderr redirects and potential pollution, while not using new arguments, not breaking existing code and allow easy introduction to existing formatters.

Close #17462

czoido · 2024-12-19T06:31:22Z

Related to #17462

AbrilRBS

Question, otherwise looks good, but I think checking with the team in as look-into is a good idea first, good call

AbrilRBS · 2024-12-19T09:26:26Z

conan/api/output.py

    """
    Output to be used by formatters to dump information to stdout
    """
+    if filename is not None:
+        ConanOutput().info(f"Formatted output saved to '{filename}'")
+        save(filename, data)


There are a few commands whose formatter calls cli_out_write multiple times in a row.

Should we convert those to only call it once, or should this new functionality support appending?

Good catch. Not sure to be honest, I tend to think to re-write the formatter to include only 1 call, but there are some case that use colors and that would be a bit less elegant, but still doable

czoido · 2024-12-19T10:38:54Z

I'm not sure I prefer this over having a new --output-file argument to specify the file... I was thinking... what about something at the midpoint like: --format="json:myfile.json" ?

memsharded · 2024-12-19T10:42:24Z

I'm not sure I prefer this over having a new --output-file argument to specify the file... I was thinking... what about something at the midpoint like: --format="json:myfile.json" ?

Yes, I like that midpoint. I thought about the extra argument, but that further pollutes the CLI, like --format=json --output-file=graph.json is like "Hey, I am already telling you in the filename that I want a json file, don't be that dumb".

But if you think the --output-file is better, we can do it too, I am not strongly opposed either.

perseoGI · 2024-12-19T10:52:12Z

I agree with having another CLI option, it might be confusing that the fact of defining an output format could also lead to defining the output file.

In conanlint we have made something similar:

Also, it may cause issues with future formats which could contain a dot delimiter.

All in all, the feature is a good one! Some console redirections are not always consistent.
(In the linter I add this option to allow progress bar + redirection to file)

czoido · 2024-12-19T11:12:06Z

I'm not sure I prefer this over having a new --output-file argument to specify the file... I was thinking... what about something at the midpoint like: --format="json:myfile.json" ?

Yes, I like that midpoint. I thought about the extra argument, but that further pollutes the CLI, like --format=json --output-file=graph.json is like "Hey, I am already telling you in the filename that I want a json file, don't be that dumb".

But if you think the --output-file is better, we can do it too, I am not strongly opposed either.

The main thing for me is to maintain the concept of a formatter. A --format=file.json argument doesn’t feel like a formatter anymore. The original idea was that the formatter should define how the output is formatted (HTML, JSON, etc.), not where it’s written. So, for me, any idea that maintains in some way the meaning of the formatter is ok.

memsharded

Looking good

memsharded · 2024-12-19T18:12:31Z

conan/cli/commands/graph.py

@@ -192,6 +192,9 @@ def graph_info(conan_api, parser, subparser, *args):
                           help="Deployer output folder, base build folder by default if not set")
    subparser.add_argument("--build-require", action='store_true', default=False,
                           help='Whether the provided reference is a build-require')
+    subparser.add_argument("--out-file", action=OnceArgument,


This should be injected automatically and conditionally to the existence of formatters together with the definition of the --format argument. There cannot be a --out-file without a --format definition.

Finally, my proposal is to add the --out-file always, to enable users do for example
conan config home --out-file=file.ext
and it uses the text formatter that is the one by default. Because in the end, there's always a formatter there that is the text one even if it does not show in the --help, but let's discuss about this with the team.

EDIT: Looks like this works in the test suite but not in my CLI, I'm leaving the PR in draft until I understand why.

czoido · 2024-12-20T07:30:39Z

I'm leaving this as draft, apparently these lines:

conan/conan/api/output.py

Lines 298 to 300 in 003e2cf

    
           colorama.deinit() 
        
           sys.stdout.write(data) 
        
           colorama.reinit()

are making that this approach redirecting the stdout do not work on a real console but they do work on testing because we capture the streams. I'll investigate a solution that also prevent #17245 to not happen any more.

conan/cli/command.py

czoido · 2024-12-30T16:25:29Z

conan/cli/command.py

            parser.add_argument('-f', '--format', action=OnceArgument, help=help_message)

+        parser.add_argument("--out-file", action=OnceArgument,


finally keeping this unconditionally as a direct replacement to redirect output with >

memsharded

Looks good to me, can be merged.

…io#17507) * proposing new --format=file.ext feature * change approach and some simplifications * remove unused * minor changes * minor changes * simplify a bit * fix format * minor changes * fix * move colorama deinit/reinit * add out-file only if formatters * revert * wip * do not deinit/reinit if not tty * clean import * minor changes --------- Co-authored-by: czoido <mrgalleta@gmail.com>

* extra patches path * Update conan/tools/files/patches.py Co-authored-by: Andrey Filipenkov <kambaladecapitator@gmail.com> * fix docstring of Environment.vars() (#17521) fix docstring of vars() * GnuToolchain's make_args handle empty values correctly (#17532) test * Change replace_in_file to return True on success (#17531) - was inconsistent before: returned False if the pattern was not found (with strict off), otherwise None * upgrade bottle version (#17534) * better traces and msgs (#17517) * better traces and msgs * fix tests * Feature/workspace api (#17501) * working in workspace_api * fixed loader + run() error * update webtest testing version (#17537) * proposing new --format=<format> --out-file=<file.ext> feature (#17507) * proposing new --format=file.ext feature * change approach and some simplifications * remove unused * minor changes * minor changes * simplify a bit * fix format * minor changes * fix * move colorama deinit/reinit * add out-file only if formatters * revert * wip * do not deinit/reinit if not tty * clean import * minor changes --------- Co-authored-by: czoido <mrgalleta@gmail.com> * Test/remote login (#17552) * new test for remote_login * unittest->pytest * Keep sessions around between each ConanRequester (#17455) * Keep sessions around between each ConanRequester Avoid creating a new session for each ConanRequester instance, this helps a lot with performance, as it avoids the overhead duplicate handshakes * Patch new cached value on ConanRequester tests * Modify adapter as needed * Fix tests * Name * Update conans/client/rest/conan_requester.py * Refactor requester tests * Reinit is not part of this PR * Refactor request usage out of app * Make max_retries a local variable * Add rest of posible `ConanBasicApp` usages (#17555) Add rest of posible ConanBasicApp usages * Improve typing information in remove sub API (#17564) As the API actually accepts the remote to be `None`, this should also be reflected in the type information in order to support users. * small refactor to PkgConfigDeps (#17568) * Updated to use maxsplit=1 (#17571) * Updated to use maxsplit=1 * Add tests, unquote paths with spaces --------- Co-authored-by: Abril Rincón Blanco <git@rinconblanco.es> * refactor moving model conans->conan (#17572) * refactor moving model conans->conan * fix tests * allow non trim_conandata * removed print --------- Co-authored-by: Andrey Filipenkov <kambaladecapitator@gmail.com> Co-authored-by: Andrey Filipenkov <decapitator@ukr.net> Co-authored-by: Ernesto de Gracia Herranz <ernestodgh@jfrog.com> Co-authored-by: mrbean-bremen <mrbean-bremen@users.noreply.github.com> Co-authored-by: czoido <mrgalleta@gmail.com> Co-authored-by: Abril Rincón Blanco <rubenrb@jfrog.com> Co-authored-by: Martin Lambertsen <github@lambertsen.one> Co-authored-by: maitrey <maitreymishra23@gmail.com> Co-authored-by: Abril Rincón Blanco <git@rinconblanco.es>

proposing new --format=file.ext feature

410372a

memsharded added this to the 2.12.0 milestone Dec 19, 2024

memsharded assigned czoido Dec 19, 2024

memsharded added the type: look into label Dec 19, 2024

memsharded requested a review from AbrilRBS December 19, 2024 00:15

Todiq mentioned this pull request Dec 19, 2024

[bug] UTF8 error on Windows #16921

Closed

AbrilRBS reviewed Dec 19, 2024

View reviewed changes

AbrilRBS mentioned this pull request Dec 19, 2024

[package] cryptopp/8.9.0: Building with MSVC for ARM attempts to compile the intel only file rdrand.asm conan-io/conan-center-index#26205

Open

change approach and some simplifications

8a2a443

czoido marked this pull request as draft December 19, 2024 17:53

czoido added 3 commits December 19, 2024 18:54

remove unused

463493e

minor changes

0b5a991

minor changes

859c512

memsharded commented Dec 19, 2024

View reviewed changes

czoido changed the title ~~proposing new --format=file.ext feature~~ proposing new --format=<format> --out-file=<file.ext> feature Dec 20, 2024

czoido added 2 commits December 20, 2024 07:39

simplify a bit

434da37

fix format

1f72ab9

czoido marked this pull request as ready for review December 20, 2024 06:45

czoido added 3 commits December 20, 2024 07:45

Merge branch 'develop2' into feature/output_files

65a1dc1

minor changes

0a06ba3

fix

9f103e0

czoido marked this pull request as draft December 20, 2024 07:03

Merge branch 'develop2' into feature/output_files

2f5af33

memsharded commented Dec 30, 2024

View reviewed changes

conan/cli/command.py Show resolved Hide resolved

czoido added 2 commits December 30, 2024 14:39

move colorama deinit/reinit

e50e93e

add out-file only if formatters

04d4a65

czoido marked this pull request as ready for review December 30, 2024 14:02

memsharded commented Dec 30, 2024

View reviewed changes

conan/cli/command.py Outdated Show resolved Hide resolved

conan/cli/command.py Outdated Show resolved Hide resolved

revert

43bed9a

czoido marked this pull request as draft December 30, 2024 15:02

czoido added 3 commits December 30, 2024 17:07

wip

f4cb4fe

do not deinit/reinit if not tty

7108dd3

clean import

44f61e0

czoido reviewed Dec 30, 2024

View reviewed changes

czoido marked this pull request as ready for review December 30, 2024 16:26

minor changes

ac895bf

memsharded commented Dec 30, 2024

View reviewed changes

czoido added type: feature and removed type: look into labels Dec 30, 2024

czoido merged commit be3c03e into conan-io:develop2 Dec 30, 2024
33 checks passed

memsharded deleted the feature/output_files branch December 30, 2024 17:30

memsharded mentioned this pull request Jan 8, 2025

[question] Parallel jobs and independent caches #17545

Open

1 task

czoido mentioned this pull request Jan 27, 2025

Docs for --out-file conan-io/docs#3966

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

proposing new --format=<format> --out-file=<file.ext> feature #17507

proposing new --format=<format> --out-file=<file.ext> feature #17507

memsharded commented Dec 19, 2024 •

edited by czoido

Loading

czoido commented Dec 19, 2024

AbrilRBS left a comment •

edited

Loading

AbrilRBS Dec 19, 2024

memsharded Dec 19, 2024

czoido commented Dec 19, 2024

memsharded commented Dec 19, 2024

perseoGI commented Dec 19, 2024

czoido commented Dec 19, 2024 •

edited

Loading

memsharded left a comment

memsharded Dec 19, 2024

czoido Dec 20, 2024 •

edited

Loading

czoido commented Dec 20, 2024

czoido Dec 30, 2024

memsharded left a comment

		parser.add_argument('-f', '--format', action=OnceArgument, help=help_message)

		parser.add_argument("--out-file", action=OnceArgument,

proposing new --format=<format> --out-file=<file.ext> feature #17507

proposing new --format=<format> --out-file=<file.ext> feature #17507

Conversation

memsharded commented Dec 19, 2024 • edited by czoido Loading

czoido commented Dec 19, 2024

AbrilRBS left a comment • edited Loading

Choose a reason for hiding this comment

AbrilRBS Dec 19, 2024

Choose a reason for hiding this comment

memsharded Dec 19, 2024

Choose a reason for hiding this comment

czoido commented Dec 19, 2024

memsharded commented Dec 19, 2024

perseoGI commented Dec 19, 2024

czoido commented Dec 19, 2024 • edited Loading

memsharded left a comment

Choose a reason for hiding this comment

memsharded Dec 19, 2024

Choose a reason for hiding this comment

czoido Dec 20, 2024 • edited Loading

Choose a reason for hiding this comment

czoido commented Dec 20, 2024

czoido Dec 30, 2024

Choose a reason for hiding this comment

memsharded left a comment

Choose a reason for hiding this comment

memsharded commented Dec 19, 2024 •

edited by czoido

Loading

AbrilRBS left a comment •

edited

Loading

czoido commented Dec 19, 2024 •

edited

Loading

czoido Dec 20, 2024 •

edited

Loading