Adjustable outputDirectoryName

[excitoon]

Description of the problem / feature request:

Hi. I wonder if one can make outputDirectoryName (these {cpu}-{compilation_mode} parts of paths) for the build adjustable from command line. It would help to integrate Bazel into other pipelines because it is necessary to know where output files are. I can run bazel info/bazel cquery, but that will involve extracting, loading and/or configuring, I guess, and therefore it looks like extra overhead.

Have you found anything relevant by searching the web?

https://stackoverflow.com/questions/51292429/bazel-how-to-get-path-to-output-binary
https://stackoverflow.com/questions/47859615/bazel-how-do-you-get-the-path-to-a-generated-file

[gregestren]

You can use --platform_suffix to add a customizable string to the end of that. But you can't replace {cpu}-{compilation_mode} with, say, a string literal because Bazel has to support building the same output for different CPUs in the same build, and each needs a distinct path.

Depending on exactly which outputs you need, can you just look at blaze-bin?

Also see #6526 re: plans to change this naming completely. But that's a longer term effort.

[excitoon]

Thank you for pointing to the issue, @gregestren . Will that cover scenario when we gather artifacts from different platforms and place it to some kind of universal archive (in almost cpu-agnostic way, except its' dependencies)?

[gregestren]

Essentially yes.

The basic problem is Bazel's logic predates concern for multi-platform builds, so it just doesn't address that problem very well. The issue I mentioned, as well as other work, recognizes we need better interfaces for these sorts of problems. So that'll happen - but they''ll need to be properly designed.

Out of curiosity, what kind of multi-platform builds are you interested in? We're doing a lot of work to improve multi-platform support and some cool new features are coming down the pipeline even now.

[excitoon]

@gregestren Thank you for asking. In most easiest case we will need to build some C++ target for x64 and x86 and use both versions of it say in C++ program for x64. Would that be possible?

Actually this is a typical scenario in Windows world because often your program has to communicate with both Windows subsystems, Win32 and Win64.

[gregestren]

Essentially yes. You may also want to review https://bazel.build/roadmaps/configuration.html, which explores new ways to support multiple CPUs in the same binaries.

[excitoon]

I leave this hack here as a quick workaround:

--platform_suffix=/..

[moroten]

Just noticed that //command_line_option:output directory name can be used in transitions and platform mappings, but is hidden from the command line.

bazel/src/main/java/com/google/devtools/build/lib/analysis/config/CoreOptions.java

Lines 239 to 254 in 99f8a8f

	/**
	* This option is used internally to set output directory name of the <i>host</i> configuration to
	* a constant, so that the output files for the host are completely independent of those for the
	* target, no matter what options are in force (k8/piii, opt/dbg, etc).
	*/
	@Option(
	name = "output directory name",
	defaultValue = "null",
	documentationCategory = OptionDocumentationCategory.UNDOCUMENTED,
	effectTags = {
	OptionEffectTag.LOSES_INCREMENTAL_STATE,
	OptionEffectTag.AFFECTS_OUTPUTS,
	OptionEffectTag.LOADING_AND_ANALYSIS
	},
	metadataTags = {OptionMetadataTag.INTERNAL})
	public String outputDirectoryName;

[aaron-michaux]

My system supports a combinatorial set of configurations:

[gcc9, gcc10, gcc11, gcc12, clang12, clang13, ...] x [lto, no-lto] x [opt, dbg, asan, ubsan, tsan] + [clang-tidy, coverage, perf].

If bazel had a hook for a function to programatically generate the output directory, that would be a snap. As it is, switching configurations often means rebuilding.

Platform suffix is cute; however, we're talking 50+ platforms? It's madness!

Working with bazel often seems like you're a chef shopping for dinner, and the only thing available is ready cooked meals. You can sell 100 different types of frozen pizza, or 12 ingredients to create any of them.

[moroten]

@aaron-michaux Just wonder, how would you ideally like the directories organized?

[excitoon]

And alive animals are available as well :)

…

On Sat, Nov 5, 2022, 8:03 PM Aaron Michaux ***@***.***> wrote: My system supports a combinatorial set of configurations: [gcc9, gcc10, gcc11, gcc12, clang12, clang13, ...] x [lto, no-lto] x [opt, dbg, asan, ubsan, tsan] + [clang-tidy, coverage, perf]. If bazel had a hook for a function to programatically generate the output directory, that would be a snap. As it is, switching configurations often means rebuilding. Platform suffix is cute; however, we're talking 50+ platforms? It's madness! Working with bazel often seems like you're a chef shopping for dinner, and the only thing available is ready cooked meals. You can sell 100 different types of frozen pizza, or 12 ingredients to create any of them. — Reply to this email directly, view it on GitHub <#7477 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AARGSB6CFVLXVT6RUS7JMK3WG2HONANCNFSM4GYUMCCA> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[aaron-michaux]

Hi @moroten ,

Bazel, at some stage, has to determine the output directory, using things like "platform suffix" etc. It would be nice to be able to supply a function that has the necessary context (all the config_settings) so that you can just programatically return the directory that you want to use (build with the ingredients), obviating the need for pre-baked features like "platform suffix" (providing ready-made meals).