cli: Add infrastructure for new CLI drivers juliax and juliac

Following the discussion in #50974, it became clear that there is significant
objection to changes to the behavior of the julia CLI driver. Some
commenters found changes to the behavior acceptable if they were unlikely
to impact existing code (e.g. in the case of #50974 using `__main__`
instead of `main`), while other were worried about the reputational
risks of even changing behavior in the corner case. In subsequent
discussion it became relatively clear that the only way forward
that did not raise significant objection was to introduce a new CLI
driver for the new behavior. This may seem a bit drastic just for the
change in #50974, but I actually think there's a number of other
quality-of-life improvements that this would enable us to do over time,
for example:
 - Autoloading/caching of all packages in the manifest
 - auto-selection of julia versions (integrate juliaup?)

In addition, it doesn't seem so bad to have some CLI design flexibility
to make sure that the `juliac` driver is aligned with what we need.

This PR is the minimal infrastructure to add the new drivers.
In particular, it does the following:

1. Adds two new cli drivers, `juliax` and `juliac`. At the moment,
   `juliac` is a placeholder and just errors, while `juliax` behaves
   the same as `julia` (except to error on the deprecated `--math-mode=fast`,
   just as an example of a behavior difference).

2. Documents that the behavior of `julia` (but not `juliax` or `juliac`)
   is pat of the julia public API.

3. Switches the cli mode based on the argv[0] of the binary. I.e. all three
   binaries are identical, except for their name, the same way that, e.g.
   `clang` and `clang++` are the same binary just with different names.
   On Unix systems, these are symlinks. On windows, separate copies of
   the same (small) binary. There is a fallback cli option `--cli-mode`
   that can be used in case the argv[0] detection is not available (e.g.
   for some fringe embedded use cases).

4. There is currently no separate `-debug` version of the new drivres.
   My intention is to make this dependent on the ordinary debug flags,
   rather than having a separate driver.

Once this is merged, I intend to resubmit #50974 (chaning `juliax` only),
and then finish and hook up `juliac` shortly thereafter.

base/options.jl

@@ -2,6 +2,7 @@
 
 # NOTE: This type needs to be kept in sync with jl_options in src/jloptions.h
 struct JLOptions
+    cli_mode::Int8
     quiet::Int8
     banner::Int8
     julia_bindir::Ptr{UInt8}

cli/Makefile

@@ -99,7 +99,7 @@ $(build_bindir)/julia$(EXE): $(BUILDDIR)/Info.plist
 $(build_bindir)/julia-debug$(EXE): $(BUILDDIR)/Info.plist
 endif
 
-julia-release: $(build_bindir)/julia$(EXE)
+julia-release: $(build_bindir)/julia$(EXE) $(build_bindir)/juliac$(EXE) $(build_bindir)/juliax$(EXE)
 julia-debug: $(build_bindir)/julia-debug$(EXE)
 libjulia-release: $(build_shlibdir)/libjulia.$(SHLIB_EXT)
 libjulia-debug: $(build_shlibdir)/libjulia-debug.$(SHLIB_EXT)
@@ -148,6 +148,14 @@ $(build_bindir)/julia$(EXE): $(EXE_OBJS) $(build_shlibdir)/libjulia.$(SHLIB_EXT)
 $(build_bindir)/julia-debug$(EXE): $(EXE_DOBJS) $(build_shlibdir)/libjulia-debug.$(SHLIB_EXT) | $(build_bindir)
 	@$(call PRINT_LINK, $(CC) $(LOADER_CFLAGS) $(DEBUGFLAGS) $(EXE_DOBJS) -o $@ $(LOADER_LDFLAGS) $(RPATH) -ljulia-debug)
 
+ifneq ($(OS), WINNT)
+$(build_bindir)/juliac$(EXE) $(build_bindir)/juliax$(EXE): $(build_bindir)/julia$(EXE)
+	@$(call PRINT_LINK, ln -sf $(notdir $<) $@)
+else
+$(build_bindir)/juliac$(EXE) $(build_bindir)/juliax$(EXE): $(EXE_OBJS) $(build_shlibdir)/libjulia.$(SHLIB_EXT) | $(build_bindir)
+	@$(call PRINT_LINK, $(CC) $(LOADER_CFLAGS) $(SHIPFLAGS) $(EXE_OBJS) -o $@ $(LOADER_LDFLAGS) $(RPATH) -ljulia)
+endif
+
 $(BUILDDIR)/julia.expmap: $(SRCDIR)/julia.expmap.in
 	sed <'$<' >'$@' -e 's/@JULIA_SHLIB_SYMBOL_VERSION@/JL_LIBJULIA_$(SOMAJOR)/'
 

doc/man/juliac.1

@@ -0,0 +1,47 @@
+.\" To get a preview of the man page as it will actually be displayed, run
+.\"
+.\" > nroff -man juliac.1 | less
+.\"
+.\" at the terminal.
+.\"
+
+.TH JULIAC 1 2023-09-01 JULIA
+
+.\" from the front page of https://julialang.org/
+.SH NAME
+juliac - Compiler driver for the julia programming language
+
+.SH SYNOPSIS
+\fBjulia\fR [OPTIONS...] \fB--\fR PROGRAMMFILE
+
+The Julia source file \fIPROGRAMFILE\fP (optionally followed by
+arguments in \fIARGS\fP) will be compiled and a binary will be
+written out according to OPTIONS.
+
+.SH DESCRIPTION
+This is a placeholder for the upcoming compiler-driver for the julia
+programming language. It is currently non functional. See
+.BR julia (1)
+for the traditional cli driver.
+
+.SH "COMMAND-LINE OPTIONS"
+
+.SH FILES AND ENVIRONMENT
+See https://docs.julialang.org/en/v1/manual/environment-variables/
+
+.SH BUGS
+Please report any bugs using the GitHub issue tracker:
+https://github.com/julialang/julia/issues?state=open
+
+.SH AUTHORS
+Contributors: https://github.com/JuliaLang/julia/graphs/contributors
+
+.SH INTERNET RESOURCES
+Website:  https://julialang.org/
+.br
+Documentation:  https://docs.julialang.org/
+.br
+Downloads:  https://julialang.org/downloads/
+
+.SH LICENSING
+Julia is an open-source project. It is made available under the MIT license.

doc/man/juliax.1

@@ -0,0 +1,59 @@
+.\" To get a preview of the man page as it will actually be displayed, run
+.\"
+.\" > nroff -man juliax.1 | less
+.\"
+.\" at the terminal.
+.\"
+
+.TH JULIAC 1 2023-09-01 JULIA
+
+.\" from the front page of https://julialang.org/
+.SH NAME
+juliax - Experimental driver for the julia programming language
+
+.SH SYNOPSIS
+\fBjulia\fR [OPTIONS...] \fB--\fR PROGRAMMFILE
+
+If a Julia source file is given as a \fIPROGRAMFILE\fP (optionally followed by
+arguments in \fIARGS\fP) Julia will execute the program and exit.
+
+.SH DESCRIPTION
+This is the unstable, experimental driver for the Julia programming language.
+This interface is currently not stable across Julia versions. The behavior of
+this driver matches that of
+.BR julia (1)
+except for the differences noted below.
+
+.SH "COMMAND-LINE OPTIONS"
+
+.TP
+--math-mode=fast
+In juliax, this option is an error. In
+.BR julia (1)
+this options is silently ignored.
+
+.TP
+--cli-mode
+In juliax, this option is an error. In
+.BR julia (1)
+may be used to switch into juliax mode.
+
+.SH FILES AND ENVIRONMENT
+See https://docs.julialang.org/en/v1/manual/environment-variables/
+
+.SH BUGS
+Please report any bugs using the GitHub issue tracker:
+https://github.com/julialang/julia/issues?state=open
+
+.SH AUTHORS
+Contributors: https://github.com/JuliaLang/julia/graphs/contributors
+
+.SH INTERNET RESOURCES
+Website:  https://julialang.org/
+.br
+Documentation:  https://docs.julialang.org/
+.br
+Downloads:  https://julialang.org/downloads/
+
+.SH LICENSING
+Julia is an open-source project. It is made available under the MIT license.

doc/src/manual/faq.md

@@ -42,6 +42,9 @@ You can get a complete list of the public symbols from a module with `names(MyMo
 
 Package authors are encouraged to define their public API similarly.
 
+In addition, the documented and semantically observable behaviors of the `julia` CLI driver
+(but not the `juliax` and `juliac` drivers) are considered part of the public API.
+
 Anything in Julia's Public API is covered by [SemVer](https://semver.org/) and therefore
 will not be removed or receive meaningful breaking changes before Julia 2.0.
 

src/jlapi.c

@@ -26,6 +26,9 @@ extern "C" {
 #include <fenv.h>
 #endif
 
+const char juliax_name[] = "juliax";
+const char juliac_name[] = "juliac";
+
 JL_DLLEXPORT int jl_is_initialized(void)
 {
     return jl_main_module != NULL;
@@ -687,6 +690,18 @@ static void rr_detach_teleport(void) {
 }
 #endif
 
+// We match any basename that is equal to plain `name`, or `name`,
+// followed by a dot and arbitrary suffix (to allow extensions or
+// version numbers).
+static int matches_basename(const char *basename, const char *name, size_t len)
+{
+    if (strncmp(basename, name, len) != 0)
+        return 0;
+    if (!basename[len])
+        return 1;
+    return basename[len] == '.';
+}
+
 JL_DLLEXPORT int jl_repl_entrypoint(int argc, char *argv[])
 {
 #ifdef USE_TRACY
@@ -707,9 +722,24 @@ JL_DLLEXPORT int jl_repl_entrypoint(int argc, char *argv[])
         memmove(&argv[1], &argv[2], (argc-2)*sizeof(void*));
         argc--;
     }
+
+    if (argc > 0) {
+        char *exe_basename = basename(argv[0]);
+        if (matches_basename(exe_basename, juliax_name, sizeof(juliax_name) - 1)) {
+            jl_options.cli_mode = JL_OPTIONS_CLI_MODE_EXPERIMENTAL;
+        }
+        else if (matches_basename(exe_basename, juliac_name, sizeof(juliac_name) - 1)) {
+            jl_options.cli_mode = JL_OPTIONS_CLI_MODE_COMPILER;
+        }
+    }
+
     char **new_argv = argv;
     jl_parse_opts(&argc, (char***)&new_argv);
 
+    if (jl_options.cli_mode == JL_OPTIONS_CLI_MODE_COMPILER) {
+        jl_error("juliac is currently a placeholder. Check back later!");
+    }
+
     // The parent process requested that we detach from the rr session.
     // N.B.: In a perfect world, we would only do this for the portion of
     // the execution where we actually need to exclude rr (e.g. because we're

src/jloptions.c

@@ -31,7 +31,8 @@ JL_DLLEXPORT void jl_init_options(void)
     if (jl_options_initialized)
         return;
     jl_options =
-        (jl_options_t){ 0,    // quiet
+        (jl_options_t){ JL_OPTIONS_CLI_MODE_TRADITIONAL, // cli_mode
+                        0,    // quiet
                         -1,   // banner
                         NULL, // julia_bindir
                         NULL, // julia_bin
@@ -194,6 +195,8 @@ static const char opts[]  =
 
 static const char opts_hidden[]  =
     "Switches (a '*' marks the default value, if applicable):\n\n"
+    " --cli-mode=julia{|x|c}   Switch CLI driver to experimental or compiler mode.\n"
+
     // code generation options
     " --compile={yes*|no|all|min}\n"
     "                          Enable or disable JIT compiler, or request exhaustive or minimal compilation\n\n"
@@ -259,7 +262,8 @@ JL_DLLEXPORT void jl_parse_opts(int *argcp, char ***argvp)
            opt_strip_ir,
            opt_heap_size_hint,
            opt_gc_threads,
-           opt_permalloc_pkgimg
+           opt_permalloc_pkgimg,
+           opt_cli_mode,
     };
     static const char* const shortopts = "+vhqH:e:E:L:J:C:it:p:O:g:";
     static const struct option longopts[] = {
@@ -321,6 +325,7 @@ JL_DLLEXPORT void jl_parse_opts(int *argcp, char ***argvp)
         { "strip-ir",        no_argument,       0, opt_strip_ir },
         { "permalloc-pkgimg",required_argument, 0, opt_permalloc_pkgimg },
         { "heap-size-hint",  required_argument, 0, opt_heap_size_hint },
+        { "cli-mode",        required_argument, 0, opt_cli_mode },
         { 0, 0, 0, 0 }
     };
 
@@ -333,16 +338,19 @@ JL_DLLEXPORT void jl_parse_opts(int *argcp, char ***argvp)
     const char **cmds = NULL;
     int codecov = JL_LOG_NONE;
     int malloclog = JL_LOG_NONE;
+    int nprocessed = 0;
     int pkgimage_explicit = 0;
     int argc = *argcp;
     char **argv = *argvp;
     char *endptr;
     opterr = 0; // suppress getopt warning messages
+
     while (1) {
         int lastind = optind;
         int c = getopt_long(argc, argv, shortopts, longopts, 0);
         if (c == -1) break;
 restart_switch:
+        nprocessed += 1;
         switch (c) {
         case 0:
             break;
@@ -757,8 +765,11 @@ JL_DLLEXPORT void jl_parse_opts(int *argcp, char ***argvp)
         case opt_math_mode:
             if (!strcmp(optarg,"ieee"))
                 jl_options.fast_math = JL_OPTIONS_FAST_MATH_OFF;
-            else if (!strcmp(optarg,"fast"))
+            else if (!strcmp(optarg,"fast")) {
+                if (jl_options.cli_mode != JL_OPTIONS_CLI_MODE_TRADITIONAL)
+                    jl_errorf("juliax: --math-mode=fast is deprecated. It is non-functional in `julia` mode and disabled in `juliax`.");
                 jl_options.fast_math = JL_OPTIONS_FAST_MATH_DEFAULT;
+            }
             else if (!strcmp(optarg,"user"))
                 jl_options.fast_math = JL_OPTIONS_FAST_MATH_DEFAULT;
             else
@@ -855,6 +866,20 @@ JL_DLLEXPORT void jl_parse_opts(int *argcp, char ***argvp)
             else
                 jl_errorf("julia: invalid argument to --permalloc-pkgimg={yes|no} (%s)", optarg);
             break;
+        case opt_cli_mode:
+            if (nprocessed != 1)
+                jl_errorf("julia: --cli-mode must be the first argument");
+            if (jl_options.cli_mode != JL_OPTIONS_CLI_MODE_TRADITIONAL)
+                jl_errorf("julia: CLI mode switch is only available in `julia` driver");
+            if (!strcmp(optarg,"julia"))
+                jl_options.cli_mode = JL_OPTIONS_CLI_MODE_TRADITIONAL;
+            else if (!strcmp(optarg,"juliax"))
+                jl_options.cli_mode = JL_OPTIONS_CLI_MODE_EXPERIMENTAL;
+            else if (!strcmp(optarg,"juliac"))
+                jl_options.cli_mode = JL_OPTIONS_CLI_MODE_COMPILER;
+            else
+                jl_errorf("julia: invalid argument to --cli-mode=julia{|x|c} (%s)", optarg);
+            break;
         default:
             jl_errorf("julia: unhandled option -- %c\n"
                       "This is a bug, please report it.", c);

src/jloptions.h

@@ -6,6 +6,7 @@
 // NOTE: This struct needs to be kept in sync with JLOptions type in base/options.jl
 
 typedef struct {
+    int8_t cli_mode;
     int8_t quiet;
     int8_t banner;
     const char *julia_bindir;

src/julia.h

@@ -2293,6 +2293,10 @@ JL_DLLEXPORT int jl_generating_output(void) JL_NOTSAFEPOINT;
 #define JL_OPTIONS_STARTUPFILE_ON 1
 #define JL_OPTIONS_STARTUPFILE_OFF 2
 
+#define JL_OPTIONS_CLI_MODE_TRADITIONAL 0
+#define JL_OPTIONS_CLI_MODE_EXPERIMENTAL 1
+#define JL_OPTIONS_CLI_MODE_COMPILER 2
+
 #define JL_LOGLEVEL_BELOWMIN -1000001
 #define JL_LOGLEVEL_DEBUG    -1000
 #define JL_LOGLEVEL_INFO      0