Introduce GAP-Level StandardInput, StandardOutput, StandardError Objects #1815

markuspf · 2017-10-26T14:23:35Z

This introduces the InputStream StandardInput and the OutputStreams
StandardOutput and StandardError at the GAP level, and attaches

InputTextFile("*stdin*") to StandardInput,
OutputTextFile("*stdout*", true) to StandardOutput
OutputTextFile("*errout*", true) to StandardError

This small change already enables refactoring error.g to PrintTo
the StandardError stream, which can be replaced by an
OutputTextString (or a different output stream) to capture error messages.

One obvious use case of this is to capture output for alternative
REPL implementations such as Jupyter.

This introduces the InputStream StandardInput and the OutputStreams StandardOutput and StandardError at the GAP level, and attaches * InputTextFile("*stdin*") to StandardInput, * OutputTextFile("*stdout*", true) to StandardOutput * OutputTextFile("*errout*", true) to StandardError This small change already enables refactoring error.g to PrintTo the StandardError stream, and it can be replaced by an OutputTextString to capture error messages. One obvious use case of this is to capture output for alternative REPL implementations such as Jupyter.

markuspf · 2017-10-26T14:25:20Z

src/sysfiles.c

+Obj StandardInput = 0;
+Obj StandardOutput = 0;
+Obj StandardError = 0;
+


These are currently unused, I seem to remeber imagining having to access these streams from the kernel level.

So, do you mean you want to keep them "in case", or that you will update this PR to drop them?!?

If you intend to keep them, please add at least a tiny comment.

The best outcome would be to not need them at the C level at all, so maybe I remove them right now, and only add them if there's a really good reason to have them.

fingolfin

So, I have no actual concerns about this PR, but am a bit mystified by the motivation for it.

You say that it enables refactoring Error to send its output to a stream, by redeeming StandardError, which then is useful for Jupyter. But I would have thought that things like Jupyter have to catch GAP's stdout and stderr anyway? Am I then wrong in that believe? And if so, does that mean it is OK for you to capture only output sent to StandardError, but not any remaining output sent to *errout*?

On a different point, note that hpcgap/lib/error.g exists; this PR increases the diffs between the two copies of error.g again.

I also wonder if these new objects work corrctly in HPC-GAP.

lib/streams.gi

fingolfin · 2017-10-27T22:18:45Z

lib/streams.gi

@@ -1797,6 +1797,10 @@ InstallGlobalFunction( "UnInstallCharReadHookFunc",
 Unbind(OnCharReadHookActive);


+InstallValue(StandardInput, InputTextFile("*stdin*"));
+InstallValue(StandardOutput, OutputTextFile("*stdout*", true));


Note that colorprompt.g also declares STDOut as BindGlobal("STDOut", OutputTextUser());, which also binds it to stdout; so perhaps change that to use StandardOutput, too?

Also, Exec in process.gi uses stdin and stdout.

fingolfin · 2017-10-27T22:19:47Z

src/sysfiles.c

+Obj StandardInput = 0;
+Obj StandardOutput = 0;
+Obj StandardError = 0;
+


So, do you mean you want to keep them "in case", or that you will update this PR to drop them?!?

If you intend to keep them, please add at least a tiny comment.

markuspf · 2017-10-30T12:54:13Z

My main motivation is to have a well-defined GAP level interface for output and input that can just be re-bound to well-defined to GAP level OutputStreams and InputStreams, and in the end to get rid of the file-and-stream-juggling exercises in scanner.c (which will, hopefully, also enable clean implementation of a replacement to READ_ALL_COMMANDS in the end!).

At the moment its fine with me for Jupyter if someone uses PrintTo or AppendTo on *errout* or *stdout*, it just means that this output will not be visible in a Jupyter front-end (hence I would consider it a bug-to-be-fixed, but there are still so many things "wrong" with Jupyter output that this is merely noise).

On the matter of HPC-GAP: while this change will not immediately work with it, the multi-threaded console-ui does something very similar to the approach used here: it defines a DEFAULT_INPUT_STREAM and a DEFAULT_OUTPUT_STREAM for each thread.

Once we managed to make input and output go through a well-defined GAP level object, the console ui of HPC-GAP can also be simplified.

fingolfin · 2017-11-01T14:38:19Z

What I don't understand is why we need to define new global objects for this, and then change all code to print to them; note that nothing would stop new code from again printing to e.g. *errout*.

Wouldn't it be less invasive to do the reverse, and change the code which accepts *errout* etc. as input to use that as a special hint to print to respective streams?

markuspf · 2017-11-06T16:16:44Z

We can add functionality to re-route stuff that uses the "magic strings" through these streams.

Since I would quite like to clean things up a bit (and that would include chasing up the definition of STDOut in colorprompt.g, so clearly there has been a desire to have the output stream around in the past, and adding documentation!), adding an indirection is of course complicating the input/output path.

Another usecase I imagine for this PR is getting rid of all the specific LogTo streams, since we can implement all the logging through LogStreams that are implemented on the GAP level.

frankluebeck · 2018-06-07T15:18:16Z

I'm wondering if some abstraction layer would be useful here. So, e.g., StandardOutput is not directly connected to the stdout stream. Instead there is some data structure that contains the information about one or several streams where this output should be sent to (and the set of these output streams can be changed at runtime, e.g., adding or removing a log file at runtime). So, printing to the StandardOutput is done to distinguish this output from output for StandardError (and maybe we want to distinguish further cases).

Uses the new command line option --alwaystrace. This lets GAP also print tracebacks when the break loop is disabled by -T. This still overwrites the library functions Where, WHERE, and ErrorInner. This can be avoided once there is a central GAP error stream where these functions send their output, see gap-system/gap#1815. Fixes gap-packages#38.

Uses the new command line option --alwaystrace. This lets GAP also print tracebacks when the break loop is disabled by -T. This still overwrites the library functions Where, WHERE, and ErrorInner. This can be avoided once there is a central GAP error stream where these functions send their output, see gap-system/gap#1815. For versions older than 4.10 a backwards compatible version of gap/JupyterError.gi is loaded. Fixes gap-packages#38.

Uses the new command line option --alwaystrace. This lets GAP also print tracebacks when the break loop is disabled by -T. This still overwrites the library functions Where, WHERE, and ErrorInner. This can be avoided once there is a central GAP error stream where these functions send their output, see gap-system/gap#1815. Since redirecting error messages was not available in 4.9 this also sets the required GAP version to >= 4.10. Fixes gap-packages#38.

Uses the new command line option --alwaystrace. This lets GAP also print tracebacks when the break loop is disabled by -T. This still overwrites the library functions Where, WHERE, and ErrorInner. This can be avoided once there is a central GAP error stream where these functions send their output, see gap-system/gap#1815. Since redirecting error messages was not available in 4.9 this also sets the required GAP version to >= 4.10. Fixes #38.

@fingolfin

This commit introduces new global variables StandardInput, StandardOutput, ErrorOutput. ADD_SET is used since the new global variables are initialized relatively early in the start-up process, AddSet is not yet available at this point. The lib/colorprompt.g was changed to use StandardOutput instead of defining its own STDOut. As a next step one can make it possible to change the targets of the magic strings "*stdout*", "*stdin*", and "*errout*", as suggested by @fingolfin in gap-system#1815. See gap-system#2822.

@fingolfin

This commit introduces new global variables StandardInput, StandardOutput, ErrorOutput. ADD_SET is used since the new global variables are initialized relatively early in the start-up process, AddSet is not yet available at this point. As a next step one can make it possible to change the targets of the magic strings "*stdout*", "*stdin*", and "*errout*", as suggested by @fingolfin in gap-system#1815. See gap-system#2822.

@fingolfin

This commit introduces a new global variable ErrorOutput. ADD_SET is used since the new global variables are initialized relatively early in the start-up process, AddSet is not yet available at this point. As a next step one can make it possible to change the targets of the magic strings "*stdout*", "*stdin*", and "*errout*", as suggested by @fingolfin in gap-system#1815. See gap-system#2822.

@fingolfin

This commit introduces a new global variable ErrorOutput. ADD_SET is used since the new global variables are initialized relatively early in the start-up process, AddSet is not yet available at this point. As a next step one can make it possible to change the targets of the magic strings "*stdout*", "*stdin*", and "*errout*", as suggested by @fingolfin in gap-system#1815. See gap-system#2822.

@fingolfin