Read --{in,ex}clude values from stdin

Today, `dotnet-format` accepts space-separated list of files. This
allows us to pass files via shell's native globbing expansion,
heredoc style redirections as well as via pipelines (see dotnet#551 for
examples). However, in case of pipeline it requires a second utility
(`xarg`) to transform pipeline input as space-separated list to
`dotnet-format`.

This PR implements native pipeline reading support for `--include`
and `--exclude` options, while keeping the space-separated list
intact.

Usage examples:

1. Include all C# source files under `tests/Utilities` directory
that are in git source tree:

    ```sh git ls-files :/tests/Utilities/*.cs | dotnet format
    --include - --folder ```
2. Exclude certain `*.cs` and `*.vb` files using `ls(1)`:

    ```sh ls ../../../../../src/generated/{*.cs,*.vb} | dotnet
    format --exclude /dev/stdin --folder ```

Rules:

* Based on Guideline 13 of [IEEE
1003.1-2017](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html#tag_12_02)
(POSIX),  it accepts `-` as an explicit marker for stdin with
addition of:
  * `/dev/stdin` - which is a universal synonym of `-` on all Unices.
  * It *only* accepts explicit markers (`/dev/stdin` or `-`) and
  does not implicit deduce the output if standard input was redirected,
  but marker was not present. This is because our usage is multi-purpose
  (both `--include` and `--exclude`).
* It is an error if both `--include` and `--exclude` are using stdin
marker (`/dev/stdin` or `-`).

Limitation:

* Currently, it reads the entire input from pipeline in `include`/`exclude`
buffer, and then runs the operation on the whole batch. In order
to make the pipele , it would require some refactorings; so files
`yield return` and enumerator dispatches format operation per file.
* At present, we do not have out-of-process functional tests for
CLI to effectively validate these kind of use-cases (redirection
and shell globbing vs. dotnet-format's built-in globbing support);
so PR did not included any new mechanism.

README.md

@@ -86,15 +86,18 @@ Add `format` after `dotnet` and before the command arguments that you want to ru
 
 | Examples                                                   | Description                                                                                   |
 | ---------------------------------------------------------- |---------------------------------------------------------------------------------------------- |
-| dotnet **format**                                          | Formats the project or solution in the current directory.                                     |
-| dotnet **format** <workspace>                        | Formats a specific project or solution.                                                       |
-| dotnet **format** <workspace> -f                     | Formats a particular folder and subfolders.                                                   |
-| dotnet **format** <workspace> --fix-style warn       | Formats and fixes codestyle analyzer warnings.                                                |
-| dotnet **format** <workspace> --fix-analyzers        | Formats and fixes 3rd party analyzer errors.                                                  |
-| dotnet **format** -v diag                                  | Formats with very verbose logging.                                                            |
-| dotnet **format** --include Programs.cs Utility\Logging.cs | Formats the files Program.cs and Utility\Logging.cs                                           |
-| dotnet **format** --check                                  | Formats but does not save. Returns a non-zero exit code if any files would have been changed. |
-| dotnet **format** --report <report-path>             | Formats and saves a json report file to the given directory.                                  |
+| `dotnet format`                                          | Formats the project or solution in the current directory.                                     |
+| `dotnet format <workspace>`                        | Formats a specific project or solution.                                                       |
+| `dotnet format <workspace> -f`                     | Formats a particular folder and subfolders.                                                   |
+| `dotnet format <workspace> --fix-style warn`       | Formats and fixes codestyle analyzer warnings.                                                |
+| `dotnet format <workspace> --fix-analyzers`        | Formats and fixes 3rd party analyzer errors.                                                  |
+| `dotnet format -v diag`                                 | Formats with very verbose logging.                                                            |
+| `dotnet format --include Programs.cs Utility\Logging.cs` | Formats the files Program.cs and Utility\Logging.cs                                           |
+| `dotnet format --check`                                  | Formats but does not save. Returns a non-zero exit code if any files would have been changed. |
+| `dotnet format --report <report-path>`             | Formats and saves a json report file to the given directory. |
+| `dotnet format --include test/Utilities/*.cs --folder` | Formats the files expanded from native shell globbing (e.g. bash). Space-separated list of files are fed to formatter in this case. Also applies to `--exclude` option. |
+| `dotnet format --include 'test/Utilities/*.cs' --folder` | With single quotes, formats the files expanded from built-in glob expansion. A single file pattern is fed to formatter, which gets expanded internally. Also applies to `--exclude` option. |
+| `ls tests/Utilities/*.cs \| dotnet format --include - --folder` | Formats the list of files redirected from pipeline via standard input. Formatter will iterate over `Console.In` to read the list of files. Also applies to `--exclude` option. |
 
 ### How To Uninstall
 

src/Program.cs

@@ -7,6 +7,7 @@
 using System.Diagnostics.CodeAnalysis;
 using System.IO;
 using System.Linq;
+using System.Text;
 using System.Threading;
 using System.Threading.Tasks;
 using Microsoft.CodeAnalysis.Tools.Logging;
@@ -24,6 +25,8 @@ internal class Program
         internal const int UnableToLocateMSBuildExitCode = 3;
         internal const int UnableToLocateDotNetCliExitCode = 4;
 
+        private static readonly string[] s_standardInputKeywords = { "/dev/stdin", "-" };
+
         private static ParseResult? s_parseResult;
 
         private static async Task<int> Main(string[] args)
@@ -131,6 +134,8 @@ public static async Task<int> Run(
                     logger.LogTrace(Resources.Using_msbuildexe_located_in_0, msBuildPath);
                 }
 
+                HandleStandardInput(logger, ref include, ref exclude);
+
                 var fileMatcher = SourceFileMatcher.CreateMatcher(include, exclude);
 
                 var formatOptions = new FormatOptions(
@@ -173,6 +178,55 @@ public static async Task<int> Run(
             }
         }
 
+        private static int HandleStandardInput(ILogger logger, ref string[] include, ref string[] exclude)
+        {
+            bool isStandardMarkerUsed = false;
+            if (include.Length == 1 && s_standardInputKeywords.Contains(include[0]))
+            {
+                if (TryReadFromStandardInput(ref include))
+                {
+                    isStandardMarkerUsed = true;
+                }
+            }
+
+            if (exclude.Length == 1 && s_standardInputKeywords.Contains(exclude[0]))
+            {
+                if (isStandardMarkerUsed)
+                {
+                    logger.LogCritical(Resources.Standard_input_used_multiple_times);
+                    return CheckFailedExitCode;
+                }
+
+                TryReadFromStandardInput(ref exclude);
+            }
+
+            return 0;
+
+            static bool TryReadFromStandardInput(ref string[] subject)
+            {
+                if (!Console.IsInputRedirected)
+                {
+                    return false; // pass
+                }
+
+                // reset the subject array
+                Array.Clear(subject, 0, subject.Length);
+                Array.Resize(ref subject, 0);
+
+                Console.InputEncoding = Encoding.UTF8;
+                using var reader = new StreamReader(Console.OpenStandardInput(8192));
+                Console.SetIn(reader);
+
+                for (int i = 0; Console.In.Peek() != -1; ++i)
+                {
+                    Array.Resize(ref subject, subject.Length + 1);
+                    subject[i] = Console.In.ReadLine();
+                }
+
+                return true;
+            }
+        }
+
         internal static int GetExitCode(WorkspaceFormatResult formatResult, bool check)
         {
             if (!check)

src/Resources.resx

@@ -222,6 +222,9 @@
   <data name="Include_generated_code_files_in_formatting_operations" xml:space="preserve">
     <value>Include generated code files in formatting operations.</value>
   </data>
+  <data name="Standard_input_used_multiple_times" xml:space="preserve">
+    <value>Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</value>
+  </data>
   <data name="Unable_to_locate_dotnet_CLI_Ensure_that_it_is_on_the_PATH" xml:space="preserve">
     <value>Unable to locate dotnet CLI. Ensure that it is on the PATH.</value>
   </data>

src/xlf/Resources.cs.xlf

@@ -202,6 +202,11 @@
         <target state="translated">Řešení {0} nemá žádné projekty.</target>
         <note />
       </trans-unit>
+      <trans-unit id="Standard_input_used_multiple_times">
+        <source>Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</source>
+        <target state="new">Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</target>
+        <note />
+      </trans-unit>
       <trans-unit id="The_dotnet_CLI_version_is_0">
         <source>The dotnet CLI version is '{0}'.</source>
         <target state="translated">Verze .NET CLI je {0}.</target>

src/xlf/Resources.de.xlf

@@ -202,6 +202,11 @@
         <target state="translated">Die Projektmappe "{0}" enthält keine Projekte.</target>
         <note />
       </trans-unit>
+      <trans-unit id="Standard_input_used_multiple_times">
+        <source>Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</source>
+        <target state="new">Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</target>
+        <note />
+      </trans-unit>
       <trans-unit id="The_dotnet_CLI_version_is_0">
         <source>The dotnet CLI version is '{0}'.</source>
         <target state="translated">Die dotnet-CLI-Version ist "{0}".</target>

src/xlf/Resources.es.xlf

@@ -202,6 +202,11 @@
         <target state="translated">La solución {0} no tiene ningún proyecto.</target>
         <note />
       </trans-unit>
+      <trans-unit id="Standard_input_used_multiple_times">
+        <source>Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</source>
+        <target state="new">Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</target>
+        <note />
+      </trans-unit>
       <trans-unit id="The_dotnet_CLI_version_is_0">
         <source>The dotnet CLI version is '{0}'.</source>
         <target state="translated">La versión de la CLI de dotnet es "{0}".</target>

src/xlf/Resources.fr.xlf

@@ -202,6 +202,11 @@
         <target state="translated">La solution {0} n'a aucun projet</target>
         <note />
       </trans-unit>
+      <trans-unit id="Standard_input_used_multiple_times">
+        <source>Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</source>
+        <target state="new">Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</target>
+        <note />
+      </trans-unit>
       <trans-unit id="The_dotnet_CLI_version_is_0">
         <source>The dotnet CLI version is '{0}'.</source>
         <target state="translated">La version de l'interface CLI dotnet est '{0}'.</target>

src/xlf/Resources.it.xlf

@@ -202,6 +202,11 @@
         <target state="translated">La soluzione {0} non contiene progetti</target>
         <note />
       </trans-unit>
+      <trans-unit id="Standard_input_used_multiple_times">
+        <source>Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</source>
+        <target state="new">Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</target>
+        <note />
+      </trans-unit>
       <trans-unit id="The_dotnet_CLI_version_is_0">
         <source>The dotnet CLI version is '{0}'.</source>
         <target state="translated">La versione dell'interfaccia della riga di comando di dotnet è '{0}'.</target>

src/xlf/Resources.ja.xlf

@@ -202,6 +202,11 @@
         <target state="translated">ソリューション {0} にプロジェクトがありません</target>
         <note />
       </trans-unit>
+      <trans-unit id="Standard_input_used_multiple_times">
+        <source>Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</source>
+        <target state="new">Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</target>
+        <note />
+      </trans-unit>
       <trans-unit id="The_dotnet_CLI_version_is_0">
         <source>The dotnet CLI version is '{0}'.</source>
         <target state="translated">dotnet CLI バージョンは '{0}' です。</target>

src/xlf/Resources.ko.xlf

@@ -202,6 +202,11 @@
         <target state="translated">{0} 솔루션에 프로젝트가 없습니다.</target>
         <note />
       </trans-unit>
+      <trans-unit id="Standard_input_used_multiple_times">
+        <source>Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</source>
+        <target state="new">Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</target>
+        <note />
+      </trans-unit>
       <trans-unit id="The_dotnet_CLI_version_is_0">
         <source>The dotnet CLI version is '{0}'.</source>
         <target state="translated">dotnet CLI 버전은 '{0}'입니다.</target>

src/xlf/Resources.pl.xlf

@@ -202,6 +202,11 @@
         <target state="translated">Rozwiązanie {0} nie zawiera projektów</target>
         <note />
       </trans-unit>
+      <trans-unit id="Standard_input_used_multiple_times">
+        <source>Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</source>
+        <target state="new">Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</target>
+        <note />
+      </trans-unit>
       <trans-unit id="The_dotnet_CLI_version_is_0">
         <source>The dotnet CLI version is '{0}'.</source>
         <target state="translated">Wersja interfejsu wiersza polecenia dotnet to „{0}”.</target>

src/xlf/Resources.pt-BR.xlf

@@ -202,6 +202,11 @@
         <target state="translated">A solução {0} não tem nenhum projeto</target>
         <note />
       </trans-unit>
+      <trans-unit id="Standard_input_used_multiple_times">
+        <source>Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</source>
+        <target state="new">Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</target>
+        <note />
+      </trans-unit>
       <trans-unit id="The_dotnet_CLI_version_is_0">
         <source>The dotnet CLI version is '{0}'.</source>
         <target state="translated">A versão do CLI do dotnet é '{0}'.</target>

src/xlf/Resources.ru.xlf

@@ -202,6 +202,11 @@
         <target state="translated">Решение {0} не содержит проектов.</target>
         <note />
       </trans-unit>
+      <trans-unit id="Standard_input_used_multiple_times">
+        <source>Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</source>
+        <target state="new">Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</target>
+        <note />
+      </trans-unit>
       <trans-unit id="The_dotnet_CLI_version_is_0">
         <source>The dotnet CLI version is '{0}'.</source>
         <target state="translated">Версия CLI dotnet: "{0}".</target>

src/xlf/Resources.tr.xlf

@@ -202,6 +202,11 @@
         <target state="translated">{0} çözümünde proje yok</target>
         <note />
       </trans-unit>
+      <trans-unit id="Standard_input_used_multiple_times">
+        <source>Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</source>
+        <target state="new">Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</target>
+        <note />
+      </trans-unit>
       <trans-unit id="The_dotnet_CLI_version_is_0">
         <source>The dotnet CLI version is '{0}'.</source>
         <target state="translated">dotnet CLI sürümü: '{0}'.</target>

src/xlf/Resources.zh-Hans.xlf

@@ -202,6 +202,11 @@
         <target state="translated">解决方案 {0} 不包含任何项目</target>
         <note />
       </trans-unit>
+      <trans-unit id="Standard_input_used_multiple_times">
+        <source>Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</source>
+        <target state="new">Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</target>
+        <note />
+      </trans-unit>
       <trans-unit id="The_dotnet_CLI_version_is_0">
         <source>The dotnet CLI version is '{0}'.</source>
         <target state="translated">dotnet CLI 版本为“{0}”。</target>

src/xlf/Resources.zh-Hant.xlf

@@ -202,6 +202,11 @@
         <target state="translated">解決方案 {0} 沒有任何專案</target>
         <note />
       </trans-unit>
+      <trans-unit id="Standard_input_used_multiple_times">
+        <source>Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</source>
+        <target state="new">Standard input markers ('/dev/stdin', '-') can only be used either with `--include` or `--exclude`, but not both.</target>
+        <note />
+      </trans-unit>
       <trans-unit id="The_dotnet_CLI_version_is_0">
         <source>The dotnet CLI version is '{0}'.</source>
         <target state="translated">dotnet CLI 版本為 '{0}'。</target>