gmake test: Windows 11, gcc 12: Use of uninitialized value $_ in substitution (s///) at ..\pod\buildtoc line 215, <$fh> chunk 2

[hakonhagland]

I am trying to build blead on Windows 11 using MinGW-w64 and gcc 12.2 from https://winlibs.com/ (using the MSVCRT runtime library) following a similar approach as in #20096. See also #20383.

After downloading WinLibs to C:\Winlibs64-Gcc12-msvcrt, I needed to rename gnu make binary to gmake.exe:

> copy C:\Winlibs64-Gcc12-msvcrt\mingw64\bin\mingw32-make.exe C:\Winlibs64-Gcc12-msvcrt\mingw64\bin\gmake.exe

This is gcc version 12.2.0, and gnu make version 4.3:

c:\PerlWinLib> git clone https://github.com/Perl/perl5.git
[...]

c:\PerlWinLib>cd perl5\win32

c:\PerlWinLib\perl5\win32>where gcc
C:\Winlibs64-Gcc12-msvcrt\mingw64\bin\gcc.exe

c:\PerlWinLib\perl5\win32>gcc --version
gcc (MinGW-W64 x86_64-ucrt-posix-seh, built by Brecht Sanders) 12.2.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

C:\PerlWinLib\perl5\win32>where gmake
C:\Winlibs64-Gcc12-msvcrt\mingw64\bin\gmake.exe

c:\PerlWinLib\perl5\win32>gmake --version
GNU Make 4.3
Built for x86_64-w64-mingw32
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

To build perl I needed to set the environment variable CCHOME:

c:\PerlWinLib\perl5\win32>set CCHOME=C:\Winlibs64-Gcc12-msvcrt\mingw64

Now, I was able to build perl:

c:\PerlWinLib\perl5\win32>powershell "gmake 2>&1 | tee gmake.log"

The log file is here: gmake.log

Then running the tests resulted in some failed tests:

c:\PerlWinLib\perl5\win32>powershell "gmake test 2>&1 | tee gmake-test.log"
[...]
Test Summary Report
-------------------
porting/customized.t                                               (Wstat: 0 Tests: 34 Failed: 34)
  Failed tests:  1-34
porting/pod_rules.t                                                (Wstat: 0 Tests: 5 Failed: 4)
  Failed tests:  2-5
  Parse errors: Bad plan.  You planned 7 tests but ran 5.
porting/regen.t                                                    (Wstat: 65280 (exited 255) Tests: 12 Failed: 1)
  Failed test:  12
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 42 tests but ran 12.
../cpan/CPAN-Meta-YAML/t/30_yaml_spec_tml.t                        (Wstat: 512 (exited 2) Tests: 0 Failed: 0)
  Non-zero exit status: 2
  Parse errors: No plan found in TAP output
../cpan/CPAN-Meta-YAML/t/31_local_tml.t                            (Wstat: 512 (exited 2) Tests: 0 Failed: 0)
  Non-zero exit status: 2
  Parse errors: No plan found in TAP output
../cpan/CPAN-Meta-YAML/t/32_world_tml.t                            (Wstat: 512 (exited 2) Tests: 0 Failed: 0)
  Non-zero exit status: 2
  Parse errors: No plan found in TAP output
../ext/IPC-Open3/t/IPC-Open3.t                                     (Wstat: 0 Tests: 45 Failed: 0)
  TODO passed:   25
Files=2730, Tests=1121645, 1457 wallclock secs (49.59 usr + 10.36 sys = 59.95 CPU)
Result: FAIL
gmake: *** [GNUmakefile:1821: test] Error 39

The complete test log is here: gmake-test.log

I will start at the top of the log, the first issue I see is at line 437:

..\perl.exe -f ..\pod\buildtoc -q
Use of uninitialized value $_ in substitution (s///) at ..\pod\buildtoc line 215, <$fh> chunk 2.
Use of uninitialized value $_ in pattern match (m//) at ..\pod\buildtoc line 216, <$fh> chunk 2.
Use of uninitialized value $_ in substitution (s///) at ..\pod\buildtoc line 219, <$fh> chunk 2.
Use of uninitialized value $_ in substitution (s///) at ..\pod\buildtoc line 219, <$fh> chunk 2.
[...]

[sisyphus]

All tests passed for me on Windows 11 using gcc-12.2.0 and building in Powershell.
(I think this is the first time I've built it in the powershell - I usually build in cmd.exe.)

The porting/customized.t failures suggest that the specified files have been changed.
Maybe it's just that their line endings have been altered from unix to windows - as could happen in the unpacking of the perl source.
@hakonhagland, did you unpack the perl source using a utility that automatically converts line endings to Windows format ?

The porting/regen.t failures don't mean much to me. Perhaps it's a similar issue.
Others here probably have a better idea.

And the diagnostics provided by the CPAN-Meta-YAML test failures don't help me much.
A better diagnostic output might be achieved by cd'ing to the cpan/CPAN-Meta-YAML directory and running:

..\..\perl -I..\..\lib t\30_yaml_spec_tml.t

Cheers,
Rob

[hakonhagland]

did you unpack the perl source using a utility that automatically converts line endings to Windows format ?

@sisyphus Yes, I think when I installed Git for Windows I chose to configure line ending conversions. I also see the following:

> git config --get core.autocrlf
true

[hakonhagland]

did you unpack the perl source using a utility that automatically converts line endings to Windows format ?

@sisyphus I now tried to set autocrlf to "input"

>git config --global core.autocrlf input

This setting should avoid having git do automatic line ending conversion on checkout. Then, recloned https://github.com/Perl/perl5.git and repeated the steps above (gmake, gmake test) and now all tests passed. I still got some errors when running gmake install afterwards. I plan to report them in a new issue later!

[hakonhagland]

I still got some errors when running gmake install afterwards. I plan to report them in a new issue

Added a new issue here: #20394

[khwilliamson]

I get these too, and it's not the compiler. I think it is line endings. Also for the pod rules and porting regen.

Apparently it works if you set git to use lf.

But I'm thinking we should work with native line endings, and that these are bugs. The porting SHA failures are likely because we have extra CRs in the file that messing things up. I would think we should just skip that test on Windows. Is there a good reason that this test needs to run on this box?

[sisyphus]

Is there a good reason that this test needs to run on this box?

Is there a good reason to run it on any box ?
The painful thing is that line endings (and permission bits) can change very easily when you edit files on Windows - though it depends upon the tools that you use.

There's nothing more annoying than submitting a PR only to find that the CI tests fail because of (unrequested and unannounced) changes to line endings.

I think that skipping the tests on Windows would make things worse. If the tests aren't skipped then at least testing your build on Windows prior to making a PR would expose the issue before the PR is made.

And I think it's good that there's something to tell us Windows people that windows endings are not allowed on git.

Perhaps the FAIL message could suggest that "hey, I see you're on Windows - maybe it's just that this file's line endings have been changed ".

Is there any reason that these files really need to have Unix line endings ?
How would it sit if we allowed them to have either - and modified the tests to allow both forms ?

Cheers,
Rob

[jkeenan]

My reading of this ticket is that the specific issue about which @hakonhagland was originally concerned was resolved as of the actions he took and described in this comment: #20391 (comment). The subsequent posts from @khwilliamson and @sisyphus broach a policy discussion (to oversimplify: how do we account for Windows line endings in our porting tests) which would, IMO, be better conducted on the mailing list.

I therefore recommend that this ticket be closed. Any objections?

[sisyphus]

No objection from me.

[jkeenan]

My reading of this ticket is that the specific issue about which @hakonhagland was originally concerned was resolved as of the actions he took and described in this comment: #20391 (comment). The subsequent posts from @khwilliamson and @sisyphus broach a policy discussion (to oversimplify: how do we account for Windows line endings in our porting tests) which would, IMO, be better conducted on the mailing list.

I therefore recommend that this ticket be closed. Any objections?

No objections received. Closing ticket.