[macOS-arm64] 55 System.DirectoryServices.Protocols.Tests fail on CoreCLR

[sdmaclea]

See https://dev.azure.com/dnceng/public/_build/results?buildId=1021947

First few failures.

    System.DirectoryServices.Protocols.Tests.PageResultRequestControlTests.Ctor_Cookie(cookie: null, expectedValue: [48, 6, 2, 2, 2, ...]) [FAIL]
      System.DirectoryServices.Protocols.BerConversionException : An error occurred during the BER conversion.
      Stack Trace:
        /_/src/libraries/System.DirectoryServices.Protocols/src/System/DirectoryServices/Protocols/common/BerConverter.cs(209,0): at System.DirectoryServices.Protocols.BerConverter.Encode(String format, Object[] value)
        /_/src/libraries/System.DirectoryServices.Protocols/src/System/DirectoryServices/Protocols/common/DirectoryControl.cs(599,0): at System.DirectoryServices.Protocols.PageResultRequestControl.GetValue()
        /_/src/libraries/System.DirectoryServices.Protocols/tests/PageResultRequestControlTests.cs(72,0): at System.DirectoryServices.Protocols.Tests.PageResultRequestControlTests.Ctor_Cookie(Byte[] cookie, Byte[] expectedValue)
    System.DirectoryServices.Protocols.Tests.SecurityDescriptorFlagControlTests.Ctor_Flags(masks: Group, expectedValue: [48, 3, 2, 1, 2]) [FAIL]
      Assert.Equal() Failure
      Expected: Byte[] [48, 3, 2, 1, 2]
      Actual:   Byte[] [48, 6, 2, 4, 112, ...]
      Stack Trace:
        /_/src/libraries/System.DirectoryServices.Protocols/tests/SecurityDescriptorFlagControlTests.cs(41,0): at System.DirectoryServices.Protocols.Tests.SecurityDescriptorFlagControlTests.Ctor_Flags(SecurityMasks masks, Byte[] expectedValue)
    System.DirectoryServices.Protocols.Tests.ExtendedDNControlTests.Ctor_Default [FAIL]
      Assert.Equal() Failure
      Expected: Byte[] [48, 3, 2, 1, 0]
      Actual:   Byte[] [48, 6, 2, 4, 112, ...]
      Stack Trace:
        /_/src/libraries/System.DirectoryServices.Protocols/tests/ExtendedDNControlTests.cs(23,0): at System.DirectoryServices.Protocols.Tests.ExtendedDNControlTests.Ctor_Default()
    System.DirectoryServices.Protocols.Tests.VlvRequestControlTests.Ctor_BeforeCount_AfterCount_ByteArrayTarget(beforeCount: 0, afterCount: 0, target: null, expectedValue: [48, 14, 2, 1, 0, ...]) [FAIL]
      Assert.Equal() Failure
      Expected: Byte[] [48, 14, 2, 1, 0, ...]
      Actual:   Byte[] [48, 30, 2, 4, 112, ...]
      Stack Trace:
        /_/src/libraries/System.DirectoryServices.Protocols/tests/VlvRequestControlTests.cs(102,0): at System.DirectoryServices.Protocols.Tests.VlvRequestControlTests.Ctor_BeforeCount_AfterCount_ByteArrayTarget(Int32 beforeCount, Int32 afterCount, Byte[] target, Byte[] expectedValue)
    System.DirectoryServices.Protocols.Tests.BerConverterTests.Encode_Objects_ReturnsExpected(format: "tetie", values: [128, 0, 133, 2, 3], expected: [128, 1, 0, 133, 1, ...]) [FAIL]
      Exception of type 'Xunit.Sdk.AssertActualExpectedException' was thrown.
      Expected: 128, 1, 0, 133, 1, 2, 10, 1, 3
      Actual:   1, 112, 0, 0, 116, 4, 112, 0, 0, 101, 1, 112, 0, 0, 116, 4, 112, 0, 0, 105, 10, 4, 112, 0, 0, 101
      Stack Trace:
        /_/src/libraries/Common/tests/TestUtilities/System/AssertExtensions.cs(370,0): at System.AssertExtensions.Equal[T](T[] expected, T[] actual)
        /_/src/libraries/System.DirectoryServices.Protocols/tests/BerConverterTests.cs(49,0): at System.DirectoryServices.Protocols.Tests.BerConverterTests.Encode_Objects_ReturnsExpected(String format, Object[] values, Byte[] expected)

/cc @sandreenko These look a bit like ABI issues.

[sandreenko]

fixed by #51205

[sdmaclea]

I would assume I disabled these tests, but I don't see the DisableTest tag or a reference to the method where I disabled it... I am going to reopen it, assign it to myself to make sure these tests are enabled.

Thanks @sandreenko

[sdmaclea]

They were disabled as part of #49400.

[sdmaclea]

Looks like they were reenabled by #51205.

[VincentBu]

System.DirectoryServices.Protocols.Tests failed again in runtime 20210428.85

Some of failed tests:

net6.0-Linux-Release-arm64-CoreCLR_checked-(Alpine.312.Arm64.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.12-helix-arm64v8-20200602002604-25f8a3e
     -System.DirectoryServices.Protocols.Tests.DirSyncRequestControlTests.Ctor_Cookie(cookie: [97, 98, 99], expectedValue: [48, 13, 2, 1, 0, ...])

Error message
Assert.Equal() Failure\nExpected: Byte[] [48, 13, 2, 1, 0, ...]\nActual:   Byte[] [255, 255, 255, 255, 22, ...]


Stack trace
   at System.DirectoryServices.Protocols.Tests.DirSyncRequestControlTests.Ctor_Cookie(Byte[] cookie, Byte[] expectedValue) in /_/src/libraries/System.DirectoryServices.Protocols/tests/DirSyncRequestControlTests.cs:line 49

net6.0-Linux-Release-arm64-CoreCLR_checked-(Alpine.312.Arm64.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.12-helix-arm64v8-20200602002604-25f8a3e
    -System.DirectoryServices.Protocols.Tests.BerConverterTests.Encode_Objects_ReturnsExpected(format: \"{vv}\", values: [null, [\"abc\", \"\", null]], expected: [48, 9, 4, 3, 97, ...])

Error message
Exception of type 'Xunit.Sdk.AssertActualExpectedException' was thrown.\nExpected: 48, 9, 4, 3, 97, 98, 99, 4, 0, 4, 0\nActual:   255, 255, 255, 255, 18, 255, 255, 255, 255, 3, 97, 98, 99, 255, 255, 255, 255, 0, 255, 255, 255, 255, 0


Stack trace
   at System.AssertExtensions.Equal[T](T[] expected, T[] actual) in /_/src/libraries/Common/tests/TestUtilities/System/AssertExtensions.cs:line 370
   at System.DirectoryServices.Protocols.Tests.BerConverterTests.Encode_Objects_ReturnsExpected(String format, Object[] values, Byte[] expected) in /_/src/libraries/System.DirectoryServices.Protocols/tests/BerConverterTests.cs:line 59

[sandreenko]

Interesting, so they are failing where they were not disabled previously (arm64 Linux, no MacOS), and they did not fail in the CI for the PR and the BER testing that was done, not sure what is going on.

[sandreenko]

It looks like it is passing on x64 Unix/OSX, but does not pass on arm64 Unix/OSX. I can't find the former pipelines that were running arm64 OSX so can't check there.
Because the failures are blocking innerloop I am going to revert the change and fix the issue locally.

[sdmaclea]

@sandreenko The libraries tests are not enabled on PRs right now. They are only on rolling builds. I am not sure if they are inner or outer. If you want to trigger those runs manually, I can probably figure out where the pipeline is...

[sandreenko]

@sandreenko The libraries tests are not enabled on PRs right now. They are only on rolling builds. I am not sure if they are inner or outer. If you want to trigger those runs manually, I can probably figure out where the pipeline is...

Yes, I am searching for a pipeline that runs arm64 MacOS libraries tests, would be nice if you can point me to it.

[sdmaclea]

The libraries innerloop test are run in the runtime rolling CI builds. Here https://dev.azure.com/dnceng/public/_build/results?buildId=1114297&view=results

The libraries outerloop tests are run here https://dev.azure.com/dnceng/public/_build?definitionId=678.

Unlike the runtime tests, for libraries the inner and outer loop sets are non overlapping. Outer is not a superset of inner (like it is for runtime tests)

[sandreenko]

The libraries outerloop tests are run here https://dev.azure.com/dnceng/public/_build?definitionId=678.

Does it include the test run? I see only Libraries Build jobs there.

[sdmaclea]

I see only Libraries Build jobs there.

The libraries tests can be run as part of the build job.

https://dev.azure.com/dnceng/public/_build/results?buildId=1113767&view=results
This one clearly ran some tests as part of the build. It had failures.

The osx one did too. It didn't have failures so I had to show passed tests in the tests tab to verify.

[sandreenko]

I see, thanks Steve, looks like in some pipelines we have "Libraries Test Run" where we run libraries tests, like: https://dev.azure.com/dnceng/public/_build/results?buildId=1114559&view=logs&j=ba66422d-a35c-5bc8-b6c6-3a55f049cbee
and "Libraries Build" only build the tests as expected https://dev.azure.com/dnceng/public/_build/results?buildId=1114559&view=logs&j=914575e7-736f-5eab-a41f-513880ce642e
but in the others, we run them as part of "Libraries Build", like https://dev.azure.com/dnceng/public/_build/results?buildId=1113761&view=results

[sandreenko]

notes for me:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=46942
https://groups.google.com/g/x86-64-abi/c/E8O33onbnGQ?pli=1

(lldb) disas -a 0x00007ffff7d7bc90
liblber-2.4.so.2`ber_put_boolean:
->  0x7ffff7d7bc90 <+0>:   endbr64
    0x7ffff7d7bc94 <+4>:   subq   $0x28, %rsp
    0x7ffff7d7bc98 <+8>:   movq   %fs:0x28, %rax
    0x7ffff7d7bca1 <+17>:  movq   %rax, 0x18(%rsp)
    0x7ffff7d7bca6 <+22>:  xorl   %eax, %eax
    0x7ffff7d7bca8 <+24>:  cmpq   $-0x1, %rdx

LDAP`ber_put_boolean:
->  0x196453d14 <+0>:   pacibsp
    0x196453d18 <+4>:   sub    sp, sp, #0x30             ; =0x30
    0x196453d1c <+8>:   stp    x29, x30, [sp, #0x20]
    0x196453d20 <+12>:  add    x29, sp, #0x20            ; =0x20
    0x196453d24 <+16>:  adrp   x8, 344613
    0x196453d28 <+20>:  ldr    x8, [x8, #0x7e8]
    0x196453d2c <+24>:  ldr    x8, [x8]
    0x196453d30 <+28>:  stur   x8, [x29, #-0x8]
    0x196453d34 <+32>:  cmn    x2, #0x1                  ; =0x1