Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Run Linux AArch64 tests on real hardware using Cirrus CI #180

Merged
merged 2 commits into from
Sep 18, 2022

Conversation

tatsuya6502
Copy link
Member

This PR adds Cirrus CI job to run Linux AArch64 tests on real hardware.

Signed-off-by: ClSlaid <cailue@bupt.edu.cn>
@tatsuya6502
Copy link
Member Author

It seems we need to install pkg-config and openssl library on Cirrus CI containers.
https://cirrus-ci.com/task/5134163287212032

Trying to run the docker images locally to figure out what will be needed.

@tatsuya6502 tatsuya6502 linked an issue Sep 15, 2022 that may be closed by this pull request
@tatsuya6502
Copy link
Member Author

It is interesting.

  • All tests are passing on the rustlang's Rust nightly image.
  • The following tests are failing on the Docker's official Rust images (stable and MSRV).
failures:
    sync::cache::tests::drop_value_immediately_after_eviction
    sync::cache::tests::get_with
    sync::cache::tests::get_with_if
    sync::cache::tests::test_key_lock_used_by_immediate_removal_notifications
    sync::cache::tests::try_get_with
    sync::segment::tests::drop_value_immediately_after_eviction
    sync::segment::tests::get_with_if
    sync::segment::tests::try_get_with

https://cirrus-ci.com/build/6464088501911552

@tatsuya6502
Copy link
Member Author

tatsuya6502 commented Sep 18, 2022

The following tests are failing on the Docker's official Rust images (stable and MSRV).

Fixed them by limiting the numbers of concurrent jobs and test threads for cargo test.

env:
  NUM_CPUS: "2"

linux_arm64_task:
  arm_container:
    cpu: $NUM_CPUS
    matrix:
      - image: rust:slim  # docker's official latest rust stable version
      - image: rustlang/rust:nightly-slim # nightly hosted by rustlang
      - image: rust:1.51.0-slim # MSRV

  test_script:
    # Run tests (debug, sync feature)
    - cargo test -j 1 --lib --features sync -- --test-threads=$NUM_CPUS
    - ...

I found that Cirrus CI environment exposes the real number of CPU cores (64 cores) of the node, while it is allowing the test container to use only 2 cores. This made cargo test to run a way more tests at the same time, and unit tests in Moka tend to spawn many threads, causing some threads not to be given enough CPU time.

$ nproc
64

$ lscpu
Architecture:                    aarch64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
CPU(s):                          64
On-line CPU(s) list:             0-63
Thread(s) per core:              1
Core(s) per socket:              64
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       ARM
Model:                           1
Model name:                      Neoverse-N1
Stepping:                        r3p1
BogoMIPS:                        243.75
L1d cache:                       4 MiB
L1i cache:                       4 MiB
L2 cache:                        64 MiB
L3 cache:                        32 MiB
NUMA node0 CPU(s):               0-63
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:        Mitigation; __user pointer sanitization
Vulnerability Spectre v2:        Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp ssbs

@tatsuya6502 tatsuya6502 marked this pull request as ready for review September 18, 2022 16:01
Copy link
Member Author

@tatsuya6502 tatsuya6502 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merging.

- Try not to use matrix for `cargo test`s.
- Limit the number of base test threads to the available cores.
- Show the CPU info.
- Install OpenSSL library to the Docker containers.
- Disable caching as there is no `Cargo.lock` file in the repository.
@tatsuya6502 tatsuya6502 self-assigned this Sep 18, 2022
@tatsuya6502 tatsuya6502 added this to the v0.9.5 milestone Sep 18, 2022
@tatsuya6502 tatsuya6502 merged commit 3e570d4 into master Sep 18, 2022
@tatsuya6502 tatsuya6502 deleted the cirrus-ci-arm-temp branch September 18, 2022 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CI: Run Linux AArch64 tests on real hardware
2 participants