Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

debug-mismatches: single thread mode that avoids threadpool entirely #108

Merged
merged 1 commit into from
Jan 14, 2025
Merged

debug-mismatches: single thread mode that avoids threadpool entirely #108

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions commands.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,9 @@ $ solana-test-suite create-env [OPTIONS]

**Options**:

* `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path [default: /data/mjain/repos/solfuzz-agave/target/release/libsolfuzz_agave.so]
* `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path [default: impl/lib/libsolfuzz_agave_v2.0.so]
* `-h, --default-harness-type TEXT`: Harness type to use for Context protobufs [default: InstrHarness]
* `-t, --target PATH`: Shared object (.so) target file paths (pairs with --keep-passing). Targets must have required function entrypoints defined [default: /data/mjain/repos/firedancer/build/native/clang/lib/libfd_exec_sol_compat.so]
* `-t, --target PATH`: Shared object (.so) target file paths (pairs with --keep-passing). Targets must have required function entrypoints defined [default: impl/lib/libsolfuzz_firedancer.so]
* `-o, --output-dir PATH`: Output directory for messages [default: debug_mismatch]
* `-u, --repro-urls TEXT`: Comma-delimited list of FuzzCorp mismatch links
* `-n, --section-names TEXT`: Comma-delimited list of FuzzCorp section names
Expand Down Expand Up @@ -72,7 +72,7 @@ $ solana-test-suite create-fixtures [OPTIONS]

* `-i, --input PATH`: Input protobuf file or directory of protobuf files [default: corpus8]
* `-h, --default-harness-type TEXT`: Harness type to use for Context protobufs [default: InstrHarness]
* `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path [default: /data/mjain/repos/solfuzz-agave/target/release/libsolfuzz_agave.so]
* `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path [default: impl/lib/libsolfuzz_agave_v2.0.so]
* `-t, --target PATH`: Shared object (.so) target file paths (pairs with --keep-passing). Targets must have required function entrypoints defined
* `-o, --output-dir PATH`: Output directory for fixtures [default: test_fixtures]
* `-p, --num-processes INTEGER`: Number of processes to use [default: 4]
Expand All @@ -96,9 +96,9 @@ $ solana-test-suite debug-mismatches [OPTIONS]

**Options**:

* `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path [default: /data/mjain/repos/solfuzz-agave/target/release/libsolfuzz_agave.so]
* `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path [default: impl/lib/libsolfuzz_agave_v2.0.so]
* `-h, --default-harness-type TEXT`: Harness type to use for Context protobufs [default: InstrHarness]
* `-t, --target PATH`: Shared object (.so) target file paths (pairs with --keep-passing). Targets must have required function entrypoints defined [default: /data/mjain/repos/firedancer/build/native/clang/lib/libfd_exec_sol_compat.so]
* `-t, --target PATH`: Shared object (.so) target file paths (pairs with --keep-passing). Targets must have required function entrypoints defined [default: impl/lib/libsolfuzz_firedancer.so]
* `-o, --output-dir PATH`: Output directory for messages [default: debug_mismatch]
* `-u, --repro-urls TEXT`: Comma-delimited list of FuzzCorp mismatch links
* `-n, --section-names TEXT`: Comma-delimited list of FuzzCorp section names
Expand Down Expand Up @@ -211,8 +211,8 @@ $ solana-test-suite mass-regenerate-fixtures [OPTIONS]

* `-i, --input PATH`: Input test-vectors directory [default: corpus8]
* `-o, --output-dir PATH`: Output directory for regenerated fixtures [default: /tmp/regenerated_fixtures]
* `-t, --target PATH`: Shared object (.so) target file path to execute [default: /data/mjain/repos/solfuzz-agave/target/release/libsolfuzz_agave.so]
* `-s, --stubbed-target PATH`: Stubbed shared object (.so) target file path to execute [default: /data/mjain/repos/solfuzz-agave/target/debug/libsolfuzz_agave_stubbed.so]
* `-t, --target PATH`: Shared object (.so) target file path to execute [default: impl/lib/libsolfuzz_agave_v2.0.so]
* `-s, --stubbed-target PATH`: Stubbed shared object (.so) target file path to execute [default: impl/lib/libsolfuzz_firedancer.so]
* `-f, --add-feature TEXT`: List of feature pubkeys to force add to the fixtures.
* `-r, --remove-feature TEXT`: List of feature pubkeys to force remove from the fixtures.
* `-k, --rekey-feature TEXT`: List of feature pubkeys to rekey in the fixtures, formatted 'old/new' (e.g. `--rekey-feature old/new`).
Expand All @@ -233,7 +233,7 @@ $ solana-test-suite regenerate-fixtures [OPTIONS]
**Options**:

* `-i, --input PATH`: Either a file or directory containing messages [default: corpus8]
* `-t, --target PATH`: Shared object (.so) target file path to execute [default: /data/mjain/repos/solfuzz-agave/target/release/libsolfuzz_agave.so]
* `-t, --target PATH`: Shared object (.so) target file path to execute [default: impl/lib/libsolfuzz_agave_v2.0.so]
* `-o, --output-dir PATH`: Output directory for regenerated fixtures [default: regenerated_fixtures]
* `-d, --dry-run`: Only print the fixtures that would be regenerated
* `-f, --add-feature TEXT`: List of feature pubkeys to force add to the fixtures.
Expand All @@ -260,8 +260,8 @@ $ solana-test-suite run-tests [OPTIONS]

* `-i, --input PATH`: Input protobuf file or directory of protobuf files [default: corpus8]
* `-h, --default-harness-type TEXT`: Harness type to use for Context protobufs [default: InstrHarness]
* `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path [default: /data/mjain/repos/solfuzz-agave/target/release/libsolfuzz_agave.so]
* `-t, --target PATH`: Shared object (.so) target file paths [default: /data/mjain/repos/firedancer/build/native/clang/lib/libfd_exec_sol_compat.so]
* `-s, --solana-target PATH`: Solana (or ground truth) shared object (.so) target file path [default: impl/lib/libsolfuzz_agave_v2.0.so]
* `-t, --target PATH`: Shared object (.so) target file paths [default: impl/lib/libsolfuzz_firedancer.so]
* `-o, --output-dir PATH`: Output directory for test results [default: test_results]
* `-p, --num-processes INTEGER`: Number of processes to use [default: 4]
* `-r, --randomize-output-buffer`: Randomizes bytes in output buffer before shared library execution
Expand Down
50 changes: 30 additions & 20 deletions src/test_suite/test_suite.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -432,16 +432,21 @@ def run_tests(
# Process the test results in parallel
print("Running tests...")
test_case_results = []
with Pool(
processes=num_processes,
initializer=initialize_process_output_buffers,
initargs=(randomize_output_buffer,),
) as pool:
for result in tqdm.tqdm(
pool.imap(run_test, test_cases),
total=num_test_cases,
):
test_case_results.append(result)
if num_processes > 1:
with Pool(
processes=num_processes,
initializer=initialize_process_output_buffers,
initargs=(randomize_output_buffer,),
) as pool:
for result in tqdm.tqdm(
pool.imap(run_test, test_cases),
total=num_test_cases,
):
test_case_results.append(result)
else:
initialize_process_output_buffers(randomize_output_buffer)
for test_case in tqdm.tqdm(test_cases):
test_case_results.append(run_test(test_case))

print("Logging results...")
passed, failed, skipped, target_log_files, failed_tests, skipped_tests = (
Expand Down Expand Up @@ -684,16 +689,21 @@ def debug_mismatches(
num_test_cases = len(custom_data_urls)
print("Downloading tests...")
results = []
with Pool(
processes=num_processes,
initializer=initialize_process_output_buffers,
initargs=(randomize_output_buffer,),
) as pool:
for result in tqdm.tqdm(
pool.imap(download_and_process, custom_data_urls),
total=num_test_cases,
):
results.append(result)
if num_processes > 1:
with Pool(
processes=num_processes,
initializer=initialize_process_output_buffers,
initargs=(randomize_output_buffer,),
) as pool:
for result in tqdm.tqdm(
pool.imap(download_and_process, custom_data_urls),
total=num_test_cases,
):
results.append(result)
else:
initialize_process_output_buffers(randomize_output_buffer)
for url in tqdm.tqdm(custom_data_urls):
results.append(download_and_process(url))

if ld_preload is not None:
os.environ["LD_PRELOAD"] = ld_preload
Expand Down
Loading