Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[teraslice-cli] jobs-spec test fails reading file that was just created. #3844

Closed
busma13 opened this issue Nov 27, 2024 · 2 comments
Closed

Comments

@busma13
Copy link
Contributor

busma13 commented Nov 27, 2024

https://github.com/terascope/teraslice/actions/runs/11620752664/job/32363197551?pr=3813

Summary of all failing tests
FAIL packages/teraslice-cli/test/helpers/jobs-spec.ts (9.849 s)
● Job helper class › export › should export all jobs to a custom directory

/home/runner/work/teraslice/teraslice/packages/teraslice-cli/test/fixtures/job_exports/custom2/test-job-2.json: ENOENT: no such file or directory, open '/home/runner/work/teraslice/teraslice/packages/teraslice-cli/test/fixtures/job_exports/custom2/test-job-2.json'

  1110 |             jobFiles.push(fs.readJsonSync(path.join(customDir, 'test-job.json')));
  1111 |             jobFiles.push(fs.readJsonSync(path.join(customDir, 'test-job-1.json')));
> 1112 |             jobFiles.push(fs.readJsonSync(path.join(customDir, 'test-job-2.json')));
       |                              ^
  1113 |
  1114 |             expect(jobFiles).toEqual(expect.arrayContaining([
  1115 |                 expect.objectContaining(

  at Object.readFileSync (node_modules/jsonfile/index.js:50:22)
  at Object.readJsonSync (packages/teraslice-cli/test/helpers/jobs-spec.ts:1112:30)
@busma13
Copy link
Contributor Author

busma13 commented Nov 27, 2024

I think this has something to do with how jobs with the same name are saved in job.export().

await job.export();

// Since all 3 jobs have the same file name, and exportOne() is called concurrently on
// each job, 2 of the jobs may have to call createUniqueFilePath() a second or
// third time. This race condition means there is no order to the file names.

@busma13
Copy link
Contributor Author

busma13 commented Jan 31, 2025

In #3887 this was solved by setting concurreny to 1, so only one job is exported at a time.
#3936 cleans up that fix to remove the pMap entirely.

busma13 added a commit that referenced this issue Jan 31, 2025
…t pMap (#3936)

When running `teraslice-cli jobs export`, remove the pMap wrapper from
the call to `exportOne()` as it is useless if we set concurrency to 1.

Bump teraslice-cli from `2.10.1` to `2.10.2`

ref: #3844
@busma13 busma13 closed this as completed Jan 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant