Skip to content

Commit

Permalink
fix(runPipelineNode): unmounting overlapping paths
Browse files Browse the repository at this point in the history
If multiple paths are mounted such that one is a parent of the
other, then you may encounter the following exceptions:

```
Rejected promise returned by test. Reason:

  ErrnoError {
    code: 'EINVAL',
    errno: 28,
    message: 'Invalid argument',
  }

Rejected promise returned by test. Reason:

  ErrnoError {
    code: 'ENOENT',
    errno: 44,
    message: 'No such file or directory',
  }
```

This fix will determine paths that are safely unmountable,
and only unmount each path once.
  • Loading branch information
jadh4v committed Aug 2, 2024
1 parent 6e335dc commit 065e403
Show file tree
Hide file tree
Showing 13 changed files with 93 additions and 6 deletions.
Binary file not shown.
Binary file not shown.

Large diffs are not rendered by default.

Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -50,17 +50,26 @@ async function runPipelineNode (
pipelinePath
)) as PipelineEmscriptenModule
const mountedDirs: Set<string> = new Set()
const unmountable: Set<string> = new Set()
if (typeof mountDirs !== 'undefined') {
mountDirs.forEach((dir) => {
mountedDirs.add(Module.mountDir(dir))
})
/**
* Identify mountable dirs. Some paths may be parent to others.
* Only keep the parent paths, to avoid error when unmounting.
*/
Array.from(mountedDirs)
.filter((x, _, a) => a.every((y) => x === y || !x.includes(y)))
.forEach((dir) => unmountable.add(dir)
)
}
if (typeof mountDirs !== 'undefined') {
args = replaceArgumentsWithEmscriptenPaths(args, mountDirs)
}
const result = runPipelineEmscripten(Module, args, outputs, inputs)
if (typeof mountDirs !== 'undefined') {
mountedDirs.forEach((dir) => {
unmountable.forEach((dir) => {
Module.unmountDir(dir)
})
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,84 @@ test('runPipelineNode uses input and output text and binary data via memory io',
)
})

test.only('runPipelineNode uses input and output text and binary files', (t) => {
const pipelinePath = path.resolve(
'test',
'pipelines',
'emscripten-build',
'input-output-files-pipeline',
'input-output-files-test'
)
const testInputTextFile = path.resolve(
'test',
'data',
'cow.iwm',
'index.json'
)
const testInputBinFile = path.resolve(
'test',
'data',
'cow.iwm',
'data',
'cells.raw'
)
const testOutputTextFile = path.resolve(
'test',
'data',
'cow.iwm',
'data',
'output.txt'
)
const testOutputBinFile = path.resolve(
'test',
'data',
'cow.iwm',
'data',
'output.bin'
)

const args = [
'--memory-io',
'--input-text-stream',
'0',
'--input-binary-stream',
'1',
'0',
'1',
testOutputTextFile,
testOutputBinFile,
'-f',
'--input-text-file',
testInputTextFile,
'--input-binary-file',
testInputBinFile,
]
const desiredOutputs = [
{ type: InterfaceTypes.TextStream },
{ type: InterfaceTypes.BinaryStream }
]
const inputs = [
{ type: InterfaceTypes.TextStream, data: { data: 'The answer is 42.' } },
{
type: InterfaceTypes.BinaryStream,
data: { data: new Uint8Array([222, 173, 190, 239]) }
}
]

const mountDirs = new Set()
mountDirs.add(path.dirname(testInputTextFile))
mountDirs.add(path.dirname(testInputBinFile))
mountDirs.add(path.dirname(testOutputTextFile))
mountDirs.add(path.dirname(testOutputBinFile))

return runPipelineNode(pipelinePath, args, desiredOutputs, inputs, mountDirs).then(
function ({ stdout, stderr, outputs }) {
t.is(outputs[0].type, InterfaceTypes.TextStream)
t.is(outputs[1].type, InterfaceTypes.BinaryStream)
}
)
})

test('runPipelineNode uses input and output json data via memory io', (t) => {
const pipelinePath = path.resolve(
'test',
Expand Down

Large diffs are not rendered by default.

Binary file not shown.
Binary file not shown.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

0 comments on commit 065e403

Please sign in to comment.