sporadic native_engine.IntrinsicError on main with macOS12-x86_64

[cburroughs]

See:
d48b643
and
f9efe72

Example trace:

Error: 9.04 [ERROR] 1 Exception encountered:

Engine traceback:
  in root
    ..
  in pants.init.plugin_resolver.resolve_plugins
    ..

Traceback (most recent call last):
  File "/Users/runner/work/pants/pants/src/python/pants/init/plugin_resolver.py", line 77, in resolve_plugins
    plugins_pex = await Get(
  File "/Users/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 124, in __await__
    result = yield self
  File "/Users/runner/work/pants/pants/src/python/pants/backend/python/util_rules/pex.py", line 1085, in create_venv_pex
    venv_pex_result = await Get(BuildPexResult, PexRequest, seeded_venv_request)
  File "/Users/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 124, in __await__
    result = yield self
  File "/Users/runner/work/pants/pants/src/python/pants/backend/python/util_rules/pex.py", line 767, in build_pex
    result = await Get(
  File "/Users/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 124, in __await__
    result = yield self
  File "/Users/runner/work/pants/pants/src/python/pants/engine/intrinsics.py", line 107, in execute_process
    return await native_engine.execute_process(process, process_execution_environment)
native_engine.IntrinsicError: Could not identify a process to backtrack to for: Missing digest: Was not present in the local store: Digest { hash: Fingerprint<1cc4816cab0ed8c495b16f4bd7b00fa00ba0aa4d92f9450429c59678b6e0b21a>, size_bytes: 1332 }

Traceback (most recent call last):
  File "/Users/runner/work/pants/pants/src/python/pants/bin/daemon_pants_runner.py", line 134, in single_daemonized_run
    runner = LocalPantsRunner.create(
  File "/Users/runner/work/pants/pants/src/python/pants/bin/local_pants_runner.py", line 95, in create
    build_config = options_initializer.build_config(options_bootstrapper, env)
  File "/Users/runner/work/pants/pants/src/python/pants/init/options_initializer.py", line 154, in build_config
    return _initialize_build_configuration(self._plugin_resolver, options_bootstrapper, env)
  File "/Users/runner/work/pants/pants/src/python/pants/init/options_initializer.py", line 59, in _initialize_build_configuration
    working_set = plugin_resolver.resolve(options_bootstrapper, env, backends_requirements)
  File "/Users/runner/work/pants/pants/src/python/pants/init/plugin_resolver.py", line 141, in resolve
    for resolved_plugin_location in self._resolve_plugins(options_bootstrapper, env, request):
  File "/Users/runner/work/pants/pants/src/python/pants/init/plugin_resolver.py", line 166, in _resolve_plugins
    session.product_request(ResolvedPluginDistributions, [params])[0],
  File "/Users/runner/work/pants/pants/src/python/pants/engine/internals/scheduler.py", line 593, in product_request
    return self.execute(request)
  File "/Users/runner/work/pants/pants/src/python/pants/engine/internals/scheduler.py", line 534, in execute
    self._raise_on_error([t for _, t in throws])
  File "/Users/runner/work/pants/pants/src/python/pants/engine/internals/scheduler.py", line 518, in _raise_on_error
    raise ExecutionError(
pants.engine.internals.scheduler.ExecutionError: 1 Exception encountered:

Engine traceback:
  in root
    ..
  in pants.init.plugin_resolver.resolve_plugins
    ..

Traceback (most recent call last):
  File "/Users/runner/work/pants/pants/src/python/pants/init/plugin_resolver.py", line 77, in resolve_plugins
    plugins_pex = await Get(
  File "/Users/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 124, in __await__
    result = yield self
  File "/Users/runner/work/pants/pants/src/python/pants/backend/python/util_rules/pex.py", line 1085, in create_venv_pex
    venv_pex_result = await Get(BuildPexResult, PexRequest, seeded_venv_request)
  File "/Users/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 124, in __await__
    result = yield self
  File "/Users/runner/work/pants/pants/src/python/pants/backend/python/util_rules/pex.py", line 767, in build_pex
    result = await Get(
  File "/Users/runner/work/pants/pants/src/python/pants/engine/internals/selectors.py", line 124, in __await__
    result = yield self
  File "/Users/runner/work/pants/pants/src/python/pants/engine/intrinsics.py", line 107, in execute_process
    return await native_engine.execute_process(process, process_execution_environment)
native_engine.IntrinsicError: Could not identify a process to backtrack to for: Missing digest: Was not present in the local store: Digest { hash: Fingerprint<1cc4816cab0ed8c495b16f4bd7b00fa00ba0aa4d92f9450429c59678b6e0b21a>, size_bytes: 1332 }

[cburroughs]

cc @tdyas & @benjyw as commit authors. No idea if it is causal or unlucky.

[tdyas]

I recall it has happened for other commits in the past and not just these recent commits.

[cburroughs]

Similar failures on main:

• 01137e5
• bfb0b19
• 2c95b17

They all appear x86-64 specific, so maybe they go away along with #21333? (My understanding of the Apple ecosystem is hazy, but I thought it had been some years since one could buy a x86_64 mac)

[huonw]

This may be the same error/problem as #19748

[cburroughs]

https://github.com/pantsbuild/pants/actions/runs/11874982396/job/33091744077 from 1258080

---

Edit by @tgolsson - full info:

 native_engine.IntrinsicError: Could not identify a process to backtrack to for: Missing digest: Was not present in the local store: Digest { hash: Fingerprint<115d7bf09e568eead60e12325b0ba3c0cf31f0e704edf13f8428ecafe16335dd>, size_bytes: 6575 }, with workunit: Workunit { name: "pants.engine.intrinsics.parse_python_deps", level: Trace, span_id: SpanId(7334607171583902408), parent_ids: [SpanId(15861588436617339344)], state: Started { start_time: SystemTime { tv_sec: 1731808968, tv_nsec: 464126000 }, blocked: false }, metadata: None }