Attaching to Batch job that failed to start results in ResourceNotFoundException

[joverlee521]

Current Behavior

Slack context

Attaching to a Batch job that was submitted but failed to start results in an uncaught error

$ nextstrain build --aws-batch --attach d3ab7b14-e7ca-4795-a9d5-29281b00acaa .
Attaching to Nextstrain AWS Batch Job ID: d3ab7b14-e7ca-4795-a9d5-29281b00acaa
Job is FAILED
Traceback (most recent call last):
  File "runpy", line 196, in _run_module_as_main
  File "runpy", line 86, in _run_code
  File "nextstrain.cli.__main__", line 43, in <module>
  File "nextstrain.cli.__main__", line 19, in main
  File "nextstrain.cli", line 37, in run
  File "nextstrain.cli.command.build", line 300, in run
  File "nextstrain.cli.runner", line 290, in run
  File "nextstrain.cli.runner.aws_batch", line 407, in run
  File "nextstrain.cli.runner.aws_batch.jobs", line 125, in log_entries
  File "nextstrain.cli.runner.aws_batch.logs", line 36, in fetch_stream
  File "botocore.paginate", line 269, in __iter__
  File "botocore.paginate", line 357, in _make_request
  File "botocore.client", line 569, in _api_call
  File "botocore.client", line 1023, in _make_api_call
botocore.errorfactory.ResourceNotFoundException: An error occurred (ResourceNotFoundException) when calling the FilterLogEvents operation: The specified log stream does not exist.

Expected behavior

Handle the unexpected ResourceNotFoundException and report the underlying failure reason of the job.

[joverlee521]

Unhandled error because ResourceNotFoundException is not one of the expected errors in the LogWatcher

cli/nextstrain/cli/runner/aws_batch/logs.py

Line 98 in 8cf025d

except (ClientError, BotocoreConnectionError):

[joverlee521]

Nvm, this is because the exception is raised further down

cli/nextstrain/cli/runner/aws_batch/logs.py

Lines 98 to 101 in 8cf025d

	except (ClientError, BotocoreConnectionError):
	failure_count += 1
	if failure_count > MAX_FAILURES and not success_count:
	raise

But then is not handled outside?

[joverlee521]

Hah, I was completely off in tracking down the error here 🤦‍♀️

This will be resolved by #406

[tsibley]

Ahh, I didn't see this issue thread until just now! (well after diagnosing and making the fix)