fetch and keep cloudwatch log identifiers

[geertvandeweyer]

output from (de) localization is not available in the stdout/stderr files available in job metadata.

can we extend the logGroup changes by @nathanthorpe (PR) to fetch the cloudwatch logs in full and add them to metadata ?

[geertvandeweyer]

added cloudwatch identifiers to job metadata (PR 55). Not fetching full logs to prevent overloads:

{'actualWorkflowLanguage': 'WDL',
              'actualWorkflowLanguageVersion': '1.0',
              'calls': 'Test_3.gather_data': [{'attempt': 1,
                                                'backend': 'AWSBatch',
                                                'backendStatus': 'Succeeded',
                                                'callCaching': {...},
                                                'executionEvents': [...],
                                                'executionStatus': 'Done',
                                                'inputs': {...},
                                                'jobId': 'fe55942c-c763-46de-af75-7f3843e312e2',
==> HERE                               'logConfig': {'logGroupName': '/aws/batch/job',
                                                              'logStreamName': 'cromwell_public_ecr_aws_lts_ubuntu_latest_33b1c150b7c629cfbac15d160b5b8296d5dfe374/default/c44a0d32a6bf451b82a2de31e613ad68',
                                                              'region': 'eu-west-2'},
                                                'outputs': {...},
                                                'returnCode': 0,
                                                'runtimeAttributes': {...},
                                                'shardIndex': -1,
                                                'start': '2025-01-09T12:18:11.008Z',
                                                'stderr': 's3://cromwell-temp-uza-eu-west-2/cromwell-execution/Test_3/fc3dfac1-2a67-4b72-8a9b-e9db7eb5ab89/call-gather_data/gather_data-stderr.log',
                                                'stdout': 's3://cromwell-temp-uza-eu-west-2/cromwell-execution/Test_3/fc3dfac1-2a67-4b72-8a9b-e9db7eb5ab89/call-gather_data/gather_data-stdout.log'}]}