Code

AWS Batch Executor#37618

Code

Merged

o-nikolas merged 19 commits intoapache:mainapache/airflow:mainfrom aws-mwaa:syedahsn/aws_executors_batchaws-mwaa/upstream-to-airflow:syedahsn/aws_executors_batchCopy head branch name to clipboardMar 5, 2024

+2,344-8

Commits

Commits on Mar 1, 2024

• Introduce AWS Executors to the Amazon provider package. The EcsFargateExecutor and BatchExecutor launches Airflow tasks on AWS ECS/Fargate service and the AWS Batch service, respectively.

  

  aelzeiny

  authored and

  syedahsn

  committed
  453099f

  Copy full SHA for 453099f

• Initial commit to break up code into different modules

  syedahsn

  committed
  cb873d7

  Copy full SHA for cb873d7

• WIP: Changes to get executor working

  syedahsn

  committed
  77f0ca9

  Copy full SHA for 77f0ca9

• Added option to allow users to pass in config options.

  syedahsn

  committed
  34d3998

  Copy full SHA for 34d3998

• prevent any failures in the executor from killing the scheduler

  syedahsn

  committed
  0bf0a51

  Copy full SHA for 0bf0a51

• WIP: Move add_job inside try block

  syedahsn

  committed
  04909eb

  Copy full SHA for 04909eb

• Add documentation for Batch Executor

  Show description for 7dd43ab
  syedahsn

  committed
  7dd43ab

  Copy full SHA for 7dd43ab

• Health check and Token Expiration

  syedahsn

  committed
  bcc5966

  Copy full SHA for bcc5966

• Retries on task failures

  syedahsn

  committed
  4cf1dad

  Copy full SHA for 4cf1dad

• Remove old config files

  syedahsn

  committed
  0e96b4a

  Copy full SHA for 0e96b4a

• Add missing doc string, and update outdated doc string

  syedahsn

  committed
  160cfab

  Copy full SHA for 160cfab

• Add unit tests, and move Dockerfile to executor folder

  syedahsn

  committed
  ff71352

  Copy full SHA for ff71352

• Fix failing tests

  syedahsn

  committed
  0081d17

  Copy full SHA for 0081d17

• Add doc string to base config key class

  syedahsn

  committed
  c054b09

  Copy full SHA for c054b09

• Remove assert statement from non testing code

  Show description for 860f4eb
  syedahsn

  committed
  860f4eb

  Copy full SHA for 860f4eb

• Make requested changes

  Show description for c81477f
  syedahsn

  committed
  c81477f

  Copy full SHA for c81477f

• Fix test name

  Show description for a2e5d0b
  syedahsn

  committed
  a2e5d0b

  Copy full SHA for a2e5d0b

• Fix failing test

  syedahsn

  committed
  0129e75

  Copy full SHA for 0129e75

Commits on Mar 4, 2024

• Add comment about Batch Job failure

  Show description for e707935
  syedahsn

  committed
  e707935

  Copy full SHA for e707935