Code

improves pipeline state management#89

Code

Merged

rudolfix merged 7 commits intodeveldlt-hub/dlt:develfrom rfix/improves-pipeline-state-managementdlt-hub/dlt:rfix/improves-pipeline-state-managementCopy head branch name to clipboardNov 20, 2022

+719-249

Commits

Commits on Nov 18, 2022

• provides separate states per source, provides write only state when source/resource is read outside of the pipeline

  rudolfix

  committed
  afc2503

  Copy full SHA for afc2503

Commits on Nov 20, 2022

• injected required arguments are not converted to optional, instead dlt.config/secret.value or None must be explititly passed

  rudolfix

  committed
  211bd69

  Copy full SHA for 211bd69

• removes all whitespaces from json.dump

  rudolfix

  committed
  d1a3a20

  Copy full SHA for d1a3a20

• adds config accessors, exposes more util functions in config resolve

  rudolfix

  committed
  2ee8c9c

  Copy full SHA for 2ee8c9c

• does not restore state with unknown load_id

  rudolfix

  committed
  f9040e3

  Copy full SHA for f9040e3

• adds writing toml documents to toml provider

  rudolfix

  committed
  417d4b6

  Copy full SHA for 417d4b6

• exposes config accessors and types for secrets and credentials in dlt module

  rudolfix

  committed
  63d5a80

  Copy full SHA for 63d5a80