Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated credentials docs "Added section Destination configuration via Python" #2182

Merged
merged 8 commits into from
Jan 13, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 26 additions & 2 deletions docs/website/docs/general-usage/credentials/advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,31 @@ dlt.secrets["destination.postgres.credentials"] = BaseHook.get_connection('postg

This will mock the TOML provider to desired values.

## Configuring destination credentials in code

If you need to manage destination credentials programmatically, such as retrieving them from environment variables, you can define them directly in your pipeline code.

The following example demonstrates how to use [GcpServiceAccountCredentials](complex_types#gcp-credentials) to set up a pipeline with a BigQuery destination.

```py
import os

import dlt
from dlt.sources.credentials import GcpServiceAccountCredentials
from dlt.destinations import bigquery

# Retrieve credentials from the environment variable
creds_dict = os.getenv('BIGQUERY_CREDENTIALS')

# Create credentials instance and parse them from a native representation
gcp_credentials = GcpServiceAccountCredentials()
gcp_credentials.parse_native_representation(creds_dict)

# Pass the credentials to the BigQuery destination
pipeline = dlt.pipeline(destination=bigquery(credentials=gcp_credentials))
pipeline.run([{"key1": "value1"}], table_name="temp")
```

## Example

In the example below, the `google_sheets` source function is used to read selected tabs from Google Sheets.
Expand Down Expand Up @@ -200,5 +225,4 @@ In the example above:
:::tip
`dlt.resource` behaves in the same way, so if you have a [standalone resource](../resource.md#declare-a-standalone-resource) (one that is not an inner function
of a **source**)
:::

:::
Loading