Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add user agent to Databricks requests #20660

Merged
merged 1 commit into from
Jul 20, 2022
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
chore: add user agent to Databricks requests
  • Loading branch information
betodealmeida committed Jul 8, 2022

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit f8be567f2379b0b55962827ab5d037ba5b6ea9c4
5 changes: 1 addition & 4 deletions docs/docs/databases/databricks.mdx
Original file line number Diff line number Diff line change
@@ -33,13 +33,10 @@ You also need to add the following configuration to "Other" -> "Engine Parameter

```json
{
"connect_args": {"http_path": "sql/protocolv1/o/****"},
"http_headers": [["User-Agent", "Apache Superset"]]
"connect_args": {"http_path": "sql/protocolv1/o/****"}
}
```

The `User-Agent` header is optional, but helps Databricks identify traffic from Superset. If you need to use a different header please reach out to Databricks and let them know.

## Older driver

Originally Superset used `databricks-dbapi` to connect to Databricks. You might want to try it if you're having problems with the official Databricks connector:
2 changes: 2 additions & 0 deletions superset/constants.py
Original file line number Diff line number Diff line change
@@ -20,6 +20,8 @@
# string to use when None values *need* to be converted to/from strings
from enum import Enum

USER_AGENT = "Apache Superset"

NULL_STRING = "<NULL>"
EMPTY_STRING = "<empty string>"

18 changes: 17 additions & 1 deletion superset/db_engine_specs/databricks.py
Original file line number Diff line number Diff line change
@@ -16,11 +16,15 @@
# under the License.

from datetime import datetime
from typing import Any, Dict, Optional
from typing import Any, Dict, Optional, TYPE_CHECKING

from superset.constants import USER_AGENT
from superset.db_engine_specs.base import BaseEngineSpec
from superset.db_engine_specs.hive import HiveEngineSpec

if TYPE_CHECKING:
from superset.models.core import Database

time_grain_expressions = {
None: "{col}",
"PT1S": "date_trunc('second', {col})",
@@ -71,3 +75,15 @@ class DatabricksNativeEngineSpec(DatabricksODBCEngineSpec):
engine = "databricks"
engine_name = "Databricks Native Connector"
driver = "connector"

@staticmethod
def get_extra_params(database: "Database") -> Dict[str, Any]:
"""
Add a user agent to be used in the requests.
"""
extra = {
"http_headers": [("User-Agent", USER_AGENT)],
"_user_agent_entry": USER_AGENT,
}
extra.update(BaseEngineSpec.get_extra_params(database))
return extra