Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GitHub workflow to pack BI connectors #19

Merged
20 commits merged into from
Dec 21, 2021
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
396ce1c
Added GitHub actions script to build Tableau and Power BI connector. …
Yury-Fridlyand Dec 17, 2021
1d3cceb
A typo fix in yml
Yury-Fridlyand Dec 17, 2021
c277fe2
A dummy commit to make GitHub Actions start the desired workflow
Yury-Fridlyand Dec 17, 2021
2be30f3
Workflow fix - pack Tableau ODBC connector instead of JDBC one
Yury-Fridlyand Dec 17, 2021
604adbf
Fix line endings in the workflow file
Yury-Fridlyand Dec 17, 2021
91dd569
Another dummy commit to start the desired workflow
Yury-Fridlyand Dec 17, 2021
6b89593
Fix paths in the GitHub workflow
Yury-Fridlyand Dec 17, 2021
015fa7b
Modified workflow to trigger on changes of itself
Yury-Fridlyand Dec 17, 2021
081392e
Path fix for preparation step
Yury-Fridlyand Dec 17, 2021
450d483
Added some changes to the second PBI connector - they should differ m…
Yury-Fridlyand Dec 17, 2021
a62bdd1
Renamed PBI connector files
Yury-Fridlyand Dec 17, 2021
47530d0
Moved Tableau JDBC connector to bi-connectors
Yury-Fridlyand Dec 17, 2021
21e0bdf
Added packing Tableau JDBC connector
Yury-Fridlyand Dec 17, 2021
f5692cc
Renamed Power BI connector as "OpenSearch Project"
Yury-Fridlyand Dec 17, 2021
a432ac2
Renamed output artifact
Yury-Fridlyand Dec 17, 2021
f53a46d
Merge branch 'powerbi-main' into add-github-workflow-pack-bi-connecto…
Yury-Fridlyand Dec 18, 2021
4cadfd1
Disabled building Tableau ODBC connector, because it is not ready for…
Yury-Fridlyand Dec 20, 2021
6e37870
Updated workflow step description
Yury-Fridlyand Dec 20, 2021
ccb52f4
Removed packing ODBC driver for Tableau (re: 4cadfd18).
Yury-Fridlyand Dec 20, 2021
2e67626
Updated workflow, because connector source files were renamed after m…
Yury-Fridlyand Dec 21, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions .github/workflows/bi-connectors.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
name: Build connectors for BI tools

on:
push:
paths:
- 'bi-connectors/PowerBIConnector/**'
- 'bi-connectors/TableauConnector/**'
- '.github/workflows/bi-connectors.yml'

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- name: Pack Tableau ODBC connector

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we pack both ODBC and JDBC? I thought we were only going to use JDBC going forward?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This script build both for future reference. At the moment, repo already contains a taco for ODBC, so nothing changed for ODBC actually.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ODBC one is now deprecated for the JDBC one, we should stop building the ODBC one.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disabled in 4cadfd1.

id: pack-tableau-odbc
run: |
zip -r opensearch_sql_odbc.taco . -x *.taco
working-directory: bi-connectors/TableauConnector/opensearch_sql_odbc
- name: Pack Tableau JDBC connector
id: pack-tableau-jdbc
run: |
zip -r opensearch_sql_jdbc.taco . -x *.taco
working-directory: bi-connectors/TableauConnector/opensearch_sql_jdbc
- name: Prepare Power BI ODBC connector
run: |
cp SqlOdbcPBIConnector.pq SqlOdbcPBIConnector.m

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these names right after the rebranding that occurred?

working-directory: bi-connectors/PowerBIConnector
- name: Pack Power BI ODBC connector
id: pack-powerbi-odbc-os-proj
run: |
zip OpenSearchProject.mez *.png *.m *.resx *.pqm
working-directory: bi-connectors/PowerBIConnector
- name: Prepare the second Power BI ODBC connector

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the second one...?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are 2 connector requested by Amazon: one for OpenSearch Project which is being build from the repo as is; and the second one - for Amazon OpenSearch Service. To build the second one we need to modify few strings in the connector code.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you then name them accordingly instead of "the second one"?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed in 6e37870.

id: prep-powerbi-odbc-amz-os-svc
run: |
sed -i 's/<value>OpenSearch Project<\/value>/<value>Amazon OpenSearch Service<\/value>/g' resources.resx
sed -i 's/Documentation.Name = "OpenSearch Project"/Documentation.Name = "Amazon OpenSearch Service"/g' SqlOdbcPBIConnector.m
sed -i 's/SqlOdbcPBIConnector\([^1-9]\)/AmazonOpenSearchConnector\1/g' SqlOdbcPBIConnector.m
working-directory: bi-connectors/PowerBIConnector
- name: Pack Power BI ODBC connector for 'Amazon OpenSearch Service'
id: pack-powerbi-odbc-amz-os-svc
run: |
zip AmazonOpenSearchService.mez *.png *.m *.resx *.pqm
working-directory: bi-connectors/PowerBIConnector
- name: Upload Tableau ODBC connector
if: steps.pack-tableau-odbc.outcome == 'success'
uses: actions/upload-artifact@v2
with:
name: TableauConnectors
path: bi-connectors/TableauConnector/opensearch_sql_odbc/opensearch_sql_odbc.taco
- name: Upload Tableau JDBC connector
if: steps.pack-tableau-jdbc.outcome == 'success'
uses: actions/upload-artifact@v2
with:
name: TableauConnectors
path: bi-connectors/TableauConnector/opensearch_sql_jdbc/opensearch_sql_jdbc.taco
- name: Upload Power BI ODBC connectors
if: steps.pack-powerbi-odbc-os-proj.outcome == 'success' || (steps.prep-powerbi-odbc-amz-os-svc.outcome == 'success' && steps.pack-powerbi-odbc-amz-os-svc.outcome == 'success')
uses: actions/upload-artifact@v2
with:
name: PBIConnectors
path: 'bi-connectors/PowerBIConnector/*.mez'
16 changes: 8 additions & 8 deletions bi-connectors/PowerBIConnector/SqlOdbcPBIConnector.pq
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ SqlOdbcPBIConnectorType = type function (
])
)
as table meta [
Documentation.Name = "OpenSearch"
Documentation.Name = "OpenSearch Project"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where are these changes here, they seem unrelated to the GitHub flows?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to Amazon's request we should build 2 connectors: one for OpenSearch Project and the second one - for Amazon OpenSearch Service. OpenSearch is not a valid name.

];

SqlOdbcPBIConnectorImpl = (Server as text) as table =>
Expand Down Expand Up @@ -159,7 +159,7 @@ OnOdbcError = (errorRecord as record) =>
else
error errorRecord;

// Data Source Kind description
// Data Source Kind description.
SqlOdbcPBIConnector = [
// Required for use with Power BI Service.
TestConnection = (dataSourcePath) =>
Expand All @@ -169,7 +169,7 @@ SqlOdbcPBIConnector = [
in
{ "SqlOdbcPBIConnector.Contents", Server },

// Authentication modes
// Authentication modes.
Authentication = [
Implicit = [
Label = "NONE"
Expand All @@ -183,7 +183,7 @@ SqlOdbcPBIConnector = [
]
],

// PBIDS Handler
// PBIDS Handler.
DSRHandlers = [
#"sqlodbc" = [
GetDSR = (server, schema, object, optional options) => [ protocol = "sqlodbc", address = [ server = server ] ],
Expand All @@ -196,13 +196,13 @@ SqlOdbcPBIConnector = [
]
],

// Enable Encryption
// Enable Encryption.
SupportsEncryption = true,

Label = Extension.LoadString("DataSourceLabel")
];

// Data Source UI publishing description
// Data Source UI publishing description.
SqlOdbcPBIConnector.Publish = [
Beta = true,
Category = "Other",
Expand All @@ -221,15 +221,15 @@ SqlOdbcPBIConnector.Icons = [
Icon32 = { Extension.Contents("SqlOdbcPBIConnector32.png"), Extension.Contents("SqlOdbcPBIConnector40.png"), Extension.Contents("SqlOdbcPBIConnector48.png"), Extension.Contents("SqlOdbcPBIConnector64.png") }
];

// Load common library functions
// Load common library functions.
Extension.LoadFunction = (name as text) =>
let
binary = Extension.Contents(name),
asText = Text.FromBinary(binary)
in
Expression.Evaluate(asText, #shared);

// Diagnostics module contains multiple functions. .
// Diagnostics module contains multiple functions.
Diagnostics = Extension.LoadFunction("Diagnostics.pqm");
Diagnostics.LogValue = if (EnableTraceOutput) then Diagnostics[LogValue] else (prefix, value) => value;

Expand Down
4 changes: 2 additions & 2 deletions bi-connectors/PowerBIConnector/resources.resx
Original file line number Diff line number Diff line change
Expand Up @@ -121,9 +121,9 @@
<value>Connect to OpenSearch</value>
</data>
<data name="ButtonTitle" xml:space="preserve">
<value>OpenSearch</value>
<value>OpenSearch Project</value>
</data>
<data name="DataSourceLabel" xml:space="preserve">
<value>OpenSearch</value>
<value>OpenSearch Project</value>
</data>
</root>