Address msft feedback (#380)

* Added Tableau Connector to OpenSearch SQL

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Added CEILING and FLOOR functions

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Added IFNULL function

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Added Kyle's fix for ADDDATE and SUBDATE with resulting 00:00:00 being unexpectedly null

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Added MIN and MAX for two string arguments

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Fixed return type for MID function redefenitions, part of AOS-202

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Added CAST functions to convert to int or string as part of AOS-202

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Added HEXBINX and HEXBINY functions from the templace without any modifications

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Fixed cast to use the OSSQL type

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Added parenthesis to avoid /issues/293

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Modified Timestamp conversion formula

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Added comment to TimestampType

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Updated Company Name accroding to the customer's request

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Updated Vendor Name as well

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Added few small changes to simplify connector developing and debugging

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Typo fix

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Added a simple test from TDVT: case bool0 from test calcs_data (an expression test)
Some mofidication were made in the test framework to make the output more readable

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Modified the driver to recognize its support for conversion of types

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Removed changes from a different branch

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Removed changes from a different branch

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Made the connector convert keyword columns to string columns

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Fix requested by PR review

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Modified integration tests to reflect support for conversion types

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Removed connector changes

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Change values to bitmasks

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Add timestamp as a type for the driver

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Added timestamp to SQLGetTypeInfo for the ODBC Driver

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Reverted some previous changes

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Removed generated test output file

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Renamed SqlOdbcPBIConnector to OpenSearch

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* According to AOS-248, we have to build the server address string properly:
- add http[s]:// prefix
- add port
In scope of this task 2 new fields were added to the connection dialog (port, useSSL) and the corresponding handling.

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Added version and disable trace output as per Microsoft's feedback

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Added documentation for the OpenSearch connector

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Renamed connector markdown file to follow naming conventions

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Changed image file extention as it was not being detected by GitHub

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Updating docs according to PR #16 comments.

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Added GitHub actions script to build Tableau and Power BI connector. The PBI connector is being built twice with different names accroding to the Amazon's request.

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* A typo fix in yml

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* A dummy commit to make GitHub Actions start the desired workflow

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Workflow fix - pack Tableau ODBC connector instead of JDBC one

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Fix line endings in the workflow file

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Another dummy commit to start the desired workflow

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Fix paths in the GitHub workflow

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Modified workflow to trigger on changes of itself

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Path fix for preparation step

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Added some changes to the second PBI connector - they should differ more than in filename to make PBI distuingish them

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Renamed PBI connector files

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Moved Tableau JDBC connector to bi-connectors

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Added packing Tableau JDBC connector
For all connectors added check for successful packing before publishing

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Renamed Power BI connector as "OpenSearch Project"

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Renamed output artifact

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Update SqlOdbcPBIConnector.pq according to PR review notes

Co-authored-by: Kyle Porter <kylep@bitquilltech.com>

* Update SqlOdbcPBIConnector.pq according to PR review notes

Co-authored-by: Kyle Porter <kylep@bitquilltech.com>

* Modified code that builds the server string

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Fixed grammatical errors, added a docs directory, and updated the connection string options image

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Removed md file as it was moved to the docs directory

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Fixed grammatical errors and updated the old power bi support document

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Removed extra line

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Disabled building Tableau ODBC connector, because it is not ready for shipping

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Updated workflow step description

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Removed packing ODBC driver for Tableau (re: 4cadfd1).
Removed Tableau ODBC driver source files since the driver is not supposed to be.

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Updated workflow, because connector source files were renamed after merge

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Revert "Add GitHub workflow to pack BI connectors"

* Changed string replacement in the main connector file

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Script was modified according to PR comments

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Added limitations and issues table to connector markdown file

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Removed Tableau ODBC connector

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Merged OpenSearch-sql main to this branch and updated links as per the PR comment

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Added link to bug

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Made changes as per requested in the PR comments. Linked github issues to issues listed

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Added changes to the OpenSearch md file

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Modified the connector and workflows to address Power BI certification feedback

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Changed m filename to AmazonOpenSearchService-ODBC

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Made further changes to address PBI feedback

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Fixed github workflow

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Modified output filenames of connector workflow

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Fix workflow

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Typo fix in the fix

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

* Changed function prefixes to match connector name

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Changed DSR Handler Friendly name for Amazon OpenSearch Service connector as requested by the PBI feedback

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Removed -ODBC suffix as per code review comments

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Made workflow more concise and changed a variable name

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

* Removed irrelevant images

Signed-off-by: Guian Gumpac <guiang@bitquilltech.com>

Co-authored-by: Yury Fridlyand <yuryf@bitquilltech.com>
Co-authored-by: Yury-Fridlyand <88679692+Yury-Fridlyand@users.noreply.github.com>
Co-authored-by: Kyle Porter <kylep@bitquilltech.com>

.github/workflows/bi-connectors.yml

@@ -11,15 +11,15 @@ jobs:
   build:
     runs-on: ubuntu-latest
     steps:
-    - uses: actions/checkout@v1
+    - uses: actions/checkout@v2
     - name: Pack Tableau JDBC connector
       id: pack-tableau-jdbc
       run: |
         zip -r opensearch_sql_jdbc.taco . -x *.taco
       working-directory: bi-connectors/TableauConnector/opensearch_sql_jdbc
-    - name: Prepare Power BI ODBC connector
+    - name: Prepare Power BI ODBC connector for 'OpenSearch Project'
       run: |
-        cp OpenSearch.pq OpenSearch.m
+        cp OpenSearchProject.pq OpenSearchProject.m
       working-directory: bi-connectors/PowerBIConnector
     - name: Pack Power BI ODBC connector
       id: pack-powerbi-odbc-os-proj
@@ -29,17 +29,11 @@ jobs:
     - name: Prepare Power BI ODBC connector for 'Amazon OpenSearch Service'
       id: prep-powerbi-odbc-amz-os-svc
       run: |
+        mv OpenSearchProject.m AmazonOpenSearchService.m
         sed -i 's/<value>OpenSearch Project<\/value>/<value>Amazon OpenSearch Service<\/value>/g' resources.resx
-        sed -i 's/Documentation.Name = "OpenSearch Project"/Documentation.Name = "Amazon OpenSearch Service"/g' OpenSearch.m
-        sed -i 's/section OpenSearch;/section AmazonOpenSearchConnector;/' OpenSearch.m
-        sed -i 's/\[DataSource.Kind="OpenSearch", Publish="OpenSearch.Publish"\]/[DataSource.Kind="AmazonOpenSearchConnector", Publish="AmazonOpenSearchConnector.Publish"]/g' OpenSearch.m
-        sed -i 's/OpenSearch.Contents/AmazonOpenSearchConnector.Contents/g' OpenSearch.m
-        sed -i 's/OpenSearch.Publish/AmazonOpenSearchConnector.Publish/g' OpenSearch.m
-        sed -i 's/OpenSearch.Icons/AmazonOpenSearchConnector.Icons/g' OpenSearch.m
-        sed -i 's/OpenSearchImpl/AmazonOpenSearchConnectorImpl/g' OpenSearch.m
-        sed -i 's/OpenSearchType/AmazonOpenSearchConnectorType/g' OpenSearch.m
-        sed -i 's/OpenSearch = \[/AmazonOpenSearchConnector = [/' OpenSearch.m
-#        sed -i 's/"opensearch-sql"/"Amazon-OpenSearch-Service"/g' OpenSearch.m
+        sed -i 's/OpenSearch Project/Amazon OpenSearch Service/g' AmazonOpenSearchService.m
+        sed -i 's/OpenSearchProject/AmazonOpenSearchService/g' AmazonOpenSearchService.m
+        sed -i 's/opensearchproject/amazonopensearchservice/g' AmazonOpenSearchService.m
       working-directory: bi-connectors/PowerBIConnector
     - name: Pack Power BI ODBC connector for 'Amazon OpenSearch Service'
       id: pack-powerbi-odbc-amz-os-svc

bi-connectors/PowerBIConnector/OpenSearch.mproj → bi-connectors/PowerBIConnector/OpenSearchProject.mproj

@@ -36,7 +36,7 @@
     <Reference Include="System.Xml" />
   </ItemGroup>
   <ItemGroup>
-    <Compile Include="OpenSearch.pq">
+    <Compile Include="OpenSearchProject.pq">
       <SubType>Code</SubType>
     </Compile>
     <Compile Include="OpenSearch16.png">
@@ -72,7 +72,7 @@
     <Compile Include="OdbcConstants.pqm">
       <SubType>Content</SubType>
     </Compile>
-    <Content Include="OpenSearch.query.pq">
+    <Content Include="OpenSearchProject.query.pq">
       <SubType>Code</SubType>
     </Content>
   </ItemGroup>

bi-connectors/PowerBIConnector/OpenSearch.pq → bi-connectors/PowerBIConnector/OpenSearchProject.pq

@@ -1,18 +1,18 @@
 // This file contains Data Connector logic
 [Version = "1.0.0"]
-section OpenSearch;
+section OpenSearchProject;
 
 // When set to true, additional trace information will be written out to the User log. 
 // This should be set to false before release. Tracing is done through a call to 
 // Diagnostics.LogValue(). When EnableTraceOutput is set to false, the call becomes a 
 // no-op and simply returns the original value.
 EnableTraceOutput = false;
 
-[DataSource.Kind="OpenSearch", Publish="OpenSearch.Publish"]
-shared OpenSearch.Contents = Value.ReplaceType(OpenSearchImpl, OpenSearchType);
+[DataSource.Kind="OpenSearchProject", Publish="OpenSearchProject.Publish"]
+shared OpenSearchProject.Contents = Value.ReplaceType(OpenSearchProjectImpl, OpenSearchProjectType);
 
 // Wrapper function to provide additional UI customization.
-OpenSearchType = type function (
+OpenSearchProjectType = type function (
         Server as (type text meta [
             Documentation.FieldCaption = "Server",
             Documentation.FieldDescription = "The hostname of the OpenSearch server.",
@@ -30,10 +30,10 @@ OpenSearchType = type function (
         ])
     )
     as table meta [
-        Documentation.Name = "OpenSearch"
+        Documentation.Name = "OpenSearch Project"
     ];
 
-OpenSearchImpl = (Server as text, Port as number, UseSSL as logical) as table =>
+OpenSearchProjectImpl = (Server as text, Port as number, UseSSL as logical) as table =>
     let
         Credential = Extension.CurrentCredential(),
         AuthenticationMode = Credential[AuthenticationKind],
@@ -149,7 +149,7 @@ OpenSearchImpl = (Server as text, Port as number, UseSSL as logical) as table =>
             CredentialConnectionString = EncryptedConnectionString
         ],
 
-        FullConnectionString = Diagnostics.LogValue("Connection String", ConnectionString & CredentialConnectionString & EncryptedConnectionString),
+        FullConnectionString = (ConnectionString & CredentialConnectionString & EncryptedConnectionString),
 
         OdbcDatasource = Odbc.DataSource(FullConnectionString, OdbcOptions)
     in
@@ -171,14 +171,14 @@ OnOdbcError = (errorRecord as record) =>
             OdbcErrorCode = 202
     in
         if IsDriverNotInstalled then
-            error Error.Record("", "The OpenSearch SQL ODBC driver is not installed. Please install the driver")
+            error Error.Record("DataSource.Error", "The OpenSearch SQL ODBC driver is not installed. Please install the driver")
         else if IsHostUnreachable then 
-            error Error.Record("", "Couldn't reach server. Please double-check the server and auth. [" & ConnectionServer & "]")
+            error Error.Record("DataSource.Error", "Couldn't reach server. Please double-check the server and auth. [" & ConnectionServer & "]")
         else 
             error errorRecord;
 
 // Data Source Kind description
-OpenSearch = [
+OpenSearchProject = [
     // Required for use with Power BI Service.
     TestConnection = (dataSourcePath) => 
         let
@@ -187,7 +187,7 @@ OpenSearch = [
             Port = json[Port],
             UseSSL = json[UseSSL]
         in
-            { "OpenSearch.Contents", Server, Port, UseSSL },
+            { "OpenSearchProject.Contents", Server, Port, UseSSL },
 
     // Authentication modes
     Authentication = [
@@ -205,14 +205,14 @@ OpenSearch = [
 
     // PBIDS Handler
     DSRHandlers = [
-      #"opensearch-sql" = [
-        GetDSR = (server, schema, object, optional options) => [ protocol = "opensearch-sql", address = [ server = server, port = schema, useSSL = object ] ],
+      opensearchproject = [
+        GetDSR = (server, schema, object, optional options) => [ protocol = "opensearchproject-odbc", address = [ server = server, port = schema, useSSL = object ] ],
         GetFormula = (dsr, optional options) => () =>
           let
-            db = OpenSearch.Contents(dsr[address][server], dsr[address][port], dsr[address][useSSL])
+            db = OpenSearchProject.Contents(dsr[address][server], dsr[address][port], dsr[address][useSSL])
           in
             db,
-        GetFriendlyName = (dsr) => "OpenSearch SQL ODBC"
+        GetFriendlyName = (dsr) => "OpenSearch Project"
       ]
     ],
 
@@ -223,13 +223,12 @@ OpenSearch = [
 ];
 
 // Data Source UI publishing description
-OpenSearch.Publish = [
+OpenSearchProject.Publish = [
     Beta = true,
     Category = "Other",
     ButtonText = { Extension.LoadString("ButtonTitle"), Extension.LoadString("ButtonHelp") },
-    LearnMoreUrl = "https://www.opensearch.org/",
+    LearnMoreUrl = "https://github.com/opensearch-project/sql/blob/main/bi-connectors/PowerBIConnector/docs/OpenSearchProject.md",
 
-    // Disabling direct query due to limited SQL query support
     SupportsDirectQuery = true,
 
     SourceImage = OpenSearch.Icons,