bigquery: support SQL parameters #1854

stephenplusplus · 2016-12-02T18:04:36Z

To Dos

This introduces a nice wrapper around SQL parameter queries, allowing positional and named parameters:

bigquery.query({
  query: 'SELECT * FROM `github` WHERE owner = ?',
  params: ['stephenplusplus']
}, callback);

bigquery.query({
  query: 'SELECT * FROM `github` WHERE owner = @owner',
  params: {
    owner: 'stephenplusplus'
  }
}, callback);

Helpful resources:

stephenplusplus · 2016-12-02T18:08:13Z

@blowmage shout out for your system tests, which helped me a lot while putting this together. Did you ever find a good query for structs?

packages/bigquery/src/index.js

+
+  if (type === 'TIMESTAMP') {
+    // @TODO - Not sure why .toJSON() doesn't just work.
+    value = new Date(value).toJSON().replace(/(.*)T(.*)Z$/, '$1 $2');


blowmage · 2016-12-02T19:59:54Z

@stephenplusplus Not yet. We are looking for one though.

packages/bigquery/system-test/bigquery.js

+            });
+          });
+
+          it.skip('should work with structs', function(done) {


callmehiphop · 2016-12-06T00:58:50Z

@stephenplusplus should I wait to merge for the struct stuff? Everything else looks good.

blowmage · 2016-12-06T01:21:25Z

Probably worth noting that there are more types that can be supported. Our initial implementation missed DATETIME, TIME, and BYTES. (See googleapis/google-cloud-ruby#1100)

stephenplusplus · 2016-12-06T01:26:11Z

Thanks for the heads up!

callmehiphop · 2016-12-06T11:06:55Z

@stephenplusplus do you want to get those other types in before merge?

stephenplusplus · 2016-12-06T13:41:40Z

Yeah, I'll have a PR soon for them. In the testing phase now.

stephenplusplus · 2016-12-06T15:40:28Z

Alright, ready for another look @callmehiphop

packages/bigquery/src/index.js

+ *   }
+ * }, callback);
+ *
+ * //-


packages/bigquery/system-test/bigquery.js

+            });
+          });
+
+          it.skip('should work with DATETIME types', function(done) {


packages/bigquery/src/index.js

+
+/**
+ * A `DATETIME` data type represents a point in time. Unlike a `TIMESTAMP`, a
+ * this does not refer to an absolute instance in time. Instead, it is the civil


packages/bigquery/src/index.js

+    return 'STRUCT';
+  }
+
+  return 'STRING';


packages/bigquery/src/index.js

+ * @param {object|*[]} options.params - For positional SQL parameters, provide
+ *     an array of values. For named SQL parameters, provide an object which
+ *     maps each named parameter to its value. The supported types are integers,
+ *     floats, Date objects, Strings, Booleans, and Objects.


packages/bigquery/src/index.js

@@ -450,6 +724,39 @@ BigQuery.prototype.job = function(id) {
 * });
 *
 * //-
+ * // Positional and named SQL parameters are supported.


packages/bigquery/system-test/bigquery.js

+                  27,
+                  28,
+                  29
+                ]


stephenplusplus · 2016-12-06T21:21:39Z

Changes made!

packages/bigquery/src/index.js

+ */
+BigQuery.valueToQueryParameter_ = function(value) {
+  if (is.date(value)) {
+    value = this.timestamp(value);


coveralls · 2016-12-07T04:24:04Z

Coverage remained the same at 100.0% when pulling 4616c59 on stephenplusplus:spp--1787 into ac87e18 on GoogleCloudPlatform:master.

c0b · 2016-12-13T21:52:17Z

@blowmage 11 days ago Collaborator
Not a bug AFAIK. BigQuery uses a very specific date format.
https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#canonical-format_3

is there anyway to file an issue to BigQuery design? my app use both BigQuery and Datastore, and Datastore's DATETIME apprarently accepts RFC3339 format; there is no reason BigQuery server side can't accept this ISO standard format
https://cloud.google.com/datastore/docs/reference/gql_reference#synthetic_literals

blowmage · 2016-12-14T00:09:52Z

@c0b I am unable to answer that, clients like google-cloud-node consume the API and we don't have access to the design decisions behind it. We don't have access to Google's internal bug tracker.

cc @danoscarmike

tswast · 2016-12-14T17:54:42Z

@c0b @blowmage BigQuery uses a public issue tracker. I have filed an issue for this. https://code.google.com/p/google-bigquery/issues/detail?id=842 Please "star" the issue to show your support and receive status updates.

blowmage · 2016-12-14T18:02:39Z

Great. Thanks @tswast!

jcondit · 2016-12-14T19:38:36Z

Can you be more specific about where you'd like this format to be accepted? It's already accepted for load jobs with JSON or CSV, for example.

jcondit · 2016-12-14T19:40:22Z

Ah, never mind, just noticed this is in query parameters.

packages/bigquery/src/index.js

+  value = value || new Date();
+
+  if (is.date(value)) {
+    value = value.toJSON().replace(/^(.*)T(.*)Z$/, '$1 $2');


stephenplusplus added api: bigquery Issues related to the BigQuery API. enhancement labels Dec 2, 2016

googlebot added the cla: yes This human has signed the Contributor License Agreement. label Dec 2, 2016

stephenplusplus mentioned this pull request Dec 2, 2016

bigquery: support SQL parameters #1787

Closed

stephenplusplus commented Dec 2, 2016

View reviewed changes

packages/bigquery/src/index.js

if (type === 'TIMESTAMP') {

// @TODO - Not sure why .toJSON() doesn't just work.

value = new Date(value).toJSON().replace(/(.*)T(.*)Z$/, '$1 $2');

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

stephenplusplus assigned callmehiphop Dec 5, 2016

callmehiphop reviewed Dec 6, 2016

View reviewed changes

packages/bigquery/system-test/bigquery.js

});

});

it.skip('should work with structs', function(done) {

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

stephenplusplus added 3 commits December 5, 2016 20:28

bigquery: support SQL parameters

a08fd59

add unit tests

3a56e34

lint bug

73ed4a9

impl Time & Datetime

e027f03

stephenplusplus force-pushed the spp--1787 branch from 2c4b37e to e027f03 Compare December 6, 2016 13:45

stephenplusplus added 3 commits December 6, 2016 10:22

tests

29b4ea6

code style

be7ee69

bring up coverage

ac064c2

stephenplusplus added 2 commits December 6, 2016 12:33

re-order types

4616c59

support all time types

b508f60

callmehiphop reviewed Dec 6, 2016

View reviewed changes

packages/bigquery/src/index.js

* }

* }, callback);

*

* //-

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

add a line

8b69764

callmehiphop reviewed Dec 6, 2016

View reviewed changes

review

2f20c3b

callmehiphop reviewed Dec 6, 2016

View reviewed changes

packages/bigquery/src/index.js

*/

BigQuery.valueToQueryParameter_ = function(value) {

if (is.date(value)) {

value = this.timestamp(value);

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

context

2fb0c42

callmehiphop merged commit f5a25e7 into googleapis:master Dec 6, 2016

stephenplusplus mentioned this pull request Dec 7, 2016

bigquery: don't promisify date methods & add tests #1868

Merged

c0b reviewed Mar 2, 2017

View reviewed changes

packages/bigquery/src/index.js

value = value || new Date();

if (is.date(value)) {

value = value.toJSON().replace(/^(.*)T(.*)Z$/, '$1 $2');

This comment was marked as spam.

Sign in to view

+,
+,
+
+                              ]

bigquery: support SQL parameters #1854

bigquery: support SQL parameters #1854

Conversation

stephenplusplus commented Dec 2, 2016 • edited Loading

To Dos

stephenplusplus commented Dec 2, 2016

This comment was marked as spam.

This comment was marked as spam.

blowmage commented Dec 2, 2016

This comment was marked as spam.

This comment was marked as spam.

callmehiphop commented Dec 6, 2016

blowmage commented Dec 6, 2016

stephenplusplus commented Dec 6, 2016

callmehiphop commented Dec 6, 2016

stephenplusplus commented Dec 6, 2016

stephenplusplus commented Dec 6, 2016

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

stephenplusplus commented Dec 6, 2016

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

coveralls commented Dec 7, 2016

c0b commented Dec 13, 2016

blowmage commented Dec 14, 2016

tswast commented Dec 14, 2016

blowmage commented Dec 14, 2016

jcondit commented Dec 14, 2016

jcondit commented Dec 14, 2016

This comment was marked as spam.

stephenplusplus commented Dec 2, 2016 •

edited

Loading