Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: pull in changes from main branch #1379

Merged
merged 57 commits into from
Sep 7, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
88bf067
chore(deps): update dependency com.google.cloud:libraries-bom to v26 …
renovate-bot Jul 12, 2022
c7539f9
build: enable longpaths support for windows test (#1485) (#1310)
gcf-owl-bot[bot] Jul 13, 2022
de35414
build(deps): update dependency org.codehaus.mojo:extra-enforcer-rules…
renovate-bot Jul 13, 2022
ddae354
deps: update dependency org.graalvm.buildtools:junit-platform-native …
renovate-bot Jul 13, 2022
c0740fe
deps: update dependency org.graalvm.buildtools:native-maven-plugin to…
renovate-bot Jul 13, 2022
5ab424c
test(deps): update dependency com.google.cloud:google-cloud-conforman…
renovate-bot Jul 13, 2022
7c77879
fix: enable integration test for google-cloud-bigtable-stats (#1311)
mutianf Jul 15, 2022
85ff18c
chore: mark native image checks as required (#1313)
mpeddada1 Jul 18, 2022
0f9854b
test(deps): update dependency com.google.cloud:google-cloud-conforman…
renovate-bot Jul 18, 2022
2aa490c
ci: update template so GAPIC_AUTO repos do not require special approv…
gcf-owl-bot[bot] Jul 19, 2022
6629821
feat: use PingAndWarm request for channel priming (#1179)
mutianf Jul 19, 2022
d182bec
build(deps): update dependency org.apache.maven.plugins:maven-deploy-…
renovate-bot Jul 19, 2022
5282589
feat: add storage utilization gib per node for autoscaling (#1317)
kolea2 Jul 22, 2022
644454a
fix: fix race condition in BuiltinMetricsTracer (#1320)
mutianf Jul 26, 2022
57fe707
chore(main): release 2.10.0 (#1302)
release-please[bot] Jul 26, 2022
7655747
deps: update dependency org.junit.vintage:junit-vintage-engine to v5.…
renovate-bot Jul 26, 2022
30e7b90
chore(main): release 2.10.1-SNAPSHOT (#1321)
release-please[bot] Jul 26, 2022
1a5b3a2
fix: retry rst stream in mutations (#1327)
mutianf Jul 29, 2022
0a44123
chore(deps): update dependency com.google.cloud:google-cloud-bigtable…
renovate-bot Aug 1, 2022
4fdf37e
chore(main): release 2.10.1 (#1325)
release-please[bot] Aug 1, 2022
eca3969
build(deps): update dependency org.codehaus.mojo:extra-enforcer-rules…
renovate-bot Aug 1, 2022
fdabaaf
chore(main): release 2.10.2-SNAPSHOT (#1331)
release-please[bot] Aug 2, 2022
644aeb3
chore(deps): update dependency com.google.cloud:google-cloud-bigtable…
renovate-bot Aug 2, 2022
cb539b5
fix: add a ReadFirstRow callable to set future in onComplete (#1326)
mutianf Aug 2, 2022
98b3349
deps: upgrade shared config to 1.5.3, exclude google-http-client and …
blakeli0 Aug 2, 2022
c4b8c03
fix: The metadata could be returned in trailer or header depends on i…
mutianf Aug 3, 2022
bee0ca0
deps: update dependency com.google.cloud:google-cloud-shared-dependen…
renovate-bot Aug 3, 2022
2313980
chore(main): release 2.10.2 (#1335)
release-please[bot] Aug 8, 2022
f82104a
chore(main): release 2.10.3-SNAPSHOT (#1340)
release-please[bot] Aug 8, 2022
8071de6
fix: declaring 2 http libraries as runtime (#1341)
suztomo Aug 8, 2022
dca1535
chore(deps): update dependency com.google.cloud:google-cloud-bigtable…
renovate-bot Aug 10, 2022
a283659
chore(main): release 2.10.3 (#1343)
release-please[bot] Aug 11, 2022
6ff3970
chore(main): release 2.10.4-SNAPSHOT (#1348)
release-please[bot] Aug 11, 2022
b865578
chore(deps): update dependency com.google.cloud:google-cloud-bigtable…
renovate-bot Aug 11, 2022
7ce915e
feat: add stackdriver exporter (#1247)
mutianf Aug 12, 2022
d315473
chore(deps): update dependency com.google.cloud:libraries-bom to v26.…
renovate-bot Aug 12, 2022
8d353a2
chore: add opencensus lincese and update readme (#1353)
mutianf Aug 15, 2022
f399240
chore: add instructions to enable builtin metrics (#1358)
mutianf Aug 16, 2022
f8d97e5
deps: update dependency com.google.cloud:google-cloud-monitoring-bom …
renovate-bot Aug 16, 2022
cb2ff66
test(deps): update dependency org.mockito:mockito-core to v4.7.0 (#1356)
renovate-bot Aug 16, 2022
dcfd512
test(deps): update dependency com.google.cloud:google-cloud-conforman…
renovate-bot Aug 16, 2022
c7ffd6b
build(deps): update dependency org.apache.maven.plugins:maven-javadoc…
renovate-bot Aug 17, 2022
8c3edb0
chore(main): release 2.11.0 (#1350)
release-please[bot] Aug 22, 2022
f81f1a5
chore(main): release 2.11.1-SNAPSHOT (#1361)
release-please[bot] Aug 22, 2022
1ac859b
chore(deps): update dependency com.google.cloud:google-cloud-bigtable…
renovate-bot Aug 23, 2022
c039a83
test: add integration test for builtin metrics (#1360)
mutianf Aug 25, 2022
faa5b36
build(deps): update dependency org.apache.maven.shared:maven-dependen…
renovate-bot Aug 25, 2022
1683365
fix: reset a measure map everytime the stats are recorded (#1364)
mutianf Aug 26, 2022
d1b269f
chore(main): release 2.11.1 (#1365)
release-please[bot] Aug 29, 2022
6f4c01d
chore(main): release 2.11.2-SNAPSHOT (#1366)
release-please[bot] Aug 29, 2022
54bf5f5
chore(deps): update dependency com.google.cloud:google-cloud-bigtable…
renovate-bot Aug 29, 2022
d29fada
chore(deps): update dependency com.google.cloud:libraries-bom to v26.…
renovate-bot Aug 31, 2022
930d043
fix: make cloud-monitoring a runtime dependency (#1371)
mutianf Aug 31, 2022
72a7387
test: disable integration test (#1375)
mutianf Sep 1, 2022
4174f0d
deps: update dependency com.google.cloud:google-cloud-monitoring-bom …
renovate-bot Sep 2, 2022
b3cc7f0
test: fix metrics integration test, remove the server latency since t…
mutianf Sep 2, 2022
944f4d0
Merge remote-tracking branch 'origin/main' into cdc_merge
Sep 7, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix: add a ReadFirstRow callable to set future in onComplete (#1326)
* fix: add a ReadFirstRow callable to set future in onComplete

* use ReadRowsFirst callable instead

* don't use atomic

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
  • Loading branch information
mutianf and gcf-owl-bot[bot] authored Aug 2, 2022
commit cb539b50d98ec2a8538ce4691b2639426ca95464
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@
import com.google.cloud.bigtable.data.v2.stub.mutaterows.MutateRowsRetryingCallable;
import com.google.cloud.bigtable.data.v2.stub.readrows.FilterMarkerRowsCallable;
import com.google.cloud.bigtable.data.v2.stub.readrows.ReadRowsBatchingDescriptor;
import com.google.cloud.bigtable.data.v2.stub.readrows.ReadRowsFirstCallable;
import com.google.cloud.bigtable.data.v2.stub.readrows.ReadRowsResumptionStrategy;
import com.google.cloud.bigtable.data.v2.stub.readrows.ReadRowsRetryCompletedCallable;
import com.google.cloud.bigtable.data.v2.stub.readrows.ReadRowsUserCallable;
Expand Down Expand Up @@ -366,10 +367,16 @@ public <RowT> UnaryCallable<Query, RowT> createReadRowCallable(RowAdapter<RowT>
.build(),
rowAdapter);

UnaryCallable<Query, RowT> readRowCallable =
new ReadRowsUserCallable<>(readRowsCallable, requestContext).first();
ReadRowsUserCallable<RowT> readRowCallable =
new ReadRowsUserCallable<>(readRowsCallable, requestContext);

ServerStreamingCallable<Query, RowT> traced =
new TracedServerStreamingCallable<>(
readRowCallable, clientContext.getTracerFactory(), getSpanName("ReadRow"));

ReadRowsFirstCallable<RowT> firstRow = new ReadRowsFirstCallable<>(traced);

return createUserFacingUnaryCallable("ReadRow", readRowCallable);
return firstRow.withDefaultCallContext(clientContext.getDefaultCallContext());
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,23 +16,64 @@
package com.google.cloud.bigtable.data.v2.stub.readrows;

import com.google.api.core.ApiFuture;
import com.google.api.core.InternalApi;
import com.google.api.core.SettableApiFuture;
import com.google.api.gax.rpc.ApiCallContext;
import com.google.api.gax.rpc.ServerStreamingCallable;
import com.google.api.gax.rpc.StateCheckingResponseObserver;
import com.google.api.gax.rpc.StreamController;
import com.google.api.gax.rpc.UnaryCallable;
import com.google.cloud.bigtable.data.v2.models.Query;

/**
* Enhancement for `readRowsCallable().first()` to gracefully limit the row count instead of
* cancelling the RPC
*/
class ReadRowsFirstCallable<RowT> extends UnaryCallable<Query, RowT> {
private final UnaryCallable<Query, RowT> inner;
@InternalApi
public class ReadRowsFirstCallable<RowT> extends UnaryCallable<Query, RowT> {

ReadRowsFirstCallable(UnaryCallable<Query, RowT> inner) {
private final ServerStreamingCallable<Query, RowT> inner;

public ReadRowsFirstCallable(ServerStreamingCallable<Query, RowT> inner) {
this.inner = inner;
}

@Override
public ApiFuture<RowT> futureCall(Query query, ApiCallContext context) {
return inner.futureCall(query.limit(1), context);
ReadRowsFirstResponseObserver<RowT> observer = new ReadRowsFirstResponseObserver<>();
this.inner.call(query.limit(1), observer, context);
return observer.getFuture();
}

private class ReadRowsFirstResponseObserver<RowT> extends StateCheckingResponseObserver<RowT> {
private StreamController innerController;
private RowT firstRow;
private SettableApiFuture<RowT> settableFuture = SettableApiFuture.create();

@Override
protected void onStartImpl(StreamController streamController) {
this.innerController = streamController;
}

@Override
protected void onResponseImpl(RowT response) {
if (firstRow == null) {
this.firstRow = response;
}
}

@Override
protected void onErrorImpl(Throwable throwable) {
settableFuture.setException(throwable);
}

@Override
protected void onCompleteImpl() {
settableFuture.set(firstRow);
}

protected ApiFuture<RowT> getFuture() {
return settableFuture;
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@
import com.google.api.gax.rpc.ApiCallContext;
import com.google.api.gax.rpc.ResponseObserver;
import com.google.api.gax.rpc.ServerStreamingCallable;
import com.google.api.gax.rpc.UnaryCallable;
import com.google.bigtable.v2.ReadRowsRequest;
import com.google.cloud.bigtable.data.v2.internal.RequestContext;
import com.google.cloud.bigtable.data.v2.models.Query;
Expand All @@ -34,27 +33,16 @@
public class ReadRowsUserCallable<RowT> extends ServerStreamingCallable<Query, RowT> {
private final ServerStreamingCallable<ReadRowsRequest, RowT> inner;
private final RequestContext requestContext;
private final ReadRowsFirstCallable<RowT> firstCallable;

public ReadRowsUserCallable(
ServerStreamingCallable<ReadRowsRequest, RowT> inner, RequestContext requestContext) {
this.inner = inner;
this.requestContext = requestContext;

this.firstCallable = new ReadRowsFirstCallable<>(super.first());
}

@Override
public void call(Query request, ResponseObserver<RowT> responseObserver, ApiCallContext context) {
ReadRowsRequest innerRequest = request.toProto(requestContext);
inner.call(innerRequest, responseObserver, context);
}

// Optimization: since the server supports row limits, override the first callable.
// This way unnecessary data doesn't need to be buffered and the number of CANCELLED request
// statuses is minimized
@Override
public UnaryCallable<Query, RowT> first() {
return firstCallable;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,13 @@
*/
package com.google.cloud.bigtable.data.v2.stub.readrows;

import com.google.api.core.SettableApiFuture;
import com.google.api.gax.rpc.UnaryCallable;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.verify;

import com.google.api.gax.grpc.GrpcCallContext;
import com.google.api.gax.rpc.ApiCallContext;
import com.google.api.gax.rpc.ResponseObserver;
import com.google.api.gax.rpc.ServerStreamingCallable;
import com.google.cloud.bigtable.data.v2.internal.RequestContext;
import com.google.cloud.bigtable.data.v2.models.Query;
import com.google.cloud.bigtable.data.v2.models.Row;
Expand All @@ -38,36 +43,33 @@ public class ReadRowsFirstCallableTest {

private static final RequestContext REQUEST_CONTEXT =
RequestContext.create("fake-project", "fake-instance", "fake-profile");
private UnaryCallable<Query, Row> innerCallable;

private ServerStreamingCallable<Query, Row> innerCallable;
private ArgumentCaptor<Query> innerQuery;
private SettableApiFuture<Row> innerResult;

@SuppressWarnings("unchecked")
@Before
public void setUp() {
innerCallable = Mockito.mock(UnaryCallable.class);
innerCallable = Mockito.mock(ServerStreamingCallable.class);
innerQuery = ArgumentCaptor.forClass(Query.class);
innerResult = SettableApiFuture.create();
Mockito.when(innerCallable.futureCall(innerQuery.capture(), Mockito.any()))
.thenReturn(innerResult);
}

@Test
public void testLimitAdded() {
ReadRowsFirstCallable<Row> callable = new ReadRowsFirstCallable<>(innerCallable);
innerResult.set(null);
callable.call(Query.create("fake-table"));

callable.futureCall(Query.create("fake-table"), GrpcCallContext.createDefault());
verify(innerCallable)
.call(innerQuery.capture(), any(ResponseObserver.class), any(ApiCallContext.class));
Truth.assertThat(innerQuery.getValue().toProto(REQUEST_CONTEXT))
.isEqualTo(Query.create("fake-table").limit(1).toProto(REQUEST_CONTEXT));
}

@Test
public void testLimitChanged() {
ReadRowsFirstCallable<Row> callable = new ReadRowsFirstCallable<>(innerCallable);
innerResult.set(null);
callable.call(Query.create("fake-table").limit(1_000));

callable.futureCall(Query.create("fake-table").limit(10), GrpcCallContext.createDefault());
verify(innerCallable)
.call(innerQuery.capture(), any(ResponseObserver.class), any(ApiCallContext.class));
Truth.assertThat(innerQuery.getValue().toProto(REQUEST_CONTEXT))
.isEqualTo(Query.create("fake-table").limit(1).toProto(REQUEST_CONTEXT));
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,17 +41,4 @@ public void testRequestConverted() {

Truth.assertThat(innerCallable.getActualRequest()).isEqualTo(query.toProto(REQUEST_CONTEXT));
}

@Test
public void testFirstIsLimited() {
ServerStreamingStashCallable<ReadRowsRequest, Row> innerCallable =
new ServerStreamingStashCallable<>();
ReadRowsUserCallable<Row> callable = new ReadRowsUserCallable<>(innerCallable, REQUEST_CONTEXT);
Query query = Query.create("fake-table");

callable.first().call(query);

Truth.assertThat(innerCallable.getActualRequest())
.isEqualTo(query.limit(1).toProto(REQUEST_CONTEXT));
}
}