Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

impl(bigtable): BulkMutator keeps pending mutations' last status #9688

Merged
merged 3 commits into from
Aug 18, 2022

Conversation

dbolduc
Copy link
Member

@dbolduc dbolduc commented Aug 15, 2022

Part of the work for #7479

In order to report the proper errors for bulk mutations that end on OK streams, the BulkMutator needs to keep track of the last known status for a mutation it may or may not retry. There is always one copy translating from grpc::Status -> google::cloud::Status. std::move() the Status from then on.

In the case where a mutation fails in a stream that also fails, I return the mutation error. My guess is that it will be more informative. But we accept either in the unit test.

Note that we retry all OK streams at the moment. This will be fixed in either 1, 2, or 4 follow up PRs. (I haven't decided)


This change is Reviewable

@product-auto-label product-auto-label bot added the api: bigtable Issues related to the Bigtable API. label Aug 15, 2022
@google-cloud-cpp-bot
Copy link
Collaborator

Google Cloud Build Logs
For commit: 47a20dad373d3f91882aef68e5e3a70d6a525c17

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

@codecov
Copy link

codecov bot commented Aug 15, 2022

Codecov Report

Merging #9688 (e8f2221) into main (1e7bbb9) will decrease coverage by 0.00%.
The diff coverage is 96.00%.

@@            Coverage Diff             @@
##             main    #9688      +/-   ##
==========================================
- Coverage   93.89%   93.89%   -0.01%     
==========================================
  Files        1496     1496              
  Lines      139303   139384      +81     
==========================================
+ Hits       130803   130876      +73     
- Misses       8500     8508       +8     
Impacted Files Coverage Δ
google/cloud/bigtable/internal/bulk_mutator.h 100.00% <ø> (ø)
...oogle/cloud/bigtable/internal/bulk_mutator_test.cc 94.68% <95.00%> (+0.11%) ⬆️
google/cloud/bigtable/internal/bulk_mutator.cc 99.13% <100.00%> (+0.88%) ⬆️
google/cloud/internal/async_rpc_details.h 95.00% <0.00%> (-5.00%) ⬇️
...bigtable/examples/bigtable_hello_instance_admin.cc 81.00% <0.00%> (-2.00%) ⬇️
...cloud/pubsub/internal/subscription_session_test.cc 98.15% <0.00%> (-0.68%) ⬇️
google/cloud/bigtable/async_read_stream_test.cc 97.32% <0.00%> (-0.67%) ⬇️
...le/cloud/internal/default_completion_queue_impl.cc 96.59% <0.00%> (-0.57%) ⬇️
google/cloud/storage/parallel_upload.cc 98.26% <0.00%> (-0.35%) ⬇️
google/cloud/pubsub/samples/samples.cc 90.77% <0.00%> (+0.07%) ⬆️
... and 3 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@coryan coryan added kokoro:force-run Add this label to force Kokoro to re-run the tests. and removed kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Aug 15, 2022
@dbolduc dbolduc marked this pull request as ready for review August 15, 2022 17:52
@dbolduc dbolduc requested a review from a team as a code owner August 15, 2022 17:52
@dbolduc dbolduc added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Aug 16, 2022
@google-cloud-cpp-bot google-cloud-cpp-bot removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Aug 16, 2022
pending_annotations_.push_back(annotation);
pending_annotations_.push_back(Annotations{
annotation.original_index, annotation.idempotency,
annotation.has_mutation_result, std::move(annotation.status)});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What state do these moves leave annotations_[i].status in?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I realize now that I should have left annotation.status as a local temporary status. That is how I am using it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. hopefully it is a little more clear where the annotation.status comes from (pending_mutations_.push_back(...)) and when the annotation.status matters (reporting fails in OnRetryDone())

@dbolduc dbolduc force-pushed the bulk-mutator-ok-streams branch from 47a20da to e0009b3 Compare August 17, 2022 20:59
@google-cloud-cpp-bot
Copy link
Collaborator

Google Cloud Build Logs
For commit: e0009b3e426c45aa93d5f9cae774416947b5f134

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

* status, we return a `FailedMutation` made from `original_index` and
* `status`. The value is meaningless if `has_mutation_result` is false.
*/
Status const status;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The const makes the struct unassignable, which is weird, and (I'd say) isn't worth it. Should other fields be const too, for example.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

has_mutation_result can't be const. The other fields could be. I will take your suggestion and drop the const here.

@google-cloud-cpp-bot
Copy link
Collaborator

Google Cloud Build Logs
For commit: e8f2221f172bb4a15a92b0773ed3a2fb54ee9518

ℹ️ NOTE: Kokoro logs are linked from "Details" below.

@dbolduc dbolduc enabled auto-merge (squash) August 18, 2022 16:44
@dbolduc dbolduc merged commit f174c62 into googleapis:main Aug 18, 2022
@dbolduc dbolduc deleted the bulk-mutator-ok-streams branch August 18, 2022 17:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigtable Issues related to the Bigtable API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants