Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch (and Transaction) dataset_id should match that of Key / Query objects when added #447

Closed
dhermes opened this issue Dec 19, 2014 · 5 comments · Fixed by #552
Closed
Assignees
Labels
api: datastore Issues related to the Datastore API. 🚨 This issue needs some love. triage me I really want to be triaged.

Comments

@dhermes
Copy link
Contributor

dhermes commented Dec 19, 2014

Proposed fix:

Do not ask for dataset_id in the constructor and instead gather it from the keys and queries used in the transaction.

This may require implementing #495 and will require removing the self.begin() call from Transaction.__enter__. (Can't begin until a query or entity is encountered.)

Title was: "Transaction does not rollback correctly when multiple datasets involved"


Witness the following code:

>>> from gcloud import datastore
>>> cnxn = datastore.get_connection()
>>> dataset1 = cnxn.dataset('PROJ1')
>>> dataset2 = cnxn.dataset('PROJ2')
>>> e1 = dataset1.entity('foo')
>>> e2 = dataset2.entity('foo')
>>> e1['bar'] = e2['bar'] = 0
>>> e1.save()
<Entity[{'kind': u'foo', 'id': 5686683802533888L}] {'bar': 0}>
>>> e2.save()
<Entity[{'kind': u'foo', 'id': 5659313586569216L}] {'bar': 0}>
>>> with dataset1.transaction():
...     e1.delete()
...     e2.delete()
... 
>>> e1_still = dataset1.get_entity(e1.key())
>>> e1_still is None
True
>>> e2_still = dataset2.get_entity(e2.key())
>>> e2_still
<Entity[{'kind': u'foo', 'id': 5659313586569216L}] {u'bar': 0L}>
@dhermes dhermes added the api: datastore Issues related to the Datastore API. label Dec 19, 2014
@dhermes
Copy link
Contributor Author

dhermes commented Dec 19, 2014

UPDATE: This is because the transaction ignores the dataset associated with e2. To see this, we use the key

<Key[{'kind': 'foo', 'id': 2}]>

in two different datasets and delete the one from dataset2 in the transaction for dataset1. After doing this, we check if that key still exists in dataset1 (it should) and dataset2 (it should, but because the transaction failed since two different datasets were used).

>>> from gcloud import datastore
>>> cnxn = datastore.get_connection()
>>> dataset1 = cnxn.dataset('PROJ1')
>>> dataset2 = cnxn.dataset('PROJ2')
>>> 
>>> e1_1 = dataset1.entity('foo')
>>> e1_1.key(e1_1.key().id(1))
<Entity[{'kind': 'foo', 'id': 1}] {}>
>>> e1_2 = dataset1.entity('foo')
>>> e1_2.key(e1_2.key().id(2))
<Entity[{'kind': 'foo', 'id': 2}] {}>
>>> 
>>> e2_2 = dataset2.entity('foo')
>>> e2_2.key(e2_2.key().id(2))
<Entity[{'kind': 'foo', 'id': 2}] {}>
>>> 
>>> e1_1['bar'] = e1_2['bar'] = e2_2['bar'] = 0
>>> 
>>> e1_1.save()
<Entity[{'kind': 'foo', 'id': 1}] {'bar': 0}>
>>> e1_2.save()
<Entity[{'kind': 'foo', 'id': 2}] {'bar': 0}>
>>> e2_2.save()
<Entity[{'kind': 'foo', 'id': 2}] {'bar': 0}>
>>> 
>>> with dataset1.transaction():
...     e1_1.delete()
...     e2_2.delete()
... 
>>> e1_1_still = dataset1.get_entity(e1_1.key())
>>> e1_1_still is None
True
>>> e1_2_still = dataset1.get_entity(e1_2.key())
>>> e1_2_still is None
True
>>> e2_2_still = dataset2.get_entity(e2_2.key())
>>> e2_2_still
<Entity[{'kind': u'foo', 'id': 2L}] {u'bar': 0L}>

@dhermes dhermes changed the title Transaction does not rollback correctly when multiple datasets involved Transaction dataset_id should come from Key / Query involved Jan 7, 2015
@dhermes dhermes changed the title Transaction dataset_id should come from Key / Query involved Transaction dataset_id should come from Key / Query objects involved Jan 7, 2015
@dhermes dhermes self-assigned this Jan 7, 2015
@dhermes dhermes changed the title Transaction dataset_id should come from Key / Query objects involved Batch (and Transaction) dataset_id should come from Key / Query objects involved Jan 9, 2015
@tseaver
Copy link
Contributor

tseaver commented Jan 15, 2015

Note that calling beginTransaction requires the dataset ID (we can't do the RPC without it). We already made the dataset ID optional, falling back to the implicit one, but we cannot possibly rely on the keys, because we don't know them yet when we start the transaction.

I think it would be reasonable to say that users who don't want to use the implicit dataset ID know that when they create the batch / transaction, and can easily pass it. We would then change the code in put / delete to reject any key not matching the one set in __init__.

@dhermes
Copy link
Contributor Author

dhermes commented Jan 15, 2015

Yes I also came to this conclusion. Implicit saves us.

This re-opens the question about reading the dataset ID from every single key via _get_dataset_id_from_keys.

@dhermes dhermes changed the title Batch (and Transaction) dataset_id should come from Key / Query objects involved Batch (and Transaction) dataset_id should match that of Key / Query objects when added Jan 15, 2015
@dhermes
Copy link
Contributor Author

dhermes commented Jan 15, 2015

@tseaver I renamed to reflect the change.

Do you want to tackle this or should I?

@tseaver
Copy link
Contributor

tseaver commented Jan 15, 2015

I'll take it.

@jgeewax jgeewax modified the milestone: Datastore Stable Jan 30, 2015
@yoshi-automation yoshi-automation added 🚨 This issue needs some love. triage me I really want to be triaged. labels Apr 7, 2020
atulep pushed a commit that referenced this issue Apr 3, 2023
* build: enable Speech v2 client library generation

PiperOrigin-RevId: 477910378

Source-Link: googleapis/googleapis@3fde7ab

Source-Link: googleapis/googleapis-gen@6369579
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNjM2OTU3OTA2Y2M3ODY4MDY1YTUwY2VmNjFmNWU5OTVhNzVhZTNlNiJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* chore: regenerate docs/index.rst

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Anthonios Partheniou <partheniou@google.com>
atulep pushed a commit that referenced this issue Apr 6, 2023
* build: enable Speech v2 client library generation

PiperOrigin-RevId: 477910378

Source-Link: googleapis/googleapis@3fde7ab

Source-Link: googleapis/googleapis-gen@6369579
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNjM2OTU3OTA2Y2M3ODY4MDY1YTUwY2VmNjFmNWU5OTVhNzVhZTNlNiJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* chore: regenerate docs/index.rst

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Anthonios Partheniou <partheniou@google.com>
atulep pushed a commit that referenced this issue Apr 6, 2023
* build: enable Speech v2 client library generation

PiperOrigin-RevId: 477910378

Source-Link: googleapis/googleapis@3fde7ab

Source-Link: googleapis/googleapis-gen@6369579
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNjM2OTU3OTA2Y2M3ODY4MDY1YTUwY2VmNjFmNWU5OTVhNzVhZTNlNiJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* chore: regenerate docs/index.rst

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Anthonios Partheniou <partheniou@google.com>
atulep pushed a commit that referenced this issue Apr 18, 2023
* build: enable Speech v2 client library generation

PiperOrigin-RevId: 477910378

Source-Link: googleapis/googleapis@3fde7ab

Source-Link: googleapis/googleapis-gen@6369579
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNjM2OTU3OTA2Y2M3ODY4MDY1YTUwY2VmNjFmNWU5OTVhNzVhZTNlNiJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* chore: regenerate docs/index.rst

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Anthonios Partheniou <partheniou@google.com>
parthea pushed a commit that referenced this issue Jun 4, 2023
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
parthea pushed a commit that referenced this issue Jun 4, 2023
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
parthea pushed a commit that referenced this issue Jun 4, 2023
… Get, List, Update, Delete (#447)

* feat: released securitycenter/v1 SHA custom modules cloud libraries: Create, Get, List, Update, Delete

The Security Health Analytics (SHA) custom modules API is now released for general availability track. Create, Get, GetEffective, List, ListEffective, ListDescendant, Update, and Delete are available in the cloud client library.

PiperOrigin-RevId: 523462834

Source-Link: googleapis/googleapis@b7b3dfd

Source-Link: googleapis/googleapis-gen@9b679ec
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiOWI2NzllYzAyYTdlMTZkZjlmMGU3ZjY2ZTA3N2UzNzllZmQyNWFkNCJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
parthea pushed a commit that referenced this issue Aug 15, 2023
Source-Link: googleapis/synthtool@703554a
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:94961fdc5c9ca6d13530a6a414a49d2f607203168215d074cdb0a1df9ec31c0b
vchudnov-g pushed a commit that referenced this issue Sep 20, 2023
- [ ] Regenerate this pull request now.

PiperOrigin-RevId: 416079722

Source-Link: googleapis/googleapis@14b7a25

Source-Link: googleapis/googleapis-gen@e1b0be1
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZTFiMGJlMTVhN2NkMzQ2NmM0NmYwODY1ODM1ZTcxNGQxZTkzMDlmYiJ9

docs: added more docs for speech model variant and improved docs format for participant
vchudnov-g pushed a commit that referenced this issue Sep 20, 2023
🤖 I have created a release *beep* *boop*
---


## [2.11.0](googleapis/python-dialogflow@v2.10.0...v2.11.0) (2022-01-13)


### Features

* support document metadata filter in article suggestion ([#442](googleapis/python-dialogflow#442)) ([6f615f9](googleapis/python-dialogflow@6f615f9))
* **v2:** added export documentation method ([#449](googleapis/python-dialogflow#449)) ([a43d1e9](googleapis/python-dialogflow@a43d1e9))
* **v2:** added filter in list documentations request ([a43d1e9](googleapis/python-dialogflow@a43d1e9))
* **v2:** added filter in list knowledge bases request ([a43d1e9](googleapis/python-dialogflow@a43d1e9))
* **v2:** added option to apply partial update to the smart messaging allowlist in reload document request ([a43d1e9](googleapis/python-dialogflow@a43d1e9))
* **v2:** added option to import custom metadata from Google Cloud Storage in reload document request ([a43d1e9](googleapis/python-dialogflow@a43d1e9))
* **v2beta1:** add support for knowledge_base in knowledge operation metadata ([0f60629](googleapis/python-dialogflow@0f60629))
* **v2beta1:** added option to configure the number of sentences in the suggestion context ([#453](googleapis/python-dialogflow#453)) ([e48ea00](googleapis/python-dialogflow@e48ea00))
* **v2beta1:** removed OPTIONAL for speech model variant ([#448](googleapis/python-dialogflow#448)) ([0f60629](googleapis/python-dialogflow@0f60629))
* **v2:** removed OPTIONAL for speech model variant ([#447](googleapis/python-dialogflow#447)) ([56efd10](googleapis/python-dialogflow@56efd10))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
parthea pushed a commit that referenced this issue Sep 22, 2023
* fix: deprecate extra field to avoid confusion

PiperOrigin-RevId: 477776089

Source-Link: googleapis/googleapis@57062cd

Source-Link: googleapis/googleapis-gen@99b76b5
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiOTliNzZiNTJiMTM4YTc5Mjc3ZmNhYzhjNDY2NDRiODE1YzdhMzdkNyJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
parthea pushed a commit that referenced this issue Oct 21, 2023
- [ ] Regenerate this pull request now.

PiperOrigin-RevId: 459095142

Source-Link: googleapis/googleapis@4f1be99

Source-Link: googleapis/googleapis-gen@ae686d9
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYWU2ODZkOWNkZTRmYzNlMzZkMGFjMDJlZmI4NjQzYjE1ODkwYzFlZCJ9

feat: add audience parameter
PiperOrigin-RevId: 456827138

Source-Link: googleapis/googleapis@23f1a15

Source-Link: googleapis/googleapis-gen@4075a85
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNDA3NWE4NTE0ZjY3NjY5MWVjMTU2Njg4YTViYmYxODNhYTk4OTNjZSJ9
parthea pushed a commit that referenced this issue Oct 21, 2023
- [ ] Regenerate this pull request now.

PiperOrigin-RevId: 473833416

Source-Link: googleapis/googleapis@565a550

Source-Link: googleapis/googleapis-gen@1ee1a06
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMWVlMWEwNmM2ZGUzY2E4Yjg0MzU3MmMxZmRlMDU0OGY4NDIzNjk4OSJ9
parthea added a commit that referenced this issue Oct 22, 2023
* build: enable Speech v2 client library generation

PiperOrigin-RevId: 477910378

Source-Link: googleapis/googleapis@3fde7ab

Source-Link: googleapis/googleapis-gen@6369579
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNjM2OTU3OTA2Y2M3ODY4MDY1YTUwY2VmNjFmNWU5OTVhNzVhZTNlNiJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* chore: regenerate docs/index.rst

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Anthonios Partheniou <partheniou@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: datastore Issues related to the Datastore API. 🚨 This issue needs some love. triage me I really want to be triaged.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants