Storing an Entity as an indexed property value always fails

[dhermes]

From the spec:

If a value is itself an entity, it cannot have indexed set to true.

This means when the entity_value is added we should set indexed to False on the property. (This is currently not easy, see #403, #404 and #512.)

Or maybe we should just document that users need to add a property name to exclude_from_indexes if that property is an Entity.

---

To replicate:

>>> import os
>>> from gcloud import datastore
>>> from gcloud.datastore.entity import Entity
>>> from gcloud.datastore.key import Key
>>> 
>>> datastore.set_default_connection()
>>> dataset_id = os.getenv('GCLOUD_TESTS_DATASET_ID')
>>> datastore.set_default_dataset_id(dataset_id)
>>> 
>>> e = Entity(key=Key('Foo', 1))
>>> e2 = Entity(key=Key('Foo', 2))
>>> e['bar'] = e2
>>> e.save()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "gcloud/datastore/entity.py", line 144, in save
    exclude_from_indexes=self.exclude_from_indexes)
  File "gcloud/datastore/connection.py", line 495, in save_entity
    result = self.commit(dataset_id, mutation)
  File "gcloud/datastore/connection.py", line 376, in commit
    datastore_pb.CommitResponse)
  File "gcloud/datastore/connection.py", line 100, in _rpc
    data=request_pb.SerializeToString())
  File "gcloud/datastore/connection.py", line 77, in _request
    raise six.moves.http_client.HTTPException(message)
httplib.HTTPException: Request failed with status code 400.
                       Error was: Entity value is indexed.

and notice it doesn't fail if the property is correctly unindexed:

>>> e = Entity(key=Key('Foo', 1), exclude_from_indexes=('bar',))
>>> e['bar'] = e2
>>> e.save()
>>> e.key.delete()

[dhermes]

Note, also setting

e['foo'] = 'a' * 501

will cause a failure

Request failed with status code 400.
Error was: Indexed blob value has more than permitted 500 bytes.

since the spec states

When indexed is true, stringValue is limited to 500 characters and the blob value
is limited to 500 bytes.

Maybe we are better off just documenting this and letting the users worry about it.

I think in terms of code complexity, we are better off doing nothing (beyond documentation) and waiting for user feedback.

[tseaver]

Is there stuff in the datastore API docs that describes these limitations in detail? I'd rather just point to them, than try to write such docs ourselves.

[dhermes]

@proppy Is there somewhere under https://cloud.google.com/datastore/docs/... that surface the comments from the proto spec?

[proppy]

@dhermes In the API section there is a link to the .proto in the github repo for the current release.
https://github.com/GoogleCloudPlatform/google-cloud-datastore/blob/v1beta2-rev1-2.1.0/proto/datastore_v1.proto

[dhermes]

@proppy Sorry I was unclear.

I was asking if the user-facing docs actually have sections to match the "docs" from the spec. For example

When indexed is true, stringValue is limited to 500 characters and the blob value
is limited to 500 bytes.

is in the spec, but is it documented anywhere else? Same for

If a value is itself an entity, it cannot have indexed set to true.

and other useful information from the spec.

[proppy]

Sorry if I was unclear too :) At first there was a proto API reference that was duplicating the proto comment, but it was removed in a favor of a link to the file in the github repo.

The JSON API Reference source the doc from the same proto, so in theory https://cloud.google.com/datastore/docs/apis/v1beta2/entity should feature the same comment for the stringValue property description. But for some reason those seems to be missing from this page.

[dhermes]

Thanks. So in general we can point people to those docs for failure cases.

[dhermes]

Deferring this to #532