Google cloud datastore slow. >1 sec / query

[nojvek]

We've been getting consistent greater than 1 second on average calls from gcds. Not sure why its ridiculously slow.

We are using it from python version : google-cloud-datastore==1.2.0

from google.cloud import datastore

client = datastore.Client('foo', 'bar')
client.query(
  kind='Blah',
  filters=(
      ('name', '=', 'yolo'),
  ),
  order=('timestamp',)
)

print(list(query.fetch()))

Very simple call but consistently very slow.

[dhermes]

@nojvek Two potential explanations immediately come to mind for me:

• Your internet connection is slow
• list(query.fetch()) is making MANY requests. query.fetch() will return an iterator and will continue paging through results (i.e. making one API request per page) until all results matching the query have been found

Do either of these seem reasonable?

[chemelnucfin]

@nojvek Hello, may I ask if @dhermes answered your question?

[nojvek]

The total number of items in the list is less than 5. So I'm amazed that its slow.

…

On Wed, Jan 10, 2018 at 9:33 PM, chemelnucfin ***@***.***> wrote: @nojvek <https://github.com/nojvek> Hello, may I ask if @dhermes <https://github.com/dhermes> answered your question? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4560 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA-JVMiilpwbkuTxbjICHsJ2ZSpfW9fZks5tJZ0ygaJpZM4Q7x4a> .

[chemelnucfin]

@dmcgrath Hi Dan, I saw your answer in #4925. May I ask if you have any insights on this issue?

Thanks.

[tseaver]

@nojvek Your example doesn't show it, but is it possible you are doing the reads in the context of a transaction? The "one per second" response time sounds like one of the limits mentioned in the Datastore API docs:

Maximum rate of transactions reading from or writing to an entity group | 1 per sec

That row links to the Transactions and Entity Groups concept doc:

There is a write throughput limit of about one transaction per second within a single entity group. This limitation exists because Cloud Datastore performs masterless, synchronous replication of each entity group over a wide geographic area to provide high reliability and fault tolerance.

@pcostell Can you suggest any other explanation?

[sduskis]

I'm going to take the initiative and close this. This thread is old, and generally latency issues benefit from working with Google Cloud support engineers who have more access to information than the client team would. Please reopen if there's no smoking gun from the service side.