-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Impossible to set non-default storage project since 0.5.0 #832
Comments
@mwitkow-io Thanks a ton for filing this! It's really great to hear that
Sorry the release notes didn't clarify some of these things. That's on me. The We removed RE: "makes it incredibly hard to operate on buckets in multiple projects", what is the issue you are experiencing? The bucket name ( |
Well, I blame Github issues not being as nice as Buganizer ;) Where does The Can you give me an example of how to use the new API against two separate projects (with separate credentials objects) and getting a |
RE: "Where does RE: "The Here is an example (we are working on documenting and improving this, see #805 and #830): from gcloud.credentials import get_for_service_account_json
from gcloud import datastore
# Working on making this part easier / shorter
creds1 = get_for_service_account_json('path/to/key1.json').create_scoped(
datastore.SCOPE)
conn1 = datastore.Connection(credentials=creds1)
creds2 = get_for_service_account_json('path/to/key2.json').create_scoped(
datastore.SCOPE)
conn2 = datastore.Connection(credentials=creds2)
bucket1 = storage.get_bucket('bucketname1', connection=conn1)
bucket2 = storage.get_bucket('bucketname2', connection=conn2)
bucket1.delete_blob('blob-name1.txt')
bucket2.delete_blob('blob-name2.txt') |
@mwitkow-io bucket names are globally unique (not per-project), which means you don't need to know the project ID except when creating a new bucket, or listing buckets associated with a given project. |
Oh, indeed. I completely forgot that buckets are globally identifiable. |
Please feel free to close :) (I wish I could) |
@mwitkow-io I really appreciate the feedback! Please feel free to open more and let us know. I am particularly interested in the
I'd love to hear more about your workload. We have better support for multithreading with our batches than we do for global defaults. I have a hunch that you'd rather be passing an explicit connection / dataset ID / project in a multithreaded environment, but again I'd love to hear how it's working in the wild. |
The upgrade to 0.5.0 broke the public API without notes in the Release by removing
storage.get_connection(project='foo')
function.Moreover, it seems that the
Connection
class no longer has aproject
parameter. Neitherstorage.get_bucket
norstorage.lookup_bucket
takeproject
as a parameter. Curiously, you can override it increate_bucket
andlist_buckets
.The only way to set a project for these is through
_helpers.set_default_project
, which makes it incredibly hard to operate on buckets in multiple projects, since you need to set the default all the time, exposing yourself to race conditions in case of multithreading.Can we put back the
project
intoConnection
? AFAIK theConnection
's credentials only make sense for a givenproject
anyway.The text was updated successfully, but these errors were encountered: