PubSub: Subscribing to a Pub/Sub topic requires additional permissions as a result of the release of google-cloud-pubsub 1.0.1

[jceresini]

Prior to todays release of the pubsub client, we were able to subscribe with only roles/pubsub.subscriber on the subscription we were trying to subscribe to. Now there is a call to get the subscription, which isn't permitted by that role.

It looks like this commit introduced the issue:
https://github.com/googleapis/google-cloud-python/pull/9268/files#diff-1004b3646a787f6b0438604df260553aR387

So after that release, our application broke (we happened to do a release a few hours after yours that picked up the latest pubsub library from pypi) because it did not have permission to get the subscription's metadata. After investigating for a while we were able to resolve the issue by adding roles/pubsub.viewer to the subscription in question. We had to enable Pub/Sub audit logs to get enough details to identify what was going on. Here's an example of the log that pointed us to the permission we were missing:

 {
   "protoPayload": {
     "@type": "type.googleapis.com/google.cloud.audit.AuditLog",
     "status": {
       "code": 7,
       "message": "PERMISSION_DENIED"
     },
     "authenticationInfo": {
       "principalEmail": "xxx@xxx.iam.gserviceaccount.com"
     },
     "requestMetadata": {
       "callerIp": "xxx",
       "callerSuppliedUserAgent": "grpc-python/1.24.0 grpc-c/8.0.0 (manylinux; chttp2; ganges),gzip(gfe)",
       "callerNetwork": "//compute.googleapis.com/projects/xxx/global/networks/__unknown__",
       "requestAttributes": {
         "time": "2019-09-27T22:41:46.042805867Z",
         "auth": {}
       },
       "destinationAttributes": {}
     },
     "serviceName": "pubsub.googleapis.com",
     "methodName": "google.pubsub.v1.Subscriber.GetSubscription",
     "authorizationInfo": [
       {
         "resource": "projects/xxx/subscriptions/xxx",
         "permission": "pubsub.subscriptions.get",
         "resourceAttributes": {}
       }
     ],
     "resourceName": "projects/xxx/subscriptions/xxx",
     "request": {
       "subscription": "projects/xxx/subscriptions/xxx",
       "@type": "type.googleapis.com/google.pubsub.v1.GetSubscriptionRequest"
     }
   },
   "insertId": "xxx",
   "resource": {
     "type": "pubsub_subscription",
     "labels": {
       "project_id": "xxx",
       "subscription_id": "projects/xxx/subscriptions/xxx"
     }
   },
   "severity": "ERROR",
   "logName": "projects/xxx/logs/cloudaudit.googleapis.com%2Fdata_access",
 }

[plamut]

@jceresini Thank you for the report, and I'm glad to hear that you managed to find a solution.

The linked change is needed to determine the default message acknowledge timeout set on the subscription before establishing a message stream. Previously, the client overrode that setting with a default value, which sometimes resulted in the ACK timeout hitting too soon, resulting in unnecessary premature re-delivery of the messages the client has not processed yet (issue #9252).

This should probably be documented better in the permissions and roles section of the docs, will escalate this thread.

[rvandegrift]

@plamut Would it be possible to catch the 403 error, emit a warning for the developer, and then fallback to the previous incorrect behavior? This'd make the required cloud change discoverable without digging through Cloud Audit.

[plamut]

@rvandegrift We were just discussing this issue, and the quickest short term solution is to change the problematic line - instead of fetching the subscription and read its ACK deadline, we can use a reasonable static default, e.g. 60 seconds.

That will, of course, re-introduce #9252, but the latter is a lesser evil than having surprising permission issues. And using e.g. 60 seconds instead of the previous 10 should mitigate #9252 to a degree.

(the goal here is that subscriptions should work out of the box with the default pubsub.subscriber role, i.e. without any custom permissions)

[plamut]

For the record - PubSub client version 1.0.2 has just been released, and it contains the fix for the permissions issue. It is again not necessary to have the subscriptions.get permission in order to use the streaming pull.