INVALID_ARGUMENT error on gcloud asset export --content-type relationship

[kbroughton]

Environment details

• MacOsx 11.13.1 big sur
• Python version: Python 3.9.6
• pip version: 21.1.3
• google-cloud-asset version: gcloud version

Google Cloud SDK 358.0.0
alpha 2021.09.17
beta 2021.09.17
bq 2.0.71
core 2021.09.17
gsutil 4.68

Steps to reproduce

According to the docs https://cloud.google.com/sdk/gcloud/reference/asset/export?hl=it
asset export should work with content-type=relationship.

1. Create a bq dataset.
2. Run the asset export command with gcloud or python sdk.

It works with content-type=resource:

gcloud   beta asset export    --content-type resource    --project $PROJECT_ID    --output-bigquery-force --bigquery-table "projects/$PROJECT_ID/datasets/$DATASET/tables/$TABLE"  --snapshot-time '2021-09-23' --partition-key REQUEST_TIME

But this does not:

gcloud   beta asset export    --content-type relationship    --project $PROJECT_ID    --output-bigquery-force --bigquery-table "projects/$PROJECT_ID/datasets/$DATASET/tables/$TABLE"  --snapshot-time '2021-09-23' --partition-key REQUEST_TIME

ERROR: (gcloud.beta.asset.export) code: 400
message: Request contains an invalid argument.
status: INVALID_ARGUMENT

Code example

      Uncomment when fixed: causes a grpc error
      if partition_key:
        output_config.bigquery_destination.table = '%s_relationship' % (bq_table)
      else:
        output_config.bigquery_destination.table = '%s_relationship_%s' % (bq_table, read_time.strftime('%Y%m%d'))

      response = client.export_assets(
          request={
              'parent': parent,
              'read_time': read_time,
              'content_type': asset_v1.ContentType.RELATIONSHIP,
              'output_config': output_config
          }
     )

Stack trace

googlecloudsdk.api_lib.util.exceptions.HttpException: code: 400
message: Request contains an invalid argument.
status: INVALID_ARGUMENT
ERROR: (gcloud.beta.asset.export) code: 400
message: Request contains an invalid argument.
status: INVALID_ARGUMENT

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

[busunkim96]

Hi @kbroughton,

Thank you for the report. It looks like the both gcloud and the library is not surfacing the detailed error. Could you share the version of google-cloud-asset you used?

[kbroughton]

bin/pip show google-cloud-asset
Name: google-cloud-asset
Version: 3.5.0
Summary: Cloud Asset API API client library
Home-page: https://github.com/googleapis/python-asset
Author: Google LLC
Author-email: googleapis-packages@google.com
License: Apache 2.0
Location: /private/tmp/py-asset/lib/python3.9/site-packages
Requires: google-api-core, google-cloud-access-context-manager, proto-plus, google-cloud-os-config, grpc-google-iam-v1, google-cloud-org-policy, packaging
Required-by:
(py-asset)

I had been using installs of the complete gcloud or sdk. I followed the install instructions in the readme to only install google-cloud-asset for this test. Same result.

[sirius-ed-hammond]

This was working in August, but it is failing now with the subject error
Google Cloud SDK [358.0.0]

gcloud beta asset export --organization=xxxxxxx --content-type relationship --output-path gs://xxxx/xxxx/relationships
ERROR: (gcloud.beta.asset.export) code: 400
message: Request contains an invalid argument.
status: INVALID_ARGUMENT

Tried alpha, beta, and GA versions of gcloud 358.0

[kbroughton]

from google.cloud import asset_v1
Here is the full setup and stacktrace

output_config = asset_v1.OutputConfig()
output_config.bigquery_destination.dataset = 'projects/$PROJECT_ID/datasets/$DATASET'
output_config.bigquery_destination.force = True
partition_spec = asset_v1.PartitionSpec()
partition_spec.partition_key = 'REQUEST_TIME'
output_config.bigquery_destination.partition_spec = partition_spec
output_config.bigquery_destination.table = 'test_resource'
client = asset_v1.AssetServiceClient()

response = client.export_assets(
... request={
... 'parent': parent,
... 'content_type': asset_v1.ContentType.RELATIONSHIP,
... 'output_config': output_config
... }
... )
Traceback (most recent call last):
File "", line 3, in
NameError: name 'parent' is not defined
parent = 'organizations/$ORG_ID'
response = client.export_assets(
... request={
... 'parent': parent,
... 'content_type': asset_v1.ContentType.RELATIONSHIP,
... 'output_config': output_config
... }
... )
Traceback (most recent call last):
File "/private/tmp/py-asset/lib/python3.9/site-packages/google/api_core/grpc_helpers.py", line 66, in error_remapped_callable
return callable_(*args, **kwargs)
File "/private/tmp/py-asset/lib/python3.9/site-packages/grpc/_channel.py", line 946, in call
return _end_unary_response_blocking(state, call, False, None)
File "/private/tmp/py-asset/lib/python3.9/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INVALID_ARGUMENT
details = "Request contains an invalid argument."
debug_error_string = "{"created":"@1632760077.006469000","description":"Error received from peer ipv4:142.250.64.170:443","file":"src/core/lib/surface/call.cc","file_line":1070,"grpc_message":"Request contains an invalid argument.","grpc_status":3}"

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "", line 1, in
File "/private/tmp/py-asset/lib/python3.9/site-packages/google/cloud/asset_v1/services/asset_service/client.py", line 475, in export_assets
response = rpc(request, retry=retry, timeout=timeout, metadata=metadata,)
File "/private/tmp/py-asset/lib/python3.9/site-packages/google/api_core/gapic_v1/method.py", line 142, in call
return wrapped_func(*args, **kwargs)
File "/private/tmp/py-asset/lib/python3.9/site-packages/google/api_core/grpc_helpers.py", line 68, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.InvalidArgument: 400 Request contains an invalid argument.

[busunkim96]

@sirius-ed-hammond Thanks for leaving a comment, it's helpful to know this worked in the past but doesn't now.

I'm able to reproduce the error through gcloud using my own project. I'm in the process of filing an issue on the Cloud Asset API team's issue tracker and will get back to you when I have a response.

[busunkim96]

Googlers, see internal issue 201301739.

[busunkim96]

CC @lvvvvvf

[lvvvvvf]

We used to have preview in gcloud beta with 1 relationship.

The original plan is GA at the end of this month, so I turned all gcloud to the v1 version which could only be used with for enable listed consumers now. After GA, all will be available to public.

Now there may be some changes to the plan, if so, I will change back the beta behavior. (next month)

Sorry about the inconvenience.

[sirius-ed-hammond]

If I understand correctly, you disabled the public /beta/ in deference to private GA release for select customers, so people who were using the public beta can no longer do so. Correct?

[lvvvvvf]

The plan is public GA 9/30, if so, all will turn public, the issue will be solved.

If the plan changed, I will switch back the gcloud beta to serve public the one relationship type next month. Thanks

[sirius-ed-hammond]

It has been a few months ... the relationship content-type is still broken for alpha, beta, and GA command invocations.
When might we expect it to be restored at least in beta mode?

$ gcloud asset export --organization=0000 --content-type relationships  --output-path gs://bucket/path
ERROR: (gcloud.asset.export) argument --content-type: Invalid choice: 'relationships'. Did you mean 'relationship'?

$ gcloud asset export --organization=0000 --content-type relationship  --output-path gs://bucket/path
ERROR: (gcloud.asset.export) code: 400
message: Request contains an invalid argument.
status: INVALID_ARGUMENT

$ gcloud --version
Google Cloud SDK 365.0.1
alpha 2021.11.19
beta 2021.11.19
bq 2.0.71
core 2021.11.19
gsutil 5.5
kpt 1.0.0-beta.4
kubectl 1.20.8

[sirius-ed-hammond]

Created issue tracker https://issuetracker.google.com/issues/209387751 to help raise visibility.

[parthea]

Hi @sirius-ed-hammond @kbroughton,

I'm going to close this issue as a duplicate of https://issuetracker.google.com/issues/209387751 as the issue is specific to product API rather than the client library.

[sirius-ed-hammond]

Anthonios Partheniou, I agree the issue tracker item<https://issuetracker.google.com/issues/209387751> should be sufficient to obtain resolution and the Git Issue<#298> is redundant. Is there an expected resolution estimate? Seems the logic is all there and ready for GA, but have been locked to selected customer accounts. When will that restriction be lifted so the interface works as is used to and is published in the documentation? From: Anthonios Partheniou ***@***.***> Reply-To: googleapis/python-asset ***@***.***> Date: Saturday, February 5, 2022 at 11:36 AM To: googleapis/python-asset ***@***.***> Cc: Ed Hammond ***@***.***>, Mention ***@***.***> Subject: [EXTERNAL] Re: [googleapis/python-asset] INVALID_ARGUMENT error on gcloud asset export --content-type relationship (#298) Hi @sirius-ed-hammond<https://urldefense.com/v3/__https:/github.com/sirius-ed-hammond__;!!Ly9OQR1JJQ!JwH7nEljNuiAicEOMXptgmW76k5suuKbNvd_9UQ8nPnn3tTSmxQwFlS46-DikKYX5w$> , I'm going to close this issue as a duplicate of https://issuetracker.google.com/issues/209387751<https://urldefense.com/v3/__https:/issuetracker.google.com/issues/209387751__;!!Ly9OQR1JJQ!JwH7nEljNuiAicEOMXptgmW76k5suuKbNvd_9UQ8nPnn3tTSmxQwFlS46-BI5D7Ivw$> as the issue is specific to product API rather than the client library. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/googleapis/python-asset/issues/298*issuecomment-1030655459__;Iw!!Ly9OQR1JJQ!JwH7nEljNuiAicEOMXptgmW76k5suuKbNvd_9UQ8nPnn3tTSmxQwFlS46-BlFlxE7A$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/ASCHE572D6MH5SFHCLBRPMLUZVGWXANCNFSM5EU35H7Q__;!!Ly9OQR1JJQ!JwH7nEljNuiAicEOMXptgmW76k5suuKbNvd_9UQ8nPnn3tTSmxQwFlS46-BFuu1O6Q$>. Triage notifications on the go with GitHub Mobile for iOS<https://urldefense.com/v3/__https:/apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675__;!!Ly9OQR1JJQ!JwH7nEljNuiAicEOMXptgmW76k5suuKbNvd_9UQ8nPnn3tTSmxQwFlS46-BJSUDBKQ$> or Android<https://urldefense.com/v3/__https:/play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign*3Dnotification-email*26utm_medium*3Demail*26utm_source*3Dgithub__;JSUlJSU!!Ly9OQR1JJQ!JwH7nEljNuiAicEOMXptgmW76k5suuKbNvd_9UQ8nPnn3tTSmxQwFlS46-AnltAJ-g$>. You are receiving this because you were mentioned.Message ID: ***@***.***> This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law or may constitute as attorney work product. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, notify us immediately by telephone and (i) destroy this message if a facsimile or (ii) delete this message immediately if this is an electronic communication. Thank you.