AllowList parsing error when exporting metadata

[niveditavijayvargiya]

I have generated my migrations using hasura console from existing project and its worked fine. Now i have created new project and i want to migrate my migrations to new instance its throwing error like issue in meta data and parsing failed.

time="2021-09-30T11:24:54+05:30" level=fatal msg="error applying metadata \n{\n "path": "$.args.metadata.query_collections[0].definition.queries[1].query",\n "error": "parsing the graphql query failed",\n "code": "parse-failed"\n}"

[BenoitRanque]

Can you share the contents of metadata/query_collections.yaml?
Specifically the second query in the first definition seems to be the problem, but I'd rather see the whole file.

I have a hunch this is related to #7608

[niveditavijayvargiya]

Hi,

• name: allowed-queries
  definition:
  queries:
  • name: getCompanies
    query: |-
    query MyQuery {
    company {
    company_addresses {
    address {
    address_line_1
    address_line_2
    city
    country
    postal_code
    state
    }
    }
    id
    name
    }
    }
  • name: addCompany
    query: "mutation MyQuery {\n insert_company(objects: \n {\n company_addresses:
    \n {\n data: {\n id: "94909f0b-e9db-4ce6-809b-0de59463caf1",
    \n address: {\n data: {\n address_line_1:
    "16, swastik circle", \n address_line_2: "", \n city:
    "Ahmedabad", \n country: "india", \n id: "1354fd0e-cb95-49ae-97d4-9ebfa225e831",
    \n postal_code: "312203", \n state: "Gujarat"},
    \n on_conflict: {constraint: address_pkey}\n }\n }\n
    \ }, \n name: "cygnet", \n id: "81c20bdf-fc22-473b-9ffb-173f4d6c6da2"}\n
    \ )\n {\n affected_rows\n }\n}"

[BenoitRanque]

Please wrap the code in 3 backticks, and add the proper formatting, so github does not alter the code:

```yaml

your:
  yaml:
  - code: "here"

```

Either way, it's weird that the mutation is a json string, while the query is a yaml multiline string...
Can you confirm that you have not edited this manually?

[niveditavijayvargiya]

No, i haven't done any change in it.

[BenoitRanque]

Can you try with this file, and let us know the results?

- name: allowed-queries
  definition:
    queries:
    - name: getCompanies
      query: |-
        query MyQuery {
          company {
            company_addresses {
              address {
                address_line_1
                address_line_2
                city
                country
                postal_code
                state
              }
            }
            id
            name
          }
        }
    - name: addCompany
      query: |-
        mutation MyQuery {
          insert_company(objects: {
            company_addresses: {
              data: {
                id: "94909f0b-e9db-4ce6-809b-0de59463caf1",
                address: {
                  data: {
                    address_line_1: "16, swastik circle", 
                    address_line_2: "", 
                    city: "Ahmedabad", 
                    country: "india", 
                    id: "1354fd0e-cb95-49ae-97d4-9ebfa225e831",
                    postal_code: "312203", 
                    state: "Gujarat"
                  },
                  on_conflict: {constraint: address_pkey}
                }
              }
            }, 
            name: "cygnet", 
            id: "81c20bdf-fc22-473b-9ffb-173f4d6c6da2"
          }) {
            affected_rows
          }
        }

Edit: Accidentally send invalid file, corrected, please try again

[niveditavijayvargiya]

I tried above in query_collection meta data but now i am getting different error.

time="2021-09-30T13:18:24+05:30" level=fatal msg="error applying metadata \n{\n "path": "$.args.metadata.query_collections[0].definition",\n "error": "When parsing the constructor CollectionDef of type Hasura.RQL.Types.QueryCollection.CollectionDef expected Object but got Null.",\n "code": "parse-failed"\n}"

[BenoitRanque]

I tried above in query_collection meta data but now i am getting different error.

time="2021-09-30T13:18:24+05:30" level=fatal msg="error applying metadata \n{\n "path": "$.args.metadata.query_collections[0].definition",\n "error": "When parsing the constructor CollectionDef of type Hasura.RQL.Types.QueryCollection.CollectionDef expected Object but got Null.",\n "code": "parse-failed"\n}"

was this after my edit?
NVM I see my mistake, updating
Edit: please try again, the file above has been edited

[niveditavijayvargiya]

I tried that updated file, now i am getting following error.

time="2021-09-30T13:25:38+05:30" level=fatal msg="error applying metadata \n{\n "path": "$.args.metadata.query_collections[0].definition.queries[1].query",\n "error": "parsing the graphql query failed",\n "code": "parse-failed"\n}"

[BenoitRanque]

That's the same error as before, meaning this isn't related to how the string is stored in the metadata file.
Looking further into this issue, this is definitely a bug

[BenoitRanque]

I've managed to narrow down this bug, it's some kind of very weird parsing error.
Here's my file that can reproduce the problem:

- name: allowed-queries
  definition:
    queries:
      - name: InsertUser
        query: |-
          mutation InsertUser {
            insert_user(objects: {
              username: "", 
              password: "81c20bdf-fc22-473b-9ffb-173f4d6c6da"
            }) {
              affected_rows
            }
          }

Importantly, there is a space after username: "", . Removing that space fixes the issue.
So does removing the password field, or providing a value that is not a UUID.
The username and password keys can be changed while still reproducing the bug.

[niveditavijayvargiya]

I am facing issue in migrating db also.
ERRO skipping applying migrations on database box-smart-poc, encountered:
{
"internal": {
"statement": "SET check_function_bodies = false;\nCREATE TABLE public.address (\n id uuid NOT NULL,\n
address_line_1 character varying(30) NOT NULL,\n address_line_2 character varying,\n city character varying NOT NULL,\n state character varying NOT NULL,\n country character varying NOT NULL,\n postal_code character varying NOT NULL\n);\nCREATE TABLE public.company (\n id uuid NOT NULL,\n name character varying\n);\nCREATE TABLE public.company_addresses (\n company_id uuid,\n address_id uuid,\n id uuid NOT NULL\n);\nCREATE TABLE public.owner (\n id uuid NOT NULL,\n first_name character varying(30)[] NOT NULL,\n last_name character varying(30)[] NOT NULL\n);\nALTER TABLE ONLY public.address\n ADD CONSTRAINT address_pkey PRIMARY KEY (id);\nALTER TABLE ONLY public.company\n ADD CONSTRAINT company_pk PRIMARY KEY (id);\nALTER TABLE ONLY public.owner\n ADD CONSTRAINT owner_pkey PRIMARY KEY (id);\nCREATE UNIQUE INDEX company_id_idx ON public.company USING btree (id);\nALTER TABLE ONLY public.company_addresses\n ADD CONSTRAINT company_addresses_fk FOREIGN KEY (address_id) REFERENCES public.address(id) ON UPDATE CASCADE ON DELETE CASCADE;\nALTER TABLE ONLY public.company_addresses\n ADD CONSTRAINT company_addresses_fk_1 FOREIGN KEY (company_id) REFERENCES public.company(id);\n",
"prepared": false,
"error": {
"exec_status": "FatalError",
"hint": null,
"message": "relation "address" already exists",
"status_code": "42P07",
"description": null
},
"arguments": []
},
"path": "$",
"error": "query execution failed",
"code": "postgres-error"
}

[BenoitRanque]

You are attempting to apply a migration that already exists on the database. Your migration state may be inconsistent.

[purush7]

Hey @niveditavijayvargiya, as @BenoitRanque stated in #7616 (comment) the issue occurred because of space and we aren't able to produce this metadata in v2.0.9. It would be great if you can mention the versions of cli and server i.e the output of hasura version command

[BenoitRanque]

@purush7 I can reproduce this on 2.0.9 as both server and cli version

[purush7]

@BenoitRanque Did you get metadata with spaces from server v2.0.9

[BenoitRanque]

Oh my bad, no what I meant was I get a parsing error with the document I posted above, which looks valid

[niveditavijayvargiya]

I am using cli version v2.0.8-cloud.3. This issue occur only i have that mutation query, if i remove this mutation end point then it allows me to migrate.

[BenoitRanque]

@niveditavijayvargiya you should be able to remove the space after name: "cygnet", , and the migration should work. But this is still a bug

[niveditavijayvargiya]

We want to add this flow in our cicd pipeline, when this issue can be resolved? Can you please help me on it?
It will not be a manual operation.

[purush7]

Hey @niveditavijayvargiya , Did you added the allow-list by uploading the graphql file or manually?

[niveditavijayvargiya]

No, i haven't added anything manually. I just created one instance in Hasura, added database from AWS RDS, added some rest endpoint with graph ql queries in it. After that i have created migration from this instance and try to apply those migration to another instance.
Followed all the steps from this documentation
https://hasura.io/docs/latest/graphql/core/migrations/migrations-setup.html#migrations-setup

[purush7]

Hey @niveditavijayvargiya, how did you added the allow-list which I cannot see that in above process?

[purush7]

BTW the error has been occurred due to the space which is present in the query itself
If you have uploaded the graphql file, please check for the spaces in query eg: name: "cygnet", after comma

[niveditavijayvargiya]

No, i haven't added. Its default one only. Ok, Let me check for space in my query.

[niveditavijayvargiya]

I haven't added allow list. Is it mandatory for migrations? Sorry i am new to Hasura.

[purush7]

 mutation MyQuery {
          insert_company(objects: {
            company_addresses: {
              data: {
                id: "94909f0b-e9db-4ce6-809b-0de59463caf1",
                address: {
                  data: {
                    address_line_1: "16, swastik circle", 
                    address_line_2: "", 
                    city: "Ahmedabad", 
                    country: "india", 
                    id: "1354fd0e-cb95-49ae-97d4-9ebfa225e831",
                    postal_code: "312203", 
                    state: "Gujarat"
                  },
                  on_conflict: {constraint: address_pkey}
                }
              }
            }, 
            name: "cygnet", 
            id: "81c20bdf-fc22-473b-9ffb-173f4d6c6da2"
          }) {
            affected_rows
          }
        }

you have spaces after

}, above name: "cygnet",
name: "cygnet",
postal_code: "312203",
country: "india",
city: "Ahmedabad",
address_line_2: "",
address_line_1: "16, swastik circle",

[purush7]

I haven't added allow list. Is it mandatory for migrations? Sorry i am new to Hasura.

It's not mandatory. It restricts your project’s GraphQL Engine to execute only queries that are present in the allowlist.

you can find the info about allow-list here https://hasura.io/docs/latest/graphql/cloud/security/allow-lists.html#manage-the-allow-list
https://hasura.io/docs/latest/graphql/core/deployment/allow-list.html#allow-list

[purush7]

@niveditavijayvargiya remove the spaces which I mentioned in #7616 (comment) and try to apply the metadata

[purush7]

@niveditavijayvargiya Is the problem solved?

[niveditavijayvargiya]

Yes Thanks. After removing extra space from my mutation query and regenerated migration so its started working. Migration applied successfully.

Still can you please fix that issue even graph ql query have any spaces at the end then while generating yaml file this space should be trimmed. It will be helpful approach as graph ql syntax allow such spaces but its Yaml syntax which create issue in case of extra spaces.

Sometime while beautify code for proper reading such spaces can be added in graph ql.

[hixus]

Got hit this today. Resolved it with stripIgnoredCharacters from graphql-js graphql/graphql-js#1802. So before inserting queries thorough metatadata api I clean them with stripIgnoredCharacters. Then the metadata export works fine.

Maybe hasura could store queries similarly and only prettyprint them in admin console

[THPubs]

For anyone else having this same issue, Prettify your code before converting it to a REST endpoint (You can Prettify by clicking the Prettify button in the Graphql playground).

If Hasura can prettify all the queries automatically before adding to the file, it would fix this whole issue.