Comment and metadata issue

[sogaiu]

It looks like if there is a comment right after some metadata the parse result may be off:

(pc/ast "^{:a true} ;; hello\n [:a]")
#_ 
(:code
 (:metadata
  (:metadata_entry
   (:map
    (:keyword ":a") (:whitespace " ")
    (:symbol "true")))
  (:whitespace " "))
 (:comment ";; hello") 
 (:whitespace "\n ")
 (:vector
  (:keyword ":a")))

Note that the comment, last whitespace, and vector are siblings of metadata and not children.

This is in contrast to:

(pc/ast "^{:a true} [:a]")
#_
(:code
 (:metadata
  (:metadata_entry
   (:map
    (:keyword ":a") (:whitespace " ")
    (:symbol "true")))
  (:whitespace " ")
  (:vector (:keyword ":a"))))

where vector is a child of metadata.

I encountered this in the wild: https://github.com/lspector/Clojush/blob/master/src/clojush/instructions/code.clj#L166-L176

[carocad]

@sogaiu thanks for the report.

From what I can see the input that you pass in the first case is not valid since a metadata entry cannot be added to a comment.

Could you please tell me if the result of ast is a failure with parcera/failure? and if so, display the error. If I recall correctly antlr4 will do its best to match the input even if the input is not valid (think of parsing a complete file with some grammar errors).

[sogaiu]

Thank for the response.

Hmm, command line interaction with clj gives the following here:

$ clj
Clojure 1.10.1
user=> (binding [*print-meta* true]
(prn (read-string "^{:a true} ;; hello\n [:a]")))
^{:a true} [:a]
nil

That looks to me like the metadata was successfully associated with some [:a].

Regarding parcera/failure?, I get:

user=> (pc/failure? (pc/ast "^{:a true} ;; hello\n [:a]"))
true

for errors, may be you mean like this?

user=> (:parcera.core/errors (meta (pc/ast "^{:a true} ;; hello\n [:a]")))
({:row 1, :column 11, :message "extraneous input ';; hello' expecting {'(', '[', '{', '^', '#^', '~', '~@', '#', '#{', SYMBOL}", :type :parser, :symbol "[@7,11:18=';; hello',<26>,1:11]", :stack (:code :input :form :reader_macro :metadata)})

[carocad]

@sogaiu thanks for the info, that helped to clear things out. I think the problem is that the current metadata parser rule does not expect a comment; which is a mistake. See here

I am not sure how the clojure parser is handling this in a more general case (whitespace, comment, discard) so I would like to check that out before fixing this.