Any plans of creating lineage for MERGE statements?

[snowmanwangyal]

Are there any plans on extracting lineage for MERGE statements?

[reata]

We have #166 sitting in the backlog for quite some time. The reason I didn't pick it up is: 1) the underlying parser (i.e, sqlparse) doesn't support MERGE statement very well. 2) the MERGE statement lineage analysis clearly can't re-use the logic for DML lineage analyzer. Most certainly we need a specified rule for it.

I'm doing a POC by replacing sqlparse to ANTLR4 (a parser generation tool), that will give me more control over AST generated by parser. The current plan is that if everything goes well, the auto generated parser works, we replace in in sqllineage, and then shall I consider support for MERGE statement.

[snowmanwangyal]

That sounds great, just read a bit about ANTLR4 and looking into it myself, hope to be of help as your POC progresses 👍

[reata]

Sure, thanks. I'll share the POC branch, or maybe a new repo, when I have something solid. Hopefully it's some time no later than next month.

[reata]

hey @snowmanwangyal , here's the new repo for POC: https://github.com/reata/sqllineage4j