[SPARK-50032][SQL] Allow use of fully qualified collation name

[stevomitric]

What changes were proposed in this pull request?

In this PR collations can now be identified by their fully qualified name, as per the collation project plan. The Collation expression has been changed to always return fully qualified name. Currently we only support predefined collations.

Why are the changes needed?

Make collation names behave as per the project spec.

Does this PR introduce any user-facing change?

Yes. Two user-facing changes are made:

1. Collation expression now returns fully qualified name:

select collation('a' collate utf8_lcase) -- returns `SYSTEM.BUILTIN.UTF8_LCASE`

2. Collations can now be identified by their full qualified name:

select contains('a' collate system.builtin.utf8_lcase, 'A') -- returns true

How was this patch tested?

New tests in this PR.

Was this patch authored or co-authored using generative AI tooling?

No.

[MaxGekk]

@stefankandic @uros-db @srielau Please, have a look at the PR.

[MaxGekk]

@stevomitric Please, fix the test failure. It seems it is related:

[info] - collation on non-explicit default collation *** FAILED *** (2 milliseconds)
[info]   Incorrect evaluation (codegen off): SYSTEM.BUILTIN.UTF8_BINARY, actual: SYSTEM.BUILTIN.UTF8_BINARY, expected: UTF8_BINARY (ExpressionEvalHelper.scala:261)
[info]   org.scalatest.exceptions.TestFailedException:

[cloud-fan]

thanks, merging to master!