Physical optimizers cannot rewrite HashJoinExec due to join_schema being private

[progval]

Is your feature request related to a problem or challenge?

HashJoinExec internally performs a projection, which is exposed through the public HashJoinExec::projection field. However, this field is a Option<Vec<usize>>, where each item in the vector is an index in HashJoinExec::join_schema, which is not public.

This means the projection performed by HashJoinExec is not available to physical optimizers (though they might be able to infer it from the left().schema() and right.schema()? this seems error-prone at best)

Describe the solution you'd like

Either:

1. make HashJoinExec::join_schema public
2. decouple projection HashJoinExec

Describe alternatives you've considered

Provide a way for library users to define their own logical_expr::Join -> ExecutionPlan compiler, so they don't need a physical optimizer to do it.

(If there is already a way to do it, then I missed it.)

Additional context

No response