Convert df to Spark DataFrame if it is a pandas or pandas-on-Spark DataFrame before writing

[dbeatty10]

resolves #316

Description

Enable pandas and pandas-on-Spark DataFrames for dbt Python models. Convert either of those types to Spark DataFrame before writing.

Checklist

• I have read the contributing guide and understand what's expected of me
• I have signed the CLA
• I have run this code in development and it appears to resolve the stated issue
• This PR includes tests, or tests are not required/relevant for this PR
• I have opened an issue to add/update docs, or docs changes are not required/relevant for this PR
• I have run changie new to create a changelog entry

[github-actions]

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the dbt-bigquery contributing guide.

[ChenyuLInx]

LGTM

[dbeatty10]

@ChenyuLInx I'm going to try adding support for importing Koalas (either in this PR or a separate PR) since:

• BigQuery dbt Python models uses Dataproc, and
• the most recent Apache Spark release Dataproc supports is v3.1, and
• pyspark.pandas wasn't introduced until Spark v3.2

[dbeatty10]

I'm going to try adding support for importing Koalas (either in this PR or a separate PR)

Opted for "separate PR":

• so that we could get the bulk of the logic merged now to achieve parity with dbt-spark, and
• since we'll want to do the same change to dbt-spark, we might as well do them at the same time