Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CARBONDATA-48] Supported spark sql cli in carbon. #33

Closed
wants to merge 1 commit into from

Conversation

ravipesala
Copy link
Contributor

@kumarvishal09
Copy link
Contributor

LGTM

@asfgit asfgit closed this in 3f56dfb Jul 12, 2016
@xiaoyesoso
Copy link
Contributor

@chenliang613
I can't load data by sql cli.It works by spark-shell with jars.
Maybe it because the carbon.storelocation is a tmp dir.
And please support that carbon-spark-sql can load property-file:
example:

./carbon-spark-sql --properties-file ../conf/carbon.properties

Now these will be ignored:

[root@holodesk01 bin]# ./carbon-spark-sql --properties-file ../conf/carbon.properties 
Warning: Ignoring non-spark config property: carbon.enable.quick.filter=false
Warning: Ignoring non-spark config property: carbon.sort.file.buffer.size=20
Warning: Ignoring non-spark config property: carbon.sort.size=500000
Warning: Ignoring non-spark config property: carbon.major.compaction.size=1024
Warning: Ignoring non-spark config property: carbon.number.of.cores.while.loading=6
Warning: Ignoring non-spark config property: carbon.number.of.cores=4
Warning: Ignoring non-spark config property: carbon.compaction.level.threshold=4,3
Warning: Ignoring non-spark config property: carbon.number.of.cores.while.compacting=2
Warning: Ignoring non-spark config property: carbon.storelocation=hdfs://holodesk01/user/CarbonStore
Warning: Ignoring non-spark config property: carbon.ddl.base.hdfs.url=hdfs://holodesk01/user/CarbonStore/data
Warning: Ignoring non-spark config property: carbon.kettle.home=$<SPARK_HOME>/carbonlib/carbonplugins
Warning: Ignoring non-spark config property: carbon.badRecords.location=/opt/Carbon/Spark/badrecords
Warning: Ignoring non-spark config property: carbon.enableXXHash=true
Warning: Ignoring non-spark config property: carbon.inmemory.record.size=120000
Warning: Ignoring non-spark config property: carbon.graph.rowset.size=100000

Beyyes pushed a commit to Beyyes/carbondata that referenced this pull request Jul 12, 2018
jackylk pushed a commit to jackylk/incubator-carbondata that referenced this pull request Dec 23, 2019
1. Registering all ModelArts UDF to spark while starting spark session
2. Added valid options to be provided while creating experiment and model
3. Added test cases
jackylk pushed a commit to jackylk/incubator-carbondata that referenced this pull request Dec 23, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants