Skip to content

Commit

Permalink
[SPARK-45145][EXAMPLE] Add JavaSparkSQLCli example
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

This PR aims to add a simple Java example, `JavaSparkSQLCli.java`. Like SparkPi example, we can take advantage of this minimal example with `bin/run-example` and we can run a simple SQL query as a canary test during RC testing and voting phase.

- https://spark.apache.org/docs/3.5.0/

```
./bin/run-example SparkPi 10
```

 After this PR,
```
$ bin/run-example sql.JavaSparkSQLCli "SELECT 'Spark SQL' col"
+---------+
|col      |
+---------+
|Spark SQL|
+---------+
```

### Why are the changes needed?

Although there exists `org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver` and `bin/spark-sql` shell environment, it requires `-Phive -Phive-thriftserver` explicitly.
```
$ bin/spark-shell --version
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 3.5.0
      /_/
...

$ bin/run-example org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver
...
Error: Failed to load class org.apache.spark.examples.org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.
Failed to load main class org.apache.spark.examples.org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.
You need to build Spark with -Phive and -Phive-thriftserver.
23/09/12 23:06:38 INFO ShutdownHookManager: Shutdown hook called
23/09/12 23:06:38 INFO ShutdownHookManager: Deleting directory /private/var/folders/d4/dr6zxyvd4cl38877bj3fxs_m0000gn/T/spark-2a9ee4a3-64c3-492c-a268-a71b9aa44a2a
```

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Manually.
```
$ build/sbt test:package
$ bin/run-example sql.JavaSparkSQLCli "SELECT 'Spark SQL' col"
```

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #42900 from dongjoon-hyun/SPARK-45145.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
  • Loading branch information
dongjoon-hyun committed Sep 13, 2023
1 parent c119b8a commit 66f42b5
Showing 1 changed file with 41 additions and 0 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.spark.examples.sql;

import org.apache.spark.sql.SparkSession;

/**
* Example Usage:
* <pre>
* bin/run-example sql.JavaSparkSQLCli "SELECT 'Spark SQL' col"
* </pre>
*/
public class JavaSparkSQLCli {

public static void main(String[] args) {
SparkSession spark = SparkSession
.builder()
.appName("Java Spark SQL Cli")
.getOrCreate();

for (String a: args) {
spark.sql(a).show(false);
}

spark.stop();
}
}

0 comments on commit 66f42b5

Please sign in to comment.