CS 511: Project 1 HDFS/Spark

To get started, spin up the cluster.

bash start-all.sh

Test HDFS deployment.

bash test_hdfs.sh

Test Spark deployment.

bash test_spark.sh

Name		Name	Last commit message	Last commit date
Latest commit History 5 Commits
resources		resources
.gitignore		.gitignore
README.md		README.md
cs511p1-common.Dockerfile		cs511p1-common.Dockerfile
cs511p1-compose.yaml		cs511p1-compose.yaml
cs511p1-main.Dockerfile		cs511p1-main.Dockerfile
cs511p1-worker.Dockerfile		cs511p1-worker.Dockerfile
setup-main.sh		setup-main.sh
setup-worker.sh		setup-worker.sh
start-all.sh		start-all.sh
start-main.sh		start-main.sh
start-worker.sh		start-worker.sh
test_hdfs.sh		test_hdfs.sh
test_spark.sh		test_spark.sh

Provide feedback