This should display below output on the console. Finally Run the Spark application SparkSessionTestĥ. Some time the dependencies in pom.xml are not automatic loaded hence, re-import the dependencies or restart the IntelliJ.Ĥ. Val sparkSession2 = SparkSession.builder()
Our hello world example doesn’t display “Hello World” text instead it creates a SparkSession and displays Spark app name, master and deployment mode to console. Now create the Spark Hello world program. Create Spark Hello world Application on IntelliJġ. Add Spark Dependencies to Maven pom.xml Fileĩ. Now delete the following from the project workspace.Ĩ.
Click on Install to install the Scala plugin.Ĥ.Select the Plugins option from the left panel.Open File > Settings (or using shot keys Ctrl + Alt + s ).You will see the project created on IntelliJ and shows the project structure on left Project panel.
On next screen, review the options for artifact-id and group-id
I am naming my project as spark-hello-world-example.Ħ. In the next window, enter the project name. Since we have selected Scala archetypes, it downloads all Scala dependencies and enables IntelliJ to write Scala code.ĥ. The archetype is a kind of templates that creates the right directory structure and downloads the required default dependencies.