Env : MapR 5.0
Spark : 1.4.1
Spark : 1.4.1
git clone https://github.com/andypetrella/spark-notebook.git // add mapr repo to sbt proxy repo vim ~/.sbt/repositories [repositories] local maven-central typesafe: http://repo.typesafe.com/typesafe/releases/ typesafe-snapshots: http://repo.typesafe.com/typesafe/snapshots/ mapr: http://repository.mapr.com/maven cd spark-notebook sbt -Dspark.version=1.4.1 -Dhadoop.version=2.7.0-mapr-1506 -Dwith.hive=true -Dwith.parquet=true clean dist add env variables $ export HADOOP_CONF_DIR=/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop $ export EXTRA_CLASSPATH=/opt/mapr/lib/commons-configuration-1.6.jar:/opt/mapr/lib/hadoop-auth-2.7.0.jar:/opt/mapr/lib/maprfs-5.0.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/zookeeper-3.4.5-mapr-1503.jar copy spark assembly jar to the /apps/spark hadoop fs -put /opt/mapr/spark/spark-1.4.1/lib/spark-assembly-1.4.1-hadoop2.5.1-mapr-1501.jar /apps/spark/spark-assembly.jar now start notebook using sbt run [info] play - Listening for HTTP on /0:0:0:0:0:0:0:0:9000 (Server started, use Ctrl+D to stop and go back to the console...) the message indicate that everything went smooth and notebook has started on port 9000. to access notebook open a browser and type address http://:9000 and u are good to go
No comments:
Post a Comment