Showing posts with label SparkThrift Server Configuration in yarn mode. Show all posts
Showing posts with label SparkThrift Server Configuration in yarn mode. Show all posts

Thursday, January 7, 2016

SparkThrift Server Configuration in yarn mode

Env: MapR,Spark 1.4.1
Start spark thrift server on mapr cluster as follows (-- start thrift server non root user)
/opt/mapr/spark/spark-1.4.1/sbin/start-thriftserver.sh --master yarn --hiveconf hive.server2.thrift.bind.host `hostname` --hiveconf hive.server2.trift.port 10000

check for any error exception in the logs(in my case it is tailf /opt/mapr/spark/spark-1.4.1/logs/spark-mapr-org.apache.spark.sql.hive.thriftserver.HiveThriftServer2-1-ip-10-0-0-233.out)
if there is no error exception in the logs try connecting using the beeline ship with the spark distribution

/opt/mapr/spark/spark-1.4.1/bin/beeline 
Beeline version 1.4.1 by Apache Hive
beeline> !connect jdbc:hive2://10.0.0.233:10000/default;auth=noSasl
scan complete in 1ms
Connecting to jdbc:hive2://10.0.0.233:10000/default;auth=noSasl
Enter username for jdbc:hive2://10.0.0.233:10000/default;auth=noSasl: 
Enter password for jdbc:hive2://10.0.0.233:10000/default;auth=noSasl: 
Connected to: Spark SQL (version 1.4.1)
Driver: Spark Project Core (version 1.4.1)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://10.0.0.233:10000/default> show tables;
+------------------------------+--------------+
|          tableName           | isTemporary  |
+------------------------------+--------------+
| b1                           | false        |
| b2                           | false        |
| bob                          | false        |
| ct                           | false        |