Thursday, January 7, 2016

SparkThrift Server Configuration in yarn mode

Env: MapR,Spark 1.4.1
Start spark thrift server on mapr cluster as follows (-- start thrift server non root user)
/opt/mapr/spark/spark-1.4.1/sbin/ --master yarn --hiveconf `hostname` --hiveconf hive.server2.trift.port 10000

check for any error exception in the logs(in my case it is tailf /opt/mapr/spark/spark-1.4.1/logs/spark-mapr-org.apache.spark.sql.hive.thriftserver.HiveThriftServer2-1-ip-10-0-0-233.out)
if there is no error exception in the logs try connecting using the beeline ship with the spark distribution

Beeline version 1.4.1 by Apache Hive
beeline> !connect jdbc:hive2://;auth=noSasl
scan complete in 1ms
Connecting to jdbc:hive2://;auth=noSasl
Enter username for jdbc:hive2://;auth=noSasl: 
Enter password for jdbc:hive2://;auth=noSasl: 
Connected to: Spark SQL (version 1.4.1)
Driver: Spark Project Core (version 1.4.1)
0: jdbc:hive2://> show tables;
|          tableName           | isTemporary  |
| b1                           | false        |
| b2                           | false        |
| bob                          | false        |
| ct                           | false        |

Post a Comment