Saturday, January 9, 2016

BTrace : Tracing a java Application

BTrace is a helpful tool that lets you run Java-like scripts on top of a live JVM to capture or aggregate any form of variable state without restarting the JVM or deploying new code. This enables you to do pretty powerful things like printing the stack traces of threads, writing to a specific file, or printing the number of items of any queue or connection pool and many more.

lets trace a simple java application to know how much time a method took to complete at runtime without modifying a code.
Installation:
-Download latest version of BTrace from https://kenai.com/projects/btrace/downloads/directory/releases and unzip it some location.
-Add the env variable as follows

export BTRACE_HOME=/root/btrace-bin
PATH=$PATH:$BTRACE_HOME/bin

This is the sample code which I want to trace
package  com.rajkrrsingh.trace.test;  
  
public  class  BtraceTest {  
  
    public  void  execute (int i) {   
        System.out.println ("executing "+i+" time");  
        try  {  
            Thread.sleep (Math.abs (( int ) (Math.random () *  1000 )));  
        }  catch  (InterruptedException e) {  
            e.printStackTrace ();  
        }  
    }  
  
    public  void  doExcute () {  
       int i=0; 
       while  ( true ) {  
           
            execute (i);
            i++;  
        }  
    }  
  
    public  static  void  main (String [] args) {  
        BtraceTest btraceTest =  new  BtraceTest ();  
        btraceTest.doExcute ();  
    }  
}  

BTrace require a tracing script which has annotated action about what to trace and can add additional capabilities.this is the user guide to know about the different annotations and what they signifies.we have a following tracing script which will trace the execute method and print its execution time. to compile this script you need to have brace-tool.jar in the class path.

package  com.rajkrrsingh.trace.test;

import com.sun.btrace.annotations.BTrace;

import static com.sun.btrace.BTraceUtils.jstack;
        import static com.sun.btrace.BTraceUtils.println;
        import static com.sun.btrace.BTraceUtils.str;
        import static com.sun.btrace.BTraceUtils.strcat;
        import static com.sun.btrace.BTraceUtils.timeMillis;

        import com.sun.btrace.annotations.BTrace;
        import com.sun.btrace.annotations.Kind;
        import com.sun.btrace.annotations.Location;
        import com.sun.btrace.annotations.OnMethod;
        import com.sun.btrace.annotations.TLS;


@BTrace
public class TraceMethodTime {

    @TLS
    static Long beginTime;

    @OnMethod(clazz="com.rajkrrsingh.trace.test.BtraceTest", method="execute")

    public static void traceExecuteBegin() {
        println("method Start!");
        beginTime = timeMillis();

    }


    @OnMethod(clazz="com.rajkrrsingh.trace.test.BtraceTest", method="execute", location=@Location(Kind.RETURN))

    public static void traceExcute() {

        println(strcat(strcat("btrace.test.MyBtraceTest.execute time is: ", str(timeMillis() - beginTime)), "ms"));
        println("method End!");
        jstack();
    }
}

now it time to trace the already running java application, to know the process id of the application use jps command which will fetch you the process id.attach btrace to process id as follows and it will start tracing the application
btrace process-id script-location
btrace 22618 com/rajkrrsingh/trace/test/TraceMethodTime.java 
Btrace will attach to the running jam and you will see the following output on the console.
method Start!
btrace.test.MyBtraceTest.execute time is: 323ms
method End!
com.rajkrrsingh.trace.test.BtraceTest.execute(BtraceTest.java:12)
com.rajkrrsingh.trace.test.BtraceTest.doExcute(BtraceTest.java:18)
com.rajkrrsingh.trace.test.BtraceTest.main(BtraceTest.java:25)
method Start!
btrace.test.MyBtraceTest.execute time is: 655ms
method End!
com.rajkrrsingh.trace.test.BtraceTest.execute(BtraceTest.java:12)
com.rajkrrsingh.trace.test.BtraceTest.doExcute(BtraceTest.java:18)
com.rajkrrsingh.trace.test.BtraceTest.main(BtraceTest.java:25)

Thursday, January 7, 2016

SparkThrift Server Configuration in yarn mode

Env: MapR,Spark 1.4.1
Start spark thrift server on mapr cluster as follows (-- start thrift server non root user)
/opt/mapr/spark/spark-1.4.1/sbin/start-thriftserver.sh --master yarn --hiveconf hive.server2.thrift.bind.host `hostname` --hiveconf hive.server2.trift.port 10000

check for any error exception in the logs(in my case it is tailf /opt/mapr/spark/spark-1.4.1/logs/spark-mapr-org.apache.spark.sql.hive.thriftserver.HiveThriftServer2-1-ip-10-0-0-233.out)
if there is no error exception in the logs try connecting using the beeline ship with the spark distribution

/opt/mapr/spark/spark-1.4.1/bin/beeline 
Beeline version 1.4.1 by Apache Hive
beeline> !connect jdbc:hive2://10.0.0.233:10000/default;auth=noSasl
scan complete in 1ms
Connecting to jdbc:hive2://10.0.0.233:10000/default;auth=noSasl
Enter username for jdbc:hive2://10.0.0.233:10000/default;auth=noSasl: 
Enter password for jdbc:hive2://10.0.0.233:10000/default;auth=noSasl: 
Connected to: Spark SQL (version 1.4.1)
Driver: Spark Project Core (version 1.4.1)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://10.0.0.233:10000/default> show tables;
+------------------------------+--------------+
|          tableName           | isTemporary  |
+------------------------------+--------------+
| b1                           | false        |
| b2                           | false        |
| bob                          | false        |
| ct                           | false        |