Monday, September 2, 2013

Apache Pig : Writiting Java UDF for Pig

In this tutorial how to write user defined function for the pig.suppose we have a sample data in the for of tab separate document as fallows.the first column depicts name of the customer, second column represent the location of the customer and the third column give the information of customer's credit rating on the scale of 10. we need to filter out the the credit who scored the bad rating e.g. less than 5.

Amit  Noida 5
Ajay Delhi 8
Abhi Lucknow 3
Dev Punjab 7
Deepak Bihar 2

Lets create a Maven java project using the fallowing command:

>mvn archetype:generate -DgroupId=com.rajkrrsingh.pig.udf -DartifactId=JavaUDF
 -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false
Above command will create a new Java project with the name of JavaUDF, open pom.xml in the project directory and add the fallowing dependencies in it.
<project xmlns="" xmlns:xsi=""
    <!-- TODO: make sure Hadoop version is compatible -->

Now from command line execute
mvn eclipse:eclipse
Import the project in the eclipse using Import from existing project and create a java package and add the fallowing class to it.
package com.rajkrrsingh.pig.udf;


import org.apache.pig.FilterFunc;
import org.apache.pig.backend.executionengine.ExecException;

public class IsGoodCreditRating extends FilterFunc {

 public Boolean exec(Tuple args) throws IOException {
  if (args == null || args.size() == 0) {
        return false;
      try {
        Object object = args.get(0);
        if (object == null) {
          return false;
        int i = (Integer) object;
         return true;
         return false;
      } catch (ExecException e) {
        throw new IOException(e);
Create the jar file using the assembly plugin and moved it to your cluster.In the next step we will write a Pig script.
records = LOAD 'sample.txt' AS (name:chararray, location:chararray, creditrating:int);
filter_records = FILTER records BY com.rajkrrsingh.pig.udf.IsGoodCreditRating(creditrating);
grouped_records = GROUP filter_records BY location;
DUMP grouped_records;

run the scrip using pig CreditScore.pig and get the result.