Sunday, January 4, 2015

Google Gauva API in a one glance: Objects Class to Implement hashcode, equals, toString and compareTo methods

Objects class helps the developer accurately and easily implement the equals(), hashCode(),toString(), and compareTo() methods. it gives the developer to provide implementation of these utility method in more clean and compact way.Developer need not to worry about the clumsy implementation involving variables and nulls.
lets demonstrate it by taking out Employee Class and override these methods with the help of Objects class
package com.rajkrrsingh.test.guava;

import java.util.ArrayList;
import java.util.List;

import com.google.common.base.Objects;
import com.google.common.collect.ComparisonChain;

public class Employee implements Comparable<Employee>{

private String empid;
private String empName;
private int salary;
private int age;
private static List<Employee> list;

static{
list = new ArrayList<Employee>();
list.add(new Employee("101", "RKS", 10000, 31));
list.add(new Employee("102", "Derek", 10500, 35));
list.add(new Employee("103", "Jack", 9000, 29));
list.add(new Employee("104", "Nick", 9600, 35));
}

public  static List<Employee> getEmployeeList(){
return list;
}

public Employee(){}

public Employee(String empid,String empName,int salary,int age){
this.empid = empid;
this.empName = empName;
this.salary = salary;
this.age = age;
}

public String getEmpid() {
return empid;
}

public void setEmpid(String empid) {
this.empid = empid;
}

public String getEmpName() {
return empName;
}

public void setEmpName(String empName) {
this.empName = empName;
}

public int getSalary() {
return salary;
}

public void setSalary(int salary) {
this.salary = salary;
}

public int getAge() {
return age;
}

public void setAge(int age) {
this.age = age;
}


@Override
public int hashCode() {
// TODO Auto-generated method stub
return Objects.hashCode(empid,empName);
}


@Override
public boolean equals(Object obj) {
if (this == obj) {
return true;
}
if (!(obj instanceof Employee)) {
return false;
}
Employee emp = (Employee) obj;
return Objects.equal(this.empid, emp.empid) && Objects.equal(this.empName, emp.empName); 
}


@Override
public String toString() {
return Objects.toStringHelper(this).add("empid", empid).add("empName", empName).toString();
}


@Override
public int compareTo(Employee o) {
// ComparisionChain 
return ComparisonChain.start().compare(empid, o.empid)
.compare(empName, o.empName)
.compare(salary, o.salary)
.compare(age, o.age)
.result();
}


}
now test our implementation using out tester class
/**
* 
*/
package com.rajkrrsingh.test.guava;

import java.util.Arrays;
import java.util.Collection;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Set;

import com.google.common.base.CharMatcher;
import com.google.common.base.Function;
import com.google.common.base.Functions;
import com.google.common.base.Joiner;
import com.google.common.base.Predicate;
import com.google.common.base.Predicates;
import com.google.common.collect.Collections2;
import com.google.common.collect.Iterables;

/**
* @author rks
* @04-Jan-2015
*/
public class GuavaObjectsDemo {

public static void main(String[] args) {
objectClass();
cleanComapreToTest();
}


// see the compareTo implementation in Employee class
public static void cleanComapreToTest(){
Set<Employee> set = new HashSet<Employee>();
// print true
System.out.println(set.add(new Employee("101", "RKS", 10000, 31)));
// print false - duplicate object
System.out.println(set.add(new Employee("101", "RKS", 10000, 31)));
}

// Object class with the helper method to implement hashcode() equals() and toString() methods
public static void objectClass(){
Employee e = new Employee("105", "Tom", 80000, 24);
// toString test
System.out.println(e);
Employee e1 = new Employee("105", "Tomm", 80000, 24);
System.out.println(e1.equals(e));
}
}

Google Gauva API in a one glance: CharMatcher

It's a class similar to Predicate,apply a boolean check on the chars, have methods to work on the char sequence e.g.
removeFrom(), replaceFrom(), trimFrom(), collapseFrom(), retainFrom()
lets see a simple example involving CharMatcher, follow along the code here
/**
*
*/
package com.rajkrrsingh.test.guava;

import java.util.Arrays;
import java.util.Collection;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;

import com.google.common.base.CharMatcher;
import com.google.common.base.Function;
import com.google.common.base.Functions;
import com.google.common.base.Joiner;
import com.google.common.base.Predicate;
import com.google.common.base.Predicates;
import com.google.common.collect.Collections2;
import com.google.common.collect.Iterables;

/**
* @author rks
* @04-Jan-2015
*/
public class GuavaCharMatcherDemo {

public static void main(String[] args) {
charMatcher();
}


public static void charMatcher(){
// remove char occurance in the range of 1 and 4
System.out.println(CharMatcher.inRange('1', '4').removeFrom(" 981 654 239"));
// negate the result obtained from previous statement
System.out.println(CharMatcher.inRange('1', '4').negate().removeFrom(" 981 654 239"));
// count no of digit in the string
System.out.println(CharMatcher.DIGIT.countIn(" 981 654 239 "));
// collapse a matching digits with the provided chars
System.out.println(CharMatcher.DIGIT.collapseFrom("collapse from 981 654 239", 'X'));
// replace digit in the string with the provide char
System.out.println(CharMatcher.DIGIT.replaceFrom("collapse from 981 654 239", 'X'));
// trim a string on matching char
System.out.println(CharMatcher.is(' ').trimFrom(" 981 654 239 "));
System.out.println(CharMatcher.is(' ').trimLeadingFrom(" 981 654 239 "));
System.out.println(CharMatcher.is(' ').trimTrailingFrom(" 981 654 239 "));
System.out.println(CharMatcher.is(' ').trimAndCollapseFrom(" 981 654 239 ",'X'));
}
}
Output
 98 65 9
1423
9
collapse from  X X X
collapse from  XXX XXX XXX
981 654 239
981 654 239 
 981 654 239
981X654X239
please follow the comment in the code to relate output.

Google Gauva API in a one glance: Joiner



@GwtCompatible
public class Joiner
extends Object
An object which joins pieces of text (specified as an array, Iterable, varargs or even a Map) with a separator. It either appends the results to an Appendable or returns them as a String.

In the following example, we join a List of String into one String using the “#” as a separator:
/**
*
*/
package com.rajkrrsingh.test.guava;

import java.util.Arrays;
import java.util.Collection;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;

import com.google.common.base.CharMatcher;
import com.google.common.base.Function;
import com.google.common.base.Functions;
import com.google.common.base.Joiner;
import com.google.common.base.Predicate;
import com.google.common.base.Predicates;
import com.google.common.collect.Collections2;
import com.google.common.collect.Iterables;

/**
* @author rks
* @04-Jan-2015
*/
public class GuavaJoinerDemo {

public static void main(String[] args) {
joinerDemo();
}


// tranform a collection into a sting
public static void joinerDemo(){
List<String> list = Arrays.asList("RKS","John","Nick","Harry");
System.out.println(Joiner.on("#").join(list));

List<String> list1 = Arrays.asList("RKS","John",null,"Nick","Harry");
//skip nulls
System.out.println(Joiner.on("#").skipNulls().join(list1));
//defualt value for null
System.out.println(Joiner.on("#").useForNull("BLANK").join(list1));

// joiner on Map
Map<String, String> map = new HashMap<String, String>();
map.put("key1", "value1");
map.put("key2", "value2");
map.put("key3", "value3");
map.put("key4", "value4");

System.out.println(Joiner.on("#").withKeyValueSeparator(":").join(map));
}
}
Please follow my comments in the code to relate with the output
RKS#John#Nick#Harry
RKS#John#Nick#Harry
RKS#John#BLANK#Nick#Harry
key4:value4#key3:value3#key2:value2#key1:value1

Google Gauva API in a one glance: Predicate

  • Predicate<T>, which has the single method boolean apply(T input). Instances of Predicate are generally expected to be side-effect-free and consistent with equals.

@GwtCompatible
public interface Predicate<T>
Determines a true or false value for a given input.The Predicates class provides common predicates and related utilities.
lets see how to use predicate to filter out some elements from the given collection. create a tester class as follows and follow along my blog post to get Employee Class
/**
*
*/
package com.rajkrrsingh.test.guava;

import java.util.Collection;
import java.util.Iterator;
import java.util.List;

import com.google.common.base.Function;
import com.google.common.base.Functions;
import com.google.common.base.Predicate;
import com.google.common.base.Predicates;
import com.google.common.collect.Collections2;
import com.google.common.collect.Iterables;

/**
* @author rks
* @04-Jan-2015
*/
public class GuavaPredicateDemo {

public static void main(String[] args) {
pridcateDemo();
}

public static void pridcateDemo(){
List<Employee> empList = Employee.getEmployeeList();

Predicate<Employee> ageOver30 = new Predicate<Employee>() {

@Override
public boolean apply(Employee emp) {
if(emp.getAge()>30){
return true;
}
return false;
}
};

Predicate<Employee> slryLt10000 = new Predicate<Employee>() {

@Override
public boolean apply(Employee emp) {
if(emp.getSalary()<10000)
return true;
return false;
}
};

System.out.println("**** print emp name whose age is greater than 30 ****");
Iterator<Employee> filterAgeIterator= Iterables.filter(empList, ageOver30).iterator();
while(filterAgeIterator.hasNext()){
System.out.println(filterAgeIterator.next().getEmpName());
}

System.out.println("**** print emp name whose age is greater than 30 and salary is less than 10000 ****");
Iterator<Employee> filterOnAgeAndSal= Iterables.filter(empList, Predicates.and(ageOver30, slryLt10000)).iterator();
while(filterOnAgeAndSal.hasNext()){
System.out.println(filterOnAgeAndSal.next().getEmpName());
}

System.out.println("**** print emp name whose age is greater than 30 OR salary is less than 10000 ****");
Iterator<Employee> filterOnAgeORSal= Iterables.filter(empList, Predicates.or(ageOver30, slryLt10000)).iterator();
while(filterOnAgeORSal.hasNext()){
System.out.println(filterOnAgeORSal.next().getEmpName());
}
}
}
Please follow my comments in the code to relate with the output
**** print emp name whose age is greater than 30 ****
RKS
Derek
Nick
**** print emp name whose age is greater than 30 and salary is less than 10000 ****
Nick
**** print emp name whose age is greater than 30 OR salary is less than 10000 ****
RKS
Derek
Jack
Nick


Google Gauva API in a one glance: Composite Function


Guava API provide a way to apply series of functions on the given collection using Functions.compose

public static <A,B,C> Function<A,C> compose(Function<B,C> g,
                                            Function<A,? extends B> f)
Returns the composition of two functions. For f: A->B and g: B->C, composition is defined as the function h such that h(a) == g(f(a)) for each a.
Parameters:
g - the second function to apply
f - the first function to apply
Returns:
the composition of f and g

to test it further please follow my blog entry and add one method to the GuavaFunctionDemo class and run it
/**
*
*/
package com.rajkrrsingh.test.guava;

import java.util.Collection;
import java.util.Iterator;

import com.google.common.base.Function;
import com.google.common.base.Functions;
import com.google.common.collect.Collections2;

/**
* @author rks
* @04-Jan-2015
*/
public class GuavaFunctionDemo {

public static void main(String[] args) {
//printEmployeeInUpperCase();
printResultofCompositeFunction();
}

// Transformation of one collection to the other collection using Guava function
public static void printEmployeeInUpperCase(){
Collection<String> upperCaseEmpName = Collections2.transform(Employee.getEmployeeList(),new Function<Employee, String>() {

@Override
public String apply(Employee emp) {
if(emp != null)
return emp.getEmpName().toUpperCase();
return "";

}
});

Iterator<String> itr = upperCaseEmpName.iterator();
while(itr.hasNext()){
System.out.println(itr.next());
}
}

// Apply a series of transformation by compostion of function
public static void printResultofCompositeFunction(){
Function<Employee, String> upperCaseEmpName = new Function<Employee, String>() {

@Override
public String apply(Employee emp) {
if(emp!=null)
return emp.getEmpName().toUpperCase();
return "";
}
};

Function<String, String> reverseEmpName = new Function<String, String>() {

@Override
public String apply(String empName) {
if (empName!=null) {
return new StringBuilder(empName).reverse().toString();
}
return "";
}

};
;
Collection<String> upperReverseCollection = Collections2.transform(Employee.getEmployeeList(), Functions.compose(reverseEmpName, upperCaseEmpName));

Iterator<String> itr = upperReverseCollection.iterator();
while(itr.hasNext()){
System.out.println(itr.next());
}
}

}

you can see here we have applied two function here, the first on is to change element of the collection in uppercase and then reverse it.

Result:
SKR
KERED
KCAJ
KCIN




Saturday, January 3, 2015

Google Gauva API in a one glance: Function

Google Guava Libraries are a nice set of Java utility classes that will probably come in handy in any project. Stuff like immutable collections, string manipulation, handling primitives and easier I/O are some of the concepts they deal with.


  • Function<A, B>, which has the single method B apply(A input). Instances of Function are generally expected to be referentially transparent -- no side effects -- and to be consistent with equals, that is, a.equals(b) implies thatfunction.apply(a).equals(function.apply(b)).

In this tutorial I will demonstarte you how to transform a collection using the function.
package com.rajkrrsingh.test.guava;

import java.util.ArrayList;
import java.util.List;

import com.google.common.base.Objects;
import com.google.common.collect.ComparisonChain;

public class Employee implements Comparable<Employee>{

private String empid;
private String empName;
private int salary;
private int age;
private static List<Employee> list;

static{
list = new ArrayList<Employee>();
list.add(new Employee("101", "RKS", 10000, 31));
list.add(new Employee("102", "Derek", 10500, 35));
list.add(new Employee("103", "Jack", 9000, 29));
list.add(new Employee("104", "Nick", 9600, 35));
}

public static List<Employee> getEmployeeList(){
return list;
}

public Employee(){}

public Employee(String empid,String empName,int salary,int age){
this.empid = empid;
this.empName = empName;
this.salary = salary;
this.age = age;
}

public String getEmpid() {
return empid;
}

public void setEmpid(String empid) {
this.empid = empid;
}

public String getEmpName() {
return empName;
}

public void setEmpName(String empName) {
this.empName = empName;
}

public int getSalary() {
return salary;
}

public void setSalary(int salary) {
this.salary = salary;
}

public int getAge() {
return age;
}

public void setAge(int age) {
this.age = age;
}


@Override
public int hashCode() {
// TODO Auto-generated method stub
return Objects.hashCode(empid,empName);
}


@Override
public boolean equals(Object obj) {
if (this == obj) {
return true;
}
if (!(obj instanceof Employee)) {
return false;
}
Employee emp = (Employee) obj;
return Objects.equal(this.empid, emp.empid) && Objects.equal(this.empName, emp.empName);
}


@Override
public String toString() {
return Objects.toStringHelper(this).add("empid", empid).add("empName", empName).toString();
}


@Override
public int compareTo(Employee o) {
// ComparisionChain
return ComparisonChain.start().compare(empid, o.empid)
.compare(empName, o.empName)
.compare(salary, o.salary)
.compare(age, o.age)
.result();
}


}

Collection2 is the utility class provided by the Gauva API which has transfrom() method with input collection and function implementation and return the transofrmed collection.

/**
 * 
 */
package com.rajkrrsingh.test.guava;

import java.util.Collection;
import java.util.Iterator;

import com.google.common.base.Function;
import com.google.common.collect.Collections2;

/**
 * @author rks
 * @04-Jan-2015
 */
public class GuavaFunctionDemo {

	public static void main(String[] args) {
		printEmployeeInUpperCase();
	}

	// Transformation of one collection to the other collection using Guava function
	public static void printEmployeeInUpperCase(){
		Collection<String> upperCaseEmpName = Collections2.transform(Employee.getEmployeeList(),new Function<Employee, String>() {

			@Override
			public String apply(Employee emp) {
				if(emp != null)
					return emp.getEmpName().toUpperCase();
				return "";

			}
		});

		Iterator<String> itr = upperCaseEmpName.iterator();
		while(itr.hasNext()){
			System.out.println(itr.next());
		}
	}

}

after running the class you will find all your collection element are in the uppercase.
RKS
DEREK
JACK
NICK

Monday, September 22, 2014

Hadoop Configuration Parameters revisited

fs.default.name specifies the default filesystem

fs.checkpoint.dir used by secondary namenode to store filesystem metadata during checkpoint operation

fs.trash.interval specifies the no of minutes the file will be available in the .Trash before final deletion

topology.script.file.name absolute path of the script to make cluster rack aware

hadoop.log.dir The directory in which log data should be written. This should be the same path as specified in HADOOP_LOG_DIR in the hadoop-env.sh file.

io.file.buffer.size (core-site.xml) general purpose buffer size to enhance read/write IO and network IO

dfs.block.size specifies default block size to store on HDFS

dfs.name.dir specifies a comma separated directories to store namenode metadata

dfs.data.dir list of directories where datanodes will store HDFS block data

dfs.datanode.du.reserved disk space reserved for the non HDFS use

dfs.namenode.handler.count count of worker thread to process RPC request by clients as well as other cluster deamon

dfs.datanode.failed.volumes.tolerated specifies the number of disks that are permitted to die before failing the entire datanode

dfs.hosts list of hostname or datanode that are allowed to communicate with the namenode.

dfs.host.exclude for decommisioning the datanode or to block the host to communicate with the namenode

dfs.permissions.supergroup specify group of user whose privileges equivalent to the super user

dfs.balance.bandwidthPerSec use by datanode to limit the bandwidth

mapred.job.tracker specifies the job tracker hostname and port

mapred.local.dir mapReduce job use the machine’s local disk to store their intermediate output to the specified directories

mapred.java.child.opts specifies the jvm heap properties like initial heap size,max heap size etc.

mapred.child.ulimit it a limit on how much virtual memory a process may consume before it is terminated.

mapred.tasktracker.map.tasks.maximum maximum no of map task can be supported by the workeer node in parallel

mapred.tasktracker.reduce.tasks.maximum maximum no of reduce task can be supported by the workeer node in parallel

io.sort.mb specifies the size of circular buffer to have intermediate key-value pair emitted by the mapper

io.sort.factor specifies the number of files/streams to merge at once

mapred.compress.map.output true/false depending on whether you want to compress the mapper emitted data

mapred.map.output.compression.codec specifies the codec that you want to use to compress the intermediate data

mapred.output.compression.type RECORD/BLOCK level compression

mapred.job.tracker.handler.count jobtracker maintains a pool of worker thread to handle RPC requests

mapred.jobtracker.taskScheduler The mapred.jobtracker.taskScheduler parameter specifies the Java class name of the scheduler plugin that should be used by the jobtracker

mapred.reduce.parallel.copies which controls the number of copies each reduce task initiates in parallel during the shuffle phase

mapred.reduce.tasks control the no of reduce tasks

tasktracker.http.threads no of threads avaiable to handle http request concurrently

mapred.reduce.slowstart.completed.maps indicates when to begin allocating reducers as a percentage of completed map tasks

mapred.acls.enabled Access control lists must be globally enabled prior to use