WebHDFS REST API -- Overview

Hadoop support all the hdfs operations using the underline java implementaion for all the HDFS commands like ls, mkdir, cat, rm, merge etc.
most of the time need arises to access the HDFS from some external applications other than the accessing the Hadoop cluster, the external system can be in any programming language other than the Java.

To support the access of HDFS from external application hadoop provides the WebHDFS REST API, which is based on the commonn http methods like GET,PUT,POST,DELETE. these methods supports the user operations like OPEN, GETFILESTATUS, LISTSTATUS are using HTTP GET, others like CREATE, MKDIRS, RENAME, SETPERMISSIONS are relying on HTTP PUT. APPEND operations is based on HTTP POST, while DELETE is using HTTP DELETE on the HDFS.


WebHDFS REST API OVERVIEW

• HTTP GET
  
  • 
    OPEN
    (see FileSystem.open)
    
  • 
    GETFILESTATUS
    (see FileSystem.getFileStatus)
    
  • 
    LISTSTATUS
    (see FileSystem.listStatus)
    
  • 
    GETCONTENTSUMMARY
    (see FileSystem.getContentSummary)
    
  • 
    GETFILECHECKSUM
    (see FileSystem.getFileChecksum)
    
  • 
    GETHOMEDIRECTORY
    (see FileSystem.getHomeDirectory)
    
  • 
    GETDELEGATIONTOKEN
    (see FileSystem.getDelegationToken)
    
• HTTP PUT
  
  • 
    CREATE
    (see FileSystem.create)
    
  • 
    MKDIRS
    (see FileSystem.mkdirs)
    
  • 
    RENAME
    (see FileSystem.rename)
    
  • 
    SETREPLICATION
    (see FileSystem.setReplication)
    
  • 
    SETOWNER
    (see FileSystem.setOwner)
    
  • 
    SETPERMISSION
    (see FileSystem.setPermission)
    
  • 
    SETTIMES
    (see FileSystem.setTimes)
    
  • 
    RENEWDELEGATIONTOKEN
    (see DistributedFileSystem.renewDelegationToken)
    
  • 
    CANCELDELEGATIONTOKEN
    (see DistributedFileSystem.cancelDelegationToken)
    
• HTTP POST
  
  • 
    APPEND
    (see FileSystem.append)
    
• HTTP DELETE
  
  • 
    DELETE
    (see FileSystem.delete)
    

In order to configure the WebHDFS, update hdfs-site.xml as follows

<property>
           <name>dfs.webhdfs.enabled</name>
           <value>true</value>
        </property>

now restart your namenode to access the WebHDFS.

in the omming post I will post about the java client to access the WebHDFS REST API... stay tuned