org.apache.metamodel.util

Class HdfsResource

java.lang.Object

org.apache.metamodel.util.AbstractResource

org.apache.metamodel.util.HdfsResource

All Implemented Interfaces:

Serializable, HasName, Resource

public class HdfsResource
extends AbstractResource
implements Serializable

A Resource implementation that connects to Apache Hadoop's HDFS distributed file system.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static String	SCHEME_EMRFS
static String	SCHEME_FTP
static String	SCHEME_HDFS
static String	SCHEME_MAPRFS
static String	SCHEME_S3
static String	SCHEME_SWIFT
static String	SYSTEM_PROPERTY_HADOOP_CONF_DIR_ENABLED

Constructor Summary

Constructors

Constructor and Description
HdfsResource(String url) Creates a HdfsResource
HdfsResource(String hostname, int port, String filepath) Creates a HdfsResource using the "hdfs" scheme
HdfsResource(String url, String hadoopConfDir) Creates a HdfsResource
HdfsResource(String scheme, String hostname, int port, String filepath, String hadoopConfDir) Creates a HdfsResource

Method Summary

Modifier and Type	Method and Description
OutputStream	append() Opens up an OutputStream to append to the resource.
boolean	equals(Object obj)
String	getFilepath()
String	getHadoopConfDir()
org.apache.hadoop.conf.Configuration	getHadoopConfiguration()
org.apache.hadoop.fs.FileSystem	getHadoopFileSystem()
org.apache.hadoop.fs.Path	getHadoopPath()
String	getHostname()
long	getLastModified() Gets the last modified timestamp value (measured in milliseconds since the epoch (00:00:00 GMT, January 1, 1970)) of the resource, if available.
String	getName() Gets the name of the resource, typically a filename or other identifying string
int	getPort()
String	getQualifiedPath() Gets the qualified path of the resource, which typically includes slash or backslash separated nodes in a hierarical tree structure.
String	getScheme()
long	getSize() Gets the size (in number of bytes) of this resource's data.
int	hashCode()
boolean	isExists() Determines if the resource referenced by this object exists or not.
boolean	isReadOnly() Determines if the file is read only, or if writes are also possible.
InputStream	read() Opens up an InputStream to read from the resource.
OutputStream	write() Opens up an OutputStream to write to the resource.

Methods inherited from class org.apache.metamodel.util.AbstractResource

append, read, read, read, toString, write

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

SYSTEM_PROPERTY_HADOOP_CONF_DIR_ENABLED

public static final String SYSTEM_PROPERTY_HADOOP_CONF_DIR_ENABLED

See Also:

Constant Field Values

SCHEME_HDFS

public static final String SCHEME_HDFS

See Also:

Constant Field Values

SCHEME_SWIFT

public static final String SCHEME_SWIFT

See Also:

Constant Field Values

SCHEME_EMRFS

public static final String SCHEME_EMRFS

See Also:

Constant Field Values

SCHEME_MAPRFS

public static final String SCHEME_MAPRFS

See Also:

Constant Field Values

SCHEME_S3

public static final String SCHEME_S3

See Also:

Constant Field Values

SCHEME_FTP

public static final String SCHEME_FTP

See Also:

Constant Field Values

Constructor Detail

HdfsResource

public HdfsResource(String url)

Creates a HdfsResource

Parameters:

url - a URL of the form: scheme://hostname:port/path/to/file

HdfsResource

public HdfsResource(String url,
                    String hadoopConfDir)

Creates a HdfsResource

Parameters:

url - a URL of the form: scheme://hostname:port/path/to/file

hadoopConfDir - the path of a directory containing the Hadoop and HDFS configuration file(s).

HdfsResource

public HdfsResource(String hostname,
                    int port,
                    String filepath)

Creates a HdfsResource using the "hdfs" scheme

Parameters:

hostname - the HDFS (namenode) hostname

port - the HDFS (namenode) port number

filepath - the path on HDFS to the file, starting with slash ('/')

HdfsResource

public HdfsResource(String scheme,
                    String hostname,
                    int port,
                    String filepath,
                    String hadoopConfDir)

Creates a HdfsResource

Parameters:

scheme - the scheme to use (consider using SCHEME_HDFS or any of the other "SCHEME_" constants).

hostname - the HDFS (namenode) hostname

port - the HDFS (namenode) port number

filepath - the path on HDFS to the file, starting with slash ('/')

hadoopConfDir - the path of a directory containing the Hadoop and HDFS configuration file(s).

Method Detail

getScheme

public String getScheme()

getFilepath

public String getFilepath()

getHostname

public String getHostname()

getPort

public int getPort()

getHadoopConfDir

public String getHadoopConfDir()

getName

public String getName()

Description copied from interface: Resource

Gets the name of the resource, typically a filename or other identifying string

Specified by:

getName in interface HasName

Specified by:

getName in interface Resource

getQualifiedPath

public String getQualifiedPath()

Description copied from interface: Resource

Gets the qualified path of the resource, which typically includes slash or backslash separated nodes in a hierarical tree structure.

Specified by:

getQualifiedPath in interface Resource

Returns:

isReadOnly

public boolean isReadOnly()

Description copied from interface: Resource

Determines if the file is read only, or if writes are also possible.

Specified by:

isReadOnly in interface Resource

Returns:

isExists

public boolean isExists()

Description copied from interface: Resource

Determines if the resource referenced by this object exists or not.

Specified by:

isExists in interface Resource

Returns:

getSize

public long getSize()

Description copied from interface: Resource

Gets the size (in number of bytes) of this resource's data. An approximated number is allowed. If the size is not determinable without actually reading through the whole contents of the resource, -1 is returned.

Specified by:

getSize in interface Resource

Returns:

getLastModified

public long getLastModified()

Description copied from interface: Resource

Gets the last modified timestamp value (measured in milliseconds since the epoch (00:00:00 GMT, January 1, 1970)) of the resource, if available. If the last modified date is not available, -1 is returned.

Specified by:

getLastModified in interface Resource

Returns:

write

public OutputStream write()
                   throws ResourceException

Description copied from interface: Resource

Opens up an OutputStream to write to the resource. Consumers of this method are expected to invoke the OutputStream.close() method manually. If possible, the other write(...) method is preferred over this one, since it guarantees proper closing of the resource's handles.

Specified by:

write in interface Resource

Returns:

Throws:

ResourceException

append

public OutputStream append()
                    throws ResourceException

Description copied from interface: Resource

Opens up an OutputStream to append to the resource. Consumers of this method are expected to invoke the OutputStream.close() method manually. If possible, the other append(...) method is preferred over this one, since it guarantees proper closing of the resource's handles.

Specified by:

append in interface Resource

Returns:

Throws:

ResourceException

read

public InputStream read()
                 throws ResourceException

Description copied from interface: Resource

Opens up an InputStream to read from the resource. Consumers of this method are expected to invoke the InputStream.close() method manually. If possible, the other read(...) methods are preferred over this one, since they guarantee proper closing of the resource's handles.

Specified by:

read in interface Resource

Returns:

Throws:

ResourceException

getHadoopConfiguration

public org.apache.hadoop.conf.Configuration getHadoopConfiguration()

getHadoopFileSystem

public org.apache.hadoop.fs.FileSystem getHadoopFileSystem()

getHadoopPath

public org.apache.hadoop.fs.Path getHadoopPath()

hashCode

public int hashCode()

Overrides:

hashCode in class Object

equals

public boolean equals(Object obj)

Overrides:

equals in class Object