@Deprecated public class ColumnFamilyOutputFormat extends org.apache.hadoop.mapreduce.OutputFormat<java.nio.ByteBuffer,java.util.List<Mutation>> implements org.apache.hadoop.mapred.OutputFormat<java.nio.ByteBuffer,java.util.List<Mutation>>
ColumnFamilyOutputFormat acts as a Hadoop-specific
OutputFormat that allows reduce tasks to store keys (and corresponding
values) as Cassandra rows (and respective columns) in a given
ColumnFamily.
As is the case with the ColumnFamilyInputFormat, you need to set the
Keyspace and ColumnFamily in your
Hadoop job Configuration. The ConfigHelper class, through its
ConfigHelper.setOutputColumnFamily(org.apache.hadoop.conf.Configuration, java.lang.String) method, is provided to make this
simple.
For the sake of performance, this class employs a lazy write-back caching mechanism, where its record writer batches mutations created based on the reduce's inputs (in a task-specific map), and periodically makes the changes official by sending a batch mutate request to Cassandra.
| Modifier and Type | Field and Description |
|---|---|
static java.lang.String |
BATCH_THRESHOLD
Deprecated.
|
static java.lang.String |
QUEUE_SIZE
Deprecated.
|
| Constructor and Description |
|---|
ColumnFamilyOutputFormat()
Deprecated.
|
| Modifier and Type | Method and Description |
|---|---|
protected void |
checkOutputSpecs(org.apache.hadoop.conf.Configuration conf)
Deprecated.
|
void |
checkOutputSpecs(org.apache.hadoop.fs.FileSystem filesystem,
org.apache.hadoop.mapred.JobConf job)
Deprecated.
|
void |
checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context)
Deprecated.
Check for validity of the output-specification for the job.
|
static Cassandra.Client |
createAuthenticatedClient(java.lang.String host,
int port,
org.apache.hadoop.conf.Configuration conf)
Deprecated.
Connects to the given server:port and returns a client based on the given socket that points to the configured
keyspace, and is logged in with the configured credentials.
|
org.apache.hadoop.mapreduce.OutputCommitter |
getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
Deprecated.
The OutputCommitter for this format does not write any data to the DFS.
|
org.apache.cassandra.hadoop.ColumnFamilyRecordWriter |
getRecordWriter(org.apache.hadoop.fs.FileSystem filesystem,
org.apache.hadoop.mapred.JobConf job,
java.lang.String name,
org.apache.hadoop.util.Progressable progress)
Deprecated.
|
org.apache.cassandra.hadoop.ColumnFamilyRecordWriter |
getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
Deprecated.
Get the
RecordWriter for the given task. |
static void |
login(java.lang.String user,
java.lang.String password,
Cassandra.Client client)
Deprecated.
|
public static final java.lang.String BATCH_THRESHOLD
public static final java.lang.String QUEUE_SIZE
public org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
throws java.io.IOException,
java.lang.InterruptedException
getOutputCommitter in class org.apache.hadoop.mapreduce.OutputFormat<java.nio.ByteBuffer,java.util.List<Mutation>>context - the task contextjava.io.IOExceptionjava.lang.InterruptedExceptionpublic void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context)
checkOutputSpecs in class org.apache.hadoop.mapreduce.OutputFormat<java.nio.ByteBuffer,java.util.List<Mutation>>context - information about the jobprotected void checkOutputSpecs(org.apache.hadoop.conf.Configuration conf)
@Deprecated
public void checkOutputSpecs(org.apache.hadoop.fs.FileSystem filesystem,
org.apache.hadoop.mapred.JobConf job)
throws java.io.IOException
checkOutputSpecs in interface org.apache.hadoop.mapred.OutputFormat<java.nio.ByteBuffer,java.util.List<Mutation>>java.io.IOExceptionpublic static Cassandra.Client createAuthenticatedClient(java.lang.String host, int port, org.apache.hadoop.conf.Configuration conf) throws java.lang.Exception
host - fully qualified host name to connect toport - RPC port of the serverconf - a job configurationjava.lang.Exception - set of thrown exceptions may be implementation defined,
depending on the used transport factorypublic static void login(java.lang.String user,
java.lang.String password,
Cassandra.Client client)
throws java.lang.Exception
java.lang.Exception@Deprecated
public org.apache.cassandra.hadoop.ColumnFamilyRecordWriter getRecordWriter(org.apache.hadoop.fs.FileSystem filesystem,
org.apache.hadoop.mapred.JobConf job,
java.lang.String name,
org.apache.hadoop.util.Progressable progress)
getRecordWriter in interface org.apache.hadoop.mapred.OutputFormat<java.nio.ByteBuffer,java.util.List<Mutation>>public org.apache.cassandra.hadoop.ColumnFamilyRecordWriter getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
throws java.lang.InterruptedException
RecordWriter for the given task.getRecordWriter in class org.apache.hadoop.mapreduce.OutputFormat<java.nio.ByteBuffer,java.util.List<Mutation>>context - the information about the current task.RecordWriter to write the output for the job.java.lang.InterruptedExceptionCopyright © 2018 The Apache Software Foundation