我的映射器需要发送以下元组:
<custID,prodID,rate>
我想将 custID 作为键发送给缩减程序,并将 prodID 和费率作为值一起发送给缩减程序,因为它们是缩减阶段所需要的。
哪种方法最好?
public void map(Object key, Text value, Context context)
throws IOException, InterruptedException {
String[] col = value.toString().split(",");
custID.set(col[0]);
data.set(col[1] + "," + col[2]);
context.write(custID, data);
}
public void reduce(Text key, Iterable<Text> values, Context context)
throws IOException, InterruptedException {
for (Text val : values) {
String[] temp = val.toString().split(",");
Text rate = new Text(temp[1]);
result.set(rate);
context.write(key, result);
}
}
最好的方法是编写 CustomWritables
这是为了双重价值。您可以将其更改为文本或字符串
import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;
import org.apache.hadoop.io.Writable;
/**
* @author Unmesha SreeVeni U.B
*
*/
public class TwovalueWritable implements Writable {
private double first;
private double second;
public TwovalueWritable() {
set(first, second);
}
public TwovalueWritable(double first, double second) {
set(first, second);
}
public void set(double first, double second) {
this.first = first;
this.second = second;
}
public double getFirst() {
return first;
}
public double getSecond() {
return second;
}
@Override
public void write(DataOutput out) throws IOException {
out.writeDouble(first);
out.writeDouble(second);
}
@Override
public void readFields(DataInput in) throws IOException {
first = in.readDouble();
second = in.readDouble();
}
/* (non-Javadoc)
* @see java.lang.Object#hashCode()
*/
@Override
public int hashCode() {
final int prime = 31;
int result = 1;
long temp;
temp = Double.doubleToLongBits(first);
result = prime * result + (int) (temp ^ (temp >>> 32));
temp = Double.doubleToLongBits(second);
result = prime * result + (int) (temp ^ (temp >>> 32));
return result;
}
/* (non-Javadoc)
* @see java.lang.Object#equals(java.lang.Object)
*/
@Override
public boolean equals(Object obj) {
if (this == obj) {
return true;
}
if (obj == null) {
return false;
}
if (!(obj instanceof TwovalueWritable)) {
return false;
}
TwovalueWritable other = (TwovalueWritable) obj;
if (Double.doubleToLongBits(first) != Double
.doubleToLongBits(other.first)) {
return false;
}
if (Double.doubleToLongBits(second) != Double
.doubleToLongBits(other.second)) {
return false;
}
return true;
}
@Override
public String toString() {
return first + "," + second;
}
}
从映射器中你可以将其发出为
context.write(key,new TwovalueWritable(prodID,rate));
希望这可以帮助。
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)