资源简介

最近在研究hadoop与mapReduce,网上教程只有个wordcount程序示范,太简单,故写了个相对复杂点的涉及到多个文件之间同时运算的矩阵乘法的代码用于实验与测试,上传供大家学习与参考。 调用方法: 执行:hadoop jar matrix.jar com.baosight.mapred.test.MatrixMulti left:/tmp/a right:/tmp/b output:/tmp/c leftprefix:000000_ rightprefix:000000_ >参数1:left:左矩阵路径 >参数2:right:右矩阵路径 >参数3:output:左矩阵乘右矩阵结果路径 >参数4:leftprefix:左矩阵文件前缀 如/tmp/a/000000_0,000000_1,000000_2……的000000_ >参数5:rightprefix:右矩阵文件前缀 >左右矩阵按列存储,每个文件只存放一列值,如2x2的单位矩阵, 000000_0存放1 0 000000_1存放 0 1

资源截图

代码片段和文件信息

/**
 * Copyright 2012 宝信软件 解决方案事业部
 * 
 * @author 樊后礼   f_houli@163.com    fanhouli@baosight.com
 */
package com.baosight.mapred.test;

import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;

import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.WritableComparable;

public class MapInputKey implements WritableComparable{

public int leftColNum = 0;
public int rightColNum = 0;

public MapInputKey(){}
@Override
public void write(DataOutput out) throws IOException {
out.writeInt(leftColNum);
out.writeInt(rightColNum);

}
@Override
public void readFields(DataInput in) throws IOException {
leftColNum = in.readInt();
rightColNum = in.readInt();

}
@Override
public int compareTo(object o) {
MapInputKey other = (MapInputKey) o;
if (this.leftColNum == other.leftColNum
&& this.rightColNum == other.rightColNum) {
return 0;
} else {
return 1;
}
}

public boolean equals(object o) {
    if (!(o instanceof MapInputKey))
      return false;
    MapInputKey other = (MapInputKey)o;
    return (this.leftColNum == other.leftColNum && this.rightColNum == other.rightColNum);
  }
}

 属性            大小     日期    时间   名称
----------- ---------  ---------- -----  ----
     文件        1197  2012-06-27 01:19  src\com\baosight\mapred\test\MapInputKey.java
     文件         787  2012-06-27 01:20  src\com\baosight\mapred\test\MapInputValue.java
     文件        2738  2012-06-27 01:20  src\com\baosight\mapred\test\MatrixInputFormat.java
     文件        2769  2012-06-27 01:20  src\com\baosight\mapred\test\MatrixInputRecordReader.java
     文件        1641  2012-06-27 01:20  src\com\baosight\mapred\test\MatrixInputSplit.java
     文件        1347  2012-06-27 01:20  src\com\baosight\mapred\test\MatrixMapper.java
     文件        2702  2012-06-27 01:20  src\com\baosight\mapred\test\MatrixMulti.java
     文件         947  2012-06-27 01:20  src\com\baosight\mapred\test\MatrixReducer.java
     文件        2932  2012-06-27 01:20  src\com\baosight\mapred\test\ReduceInputKey.java
     文件         867  2012-06-27 01:20  src\com\baosight\mapred\test\ReduceOutputValue.java
     目录           0  2012-06-27 01:29  src\com\baosight\mapred\test\
     目录           0  2012-06-27 01:09  src\com\baosight\mapred\
     目录           0  2012-06-27 01:09  src\com\baosight\
     目录           0  2012-06-27 01:09  src\com\
     目录           0  2012-06-12 07:52  src\

评论

共有 条评论