Skip to content
This repository has been archived by the owner on Jul 9, 2021. It is now read-only.

[SQOOP-3002] sqoop merge tool composite merge-key #26

Open
wants to merge 2 commits into
base: trunk
Choose a base branch
from
Open

[SQOOP-3002] sqoop merge tool composite merge-key #26

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cd1e840
sqoop merge tool composite merge-key
kevin00chen Aug 22, 2016
498bbdb
sqoop merge tool composite merge-key
kevin00chen Aug 22, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion src/java/org/apache/sqoop/mapreduce/MergeMapperBase.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,18 @@ protected void processRecord(SqoopRecord r, Context c)
if (null == fieldMap) {
throw new IOException("No field map in record " + r);
}
Object keyObj = fieldMap.get(keyColName);
Object keyObj = null;
if (keyColName.contains(",")) {
String connectStr = new String(new byte[]{1});
StringBuilder keyFieldsSb = new StringBuilder();
for (String str : keyColName.split(",")) {
keyFieldsSb.append(connectStr).append(fieldMap.get(str).toString());
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for example one table has two column, a and b

Field a Field b
a+ b
a +b

when use "+" to connect two field, two record will has same keyObj.
To avoid this i use a String contains one byte.

}
keyObj = keyFieldsSb;
} else {
keyObj = fieldMap.get(keyColName);
}

if (null == keyObj) {
throw new IOException("Cannot join values on null key. "
+ "Did you specify a key column that exists?");
Expand Down