New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Select Copier using yaml file #55

Closed

yashrajrs opened this issue Apr 9, 2018 · 1 comment

Labels

Contributor

yashrajrs commented Apr 9, 2018 •

edited

Loading

Currently the copier is decided based on the location paths.
So if the replication is from s3 to s3 https://github.com/HotelsDotCom/circus-train/blob/master/circus-train-s3-s3-copier/src/main/java/com/hotels/bdp/circustrain/s3s3copier/S3S3Copier.java is used.

I wish to use for https://github.com/HotelsDotCom/circus-train/blob/master/circus-train-s3-mapreduce-cp-copier/src/main/java/com/hotels/bdp/circustrain/s3mapreducecpcopier/S3MapReduceCpCopier.java for s3 to s3 replication.

Two possible solutions:-
1.) add a config for picking the copier by class name

2.) add a general config like “preferMR”: true and S3S3Copier returns false if that’s true

yashrajrs mentioned this issue

Issue 55: Select Copier using yaml file #57

Merged

ddcprg added the enhancement label

Contributor

ddcprg commented Apr 11, 2018

Merged in #57

ddcprg closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment