Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split Enclosed JSON #71

Open
randallwhitman opened this issue Nov 18, 2014 · 1 comment
Open

Split Enclosed JSON #71

randallwhitman opened this issue Nov 18, 2014 · 1 comment

Comments

@randallwhitman
Copy link
Contributor

Currently, Enclosed JSON - i.e. a file of complete valid Esri REST JSON - is treated as non-splittable; in other words, if used as an input to a MapReduce job, the whole file will be processed by a single Mapper.

It would be possible to skip the opening enclosing brace and the metadata section outside the "features" section, and otherwise the RecordReader could be similar to UnenclosedJsonRecordReader. In fact, if the Enclosed JSON is restricted to the convention of the metadata preceding the "features", likely only small differences from UnenclosedJsonRecordReader would suffice.

Collaborators do not have information as to how much Enclosed JSON is used as MapReduce input. The idea will be set aside, and can potentially pursued if use cases are presented.

@randallwhitman
Copy link
Contributor Author

@komamitsu FYI, in case this is of interest.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant