Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add run= to timestamped ETL folder names #294

Closed
alexanderdean opened this issue Jul 1, 2013 · 2 comments
Closed

Add run= to timestamped ETL folder names #294

alexanderdean opened this issue Jul 1, 2013 · 2 comments
Assignees
@alexanderdean
Labels
3. Enrich
Milestone
Version 0.8.8

Comments

@alexanderdean
Copy link
Member

So e.g. instead of:

snowplow-events-soak-eu/events/2013-06-06-13-54-02

Becomes:

snowplow-events-soak-eu/events/run=2013-06-06-13-54-02

This is, again, to allow Hive to read these event folders. Necessary for anyone using Hive who is not also loading events into Redshift or another storage target (because their events won't be being archived).
@ghost ghost assigned alexanderdean Jul 1, 2013
@alexanderdean alexanderdean mentioned this issue Jul 1, 2013
Closed
alexanderdean added a commit that referenced this issue Jul 7, 2013
@alexanderdean
EmrEtlRunner: removed :snowplow: section from config.yml.sample (#289)
46fdcd3 
EmrEtlRunner: simplified EmrEtlRunner and its config (#287)
EmrEtlRunner: added run= to timestamped ETL folder names (#294)
@alexanderdean
Copy link
Member Author

Unfortunately we're waiting on Scalding for a new release here:

@alexanderdean alexanderdean mentioned this issue Jul 17, 2013
Closed
@alexanderdean
Copy link
Member Author

Done in 0.8.8.

peel pushed a commit to snowplow/emr-etl-runner that referenced this issue May 26, 2020
@alexanderdean
Removed :snowplow: section from config.yml.sample (snowplow/snowplow#289
be36ffc 
)

Simplified EmrEtlRunner and its config (snowplow/snowplow#287)
Added run= to timestamped ETL folder names (snowplow/snowplow#294)
peel pushed a commit to snowplow/emr-etl-runner that referenced this issue May 28, 2020
@alexanderdean
Removed :snowplow: section from config.yml.sample (snowplow/snowplow#289
c05bc67 
)

Simplified EmrEtlRunner and its config (snowplow/snowplow#287)
Added run= to timestamped ETL folder names (snowplow/snowplow#294)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@alexanderdean alexanderdean

Labels
3. Enrich
Projects
None yet
Milestone
Version 0.8.8
Development

No branches or pull requests
1 participant
@alexanderdean