-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RDB Shredder: align PostgresConstraints with atomic.events 0.10.0 #103
Comments
Hey @alexanderdean, from my POV this is blocking whole PII enrichment feature. Should it be a part of R29? |
Prioritized to R29. |
This was referenced May 29, 2018
Closed
chuwy
changed the title
RDB Shredder: align PostgresConstraints with latest atomic events
RDB Shredder: align PostgresConstraints with atomic.events 0.10.0
May 29, 2018
chuwy
added a commit
that referenced
this issue
May 29, 2018
chuwy
added a commit
that referenced
this issue
May 29, 2018
chuwy
added a commit
that referenced
this issue
May 29, 2018
chuwy
added a commit
that referenced
this issue
May 29, 2018
Total 10 constraints were widened, which means Shredder probably silently truncated several other fields (less important though). |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This should be happening automatically. I believe we should have a new issue where we generate this somehow from the atomic events schema, which can be found here:
https://github.com/snowplow/snowplow/blob/master/4-storage/redshift-storage/sql/atomic-def.sql
This has caused temporary (recoverable) loss of data after the atomic.events field lengths were changed.
snowplow-rdb-loader/shredder/src/main/scala/com.snowplowanalytics.snowplow.storage/spark/utils/PostgresConstraints.scala
Line 18 in 2343b1d
The text was updated successfully, but these errors were encountered: