Placeholder for adding flatten transformation #19
Assignees
Comments
alexanderdean
changed the title
alexanderdean
added a commit
to snowplow/kinesis-tee
that referenced
this issue
Sep 1, 2016
ninjabear
pushed a commit
to snowplow/kinesis-tee
that referenced
this issue
Sep 12, 2016
Release 0.1.0 Update README.md Update README.md WIP initial project setup Update README.md Update README.md Update README.md WIP skeleton WIP add javascript filtering Add Snowplow Scala analytics SDK v0.1.0 Enforce JDK8 requirement for build Classloader foo for nashorn in sbt see: sbt/sbt#1214 dependency gymnastics WIP Snowplow Enriched event transform WIP snowplow enriched event addition using snowplow scala analytics sdk WIP add avro config WIP fixup avro schema WIP fixup avro schema, add test (non self describing) WIP start self describing avro WIP self describing Avro for configuration & tests WIP tee function WIP tee function tests Fixup filter test / js filter strategy Add support for getting configuration string from DDB WIP add ingress code skeleton WIP wire in routing strategy Add tests/code for interface with AWS Lambda Add tests/code for routing Add filter base64 decoding & tests Add transformation test Add stream writer using Amazon KPL Added builtIn bit /cc @ninjabear Renamed to NESTED_JSON to make clear it's not flattened JSON, per snowplow/snowplow-scala-analytics-sdk#19 /cc @ninjabear Convert KPL streamwriter to standard KCL putRecord Add config test with a TargetAccount specified Use putRecord & if no target account specified - use the region that the lambda is in Add travis build Fix filter so that returning 'true' ensures record is passed through to stream WIP travis test Silently swallow transform exceptions Add Bintray credentials to .travis.yml (close #12) Remove 'source' section from tee config Rename SNOWPLOW_TO_JSON to SNOWPLOW_TO_NESTED_JSON Update readme to remove source section/builtin Gordon skeleton Add CI/CD Add nested builtIn to transformer in configuration Update readme to point to wiki Ensure copyright notices appear in all files Scaladoc Add changelog Ensure filtering errors are consumed
ninjabear
added a commit
to snowplow/kinesis-tee
that referenced
this issue
Oct 3, 2016
Release 0.1.0 Update README.md Update README.md WIP initial project setup Update README.md Update README.md Update README.md WIP skeleton WIP add javascript filtering Add Snowplow Scala analytics SDK v0.1.0 Enforce JDK8 requirement for build Classloader foo for nashorn in sbt see: sbt/sbt#1214 dependency gymnastics WIP Snowplow Enriched event transform WIP snowplow enriched event addition using snowplow scala analytics sdk WIP add avro config WIP fixup avro schema WIP fixup avro schema, add test (non self describing) WIP start self describing avro WIP self describing Avro for configuration & tests WIP tee function WIP tee function tests Fixup filter test / js filter strategy Add support for getting configuration string from DDB WIP add ingress code skeleton WIP wire in routing strategy Add tests/code for interface with AWS Lambda Add tests/code for routing Add filter base64 decoding & tests Add transformation test Add stream writer using Amazon KPL Added builtIn bit /cc @ninjabear Renamed to NESTED_JSON to make clear it's not flattened JSON, per snowplow/snowplow-scala-analytics-sdk#19 /cc @ninjabear Convert KPL streamwriter to standard KCL putRecord Add config test with a TargetAccount specified Use putRecord & if no target account specified - use the region that the lambda is in Add travis build Fix filter so that returning 'true' ensures record is passed through to stream WIP travis test Silently swallow transform exceptions Add Bintray credentials to .travis.yml (close #12) Remove 'source' section from tee config Rename SNOWPLOW_TO_JSON to SNOWPLOW_TO_NESTED_JSON Update readme to remove source section/builtin Gordon skeleton Add CI/CD Add nested builtIn to transformer in configuration Update readme to point to wiki Ensure copyright notices appear in all files Scaladoc Add changelog Ensure filtering errors are consumed Add Vagrant environment (closes #15) Fix Avro schema path (closes #16) Ensure memory and timeout are set on lambda
ninjabear
added a commit
to snowplow/kinesis-tee
that referenced
this issue
Oct 3, 2016
Release 0.1.0 Update README.md Update README.md WIP initial project setup Update README.md Update README.md Update README.md WIP skeleton WIP add javascript filtering Add Snowplow Scala analytics SDK v0.1.0 Enforce JDK8 requirement for build Classloader foo for nashorn in sbt see: sbt/sbt#1214 dependency gymnastics WIP Snowplow Enriched event transform WIP snowplow enriched event addition using snowplow scala analytics sdk WIP add avro config WIP fixup avro schema WIP fixup avro schema, add test (non self describing) WIP start self describing avro WIP self describing Avro for configuration & tests WIP tee function WIP tee function tests Fixup filter test / js filter strategy Add support for getting configuration string from DDB WIP add ingress code skeleton WIP wire in routing strategy Add tests/code for interface with AWS Lambda Add tests/code for routing Add filter base64 decoding & tests Add transformation test Add stream writer using Amazon KPL Added builtIn bit /cc @ninjabear Renamed to NESTED_JSON to make clear it's not flattened JSON, per snowplow/snowplow-scala-analytics-sdk#19 /cc @ninjabear Convert KPL streamwriter to standard KCL putRecord Add config test with a TargetAccount specified Use putRecord & if no target account specified - use the region that the lambda is in Add travis build Fix filter so that returning 'true' ensures record is passed through to stream WIP travis test Silently swallow transform exceptions Add Bintray credentials to .travis.yml (close #12) Remove 'source' section from tee config Rename SNOWPLOW_TO_JSON to SNOWPLOW_TO_NESTED_JSON Update readme to remove source section/builtin Gordon skeleton Add CI/CD Add nested builtIn to transformer in configuration Update readme to point to wiki Ensure copyright notices appear in all files Scaladoc Add changelog Ensure filtering errors are consumed Add Vagrant environment (closes #15) Fix Avro schema path (closes #16) Ensure memory and timeout are set on lambda
|
With #53 its very easy to add these custom transformation. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This would be similar to how we flatten cosmetically nested objects (e.g.
order.customer.billingAddress.lineOne) for Redshift in Snowplow.The text was updated successfully, but these errors were encountered: