…#385)

Co-authored-by: Willem Pienaar <6728866+woop@users.noreply.github.com>

…ackoff (feast-dev#417)

Rather than the Maven protobuf plugin running on the same symlinked
definitions in several Java modules, localize this process into one
module that the others depend on.

This provides a single module that can be depended on by third-party
extensions with the bare minimum of dependencies.

Also removes proto files that are no longer used.

* Add PR template

* Remove line about not using fixes

* Add notes about CLA and release notes

* Initial commit of changelog up to 0.4.3

* Remove unreleased changes on master

* Add missing changelog manually

Co-authored-by: Khor Shu Heng <32997938+khorshuheng@users.noreply.github.com>

…on (feast-dev#421)

Bumps [hibernate-validator](https://github.com/hibernate/hibernate-validator) from 6.0.13.Final to 6.1.0.Final.
- [Release notes](https://github.com/hibernate/hibernate-validator/releases)
- [Changelog](https://github.com/hibernate/hibernate-validator/blob/master/changelog.txt)
- [Commits](hibernate/hibernate-validator@6.0.13.Final...6.1.0.Final)

Signed-off-by: dependabot[bot] <support@github.com>

This forward-ports a straggling commit from feast-dev#407: it was missed when
initially creating the datatypes module because Sonatype publishing
setup was added concurrently.

…ets (feast-dev#432)

…v#434)

* Add documentation for bigquery batch retrieval

* Fix formatting for multiline comments

Allow log level to be set via environmental variable.
Add ability to set appender type in serving.
Remove logback-classic from ingestion as it is a library so should not bring its own impl.
Upgrade log4j to 2.12.1 to support objectMessageAsJsonObject.
Fix logger config targeting feast package in serving an add same concept for core.

* Add presence, shape and domain fields to EntitySpec and FeatureSpec
These are copied from definition in Schema.proto in tensorflow data validation

* Add presence,shape and domain fields to Field class

* Add retry options to BigQuery

Add two fields to job properties and use them to set retry options in all BigQuery job waitFor calls, as per Google example.

* Change timeout defaults and add config to e2e test to fix failure.

* Change job service connection to use a connection pool

* Close connection

* Remove isConnected condition

* Add simple test to test that pool recovers broken connections

* Catch JedisExceptions separately

* Catch only JedisConnectionException

* Allow users not to set max age for batch retrieval

* Fix typo in test assertion

* Update Changelog

* Remove closed issues

* Deduplicate example notebooks

* Merge docker-compose.yml for both batch and online serving.

* Update README.md and remove versions from Helm Charts

* Fix type in README.md

* Upgrade to Java 11

* Add support for Java 1.8 in Maven enforcer

Co-authored-by: David Heryanto <david.heryanto@hotmail.com>

Move prometheus-statsd-exporter to toggleable core dependency (default false).
Add ingresses for gRPC and HTTP for both core and serving.
Refactor ConfigMaps to user Spring profiles rather than manipulating the base application.yaml.
Add ability to define and enable arbitrary Spring profiles.
Add toggle to enable prometheus scraping in core.
Add parameters to change LOG_LEVEL and LOG_TYPE (feast-dev#430).
Add parameter to specify GOOGLE_CLOUD_PROJECT.
Allow jar path to be specified (e.g. if using non-standard image).
Add missing documentation for Helm parameters.

…#473)

* Use Java 8 SDK for branch 4.0 and Java 11 for master

* Add Java 8 tests for branch v3.0

* Semantic versioning for publish java sdk

* Fix branch filter refex

…v#471)

* Make redis key creation more determinisitic (feast-dev#380)

* Add documentation to RedisKey in Redis.proto
Ensure entities are sorted by the name

Co-authored-by: David Heryanto <david.heryanto@hotmail.com>

Update v0.4.4 changelog to be consistent with the release

* Use bzip2 compressed feature set json as pipeline option

* Make decompressor and compressor more generic and extensible

* Avoid code duplication in test

…feast-dev#481)

* Fail formatting check before tests execute

By default, the spotless Maven plugin binds its check goal to the verify
phase (late in the lifecycle, after integration tests). Because we
currently only run `mvn test` for CI, it doesn't proceed as far as
verify so missed formatting is not caught by CI.

This binds the check to an earlier phase, in between test-compile and
test, so that it will fail before `mvn test` but not disrupt your dev
workflow of compiling main and test sources as you work. This strikes a
good compromise on failing fast for code standards without being _too_
nagging.

For the complete lifecycle reference, see:
https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html

* Apply spotless formatting

…ast-dev#490)

* Fix fastavro version used in feast to 0.22.9

* Print python packages version used when e2e test fails

…s to StatsD (feast-dev#486)

* Extend WriteMetricsTransform to write feature value stats to StatsD

* Apply mvn spotless

* Catch all exception not just StatsDClientException during init
Since there are other exception like UnknownHostException that can be thrown and we want to know such error. Also change the log level to error because so it's not normal for client to fail to be created"

* Change log level due to invalid feature set ref to error (previously warn)
On 2nd thought, this should constitute an error not a warning

* Apply maven spotless to metric transform codes

* Allow use of secure gRPC in Feast Python client.

* Add tests for secure gRPC in Python client.

So that it is consistent with the actual unit of timing being measured
And recommended metric names in Prometheus https://prometheus.io/docs/practices/naming/#metric-names

* Replacing Jedis With Lettuce in ingestion and serving

* Removing extra lines

* Abstacting redis connection based on store

* Check the connection before connecting as lettuce does the retry automatically

* Running spotless

* Throw Exception if the job store config is null

* Handle No enum constant RuntimeException

* Replacing Jedis With Lettuce in ingestion and serving

* Removing extra lines

* Abstacting redis connection based on store

* Check the connection before connecting as lettuce does the retry automatically

* Running spotless

* Throw Exception if the job store config is null

* Handle No enum constant RuntimeException

* Future should be cleared everytime sync is called

* Add configuration for multiprocessing so tests can run on mac

* Set windows to use spawn as well

…east-dev#511)

* Make dependency:analyze run clean on datatypes-java

* Remove stale dependencies from ingestion

Unused according to `mvn -pl ingestion dependency:analyze`, and tests.

We had a recent bump of hibernate-validator with a CVE fix (feast-dev#421) that I
was looking to backport, and it turns out it's not used anymore anyway.

Co-authored-by: Joost Rothweiler <=>

…nstraints are updated (feast-dev#512)

* add feast serving grpc calls metrics

* edited documentation to reflect correct class

* separated into two metrics

* linting updates

* Encode feature row before storing in Redis

* Include encoding as part of RedisMutationDoFn

Co-authored-by: Khor Shu Heng <khor.heng@gojek.com>

Please see this issue for more details feast-dev#527

…east-dev#534)

This seems to make the test pass more deterministically
If this value is higher than the one used for sending output (50ms) some messages may be lost
leading to failed test

)

* Update base Docker image for building Feast Serving image
- Add clean phase before packaging for more deterministic build (in case host directory is dirty)
- Move the downloading of grpc-health-probe in Feast Serving to build stage so the production stage does not need extra tools like wget, for slimmer production image.

* Update base Docker image for Feast Serving in Dockerfile.dev

* create cassandra store for registration and ingestion

  * Downgraded Guava to 25
    * Beam 2.16 uses Cassandra 3.4.0 (So we cannot use Cassandra 4.x which shades Guava)
    * Cassandra 3.4.0 uses Guava version 16.0 but has a compatibility check to use a different class when we use version > 19.0.
    * Guava version 26 (version previously used) has breaking change to method used in compatibility check in Cassandra's dependency, hence version 25
  * Using Cassandra's internal field 'writetime' to handle out of order arrivals. When older records where the primary key already exist in Cassandra are ingested, they are set as tombstones in Cassandra and ignored on retrieval.
    * Aware that this way of handling out of order arrival is specific to Cassandra, but until we have a general way to handle out of order arrivals we need to do it this way
  * Cassandra's object mapper requires stating table's name along with @table annotation
    * table_name is still part of CassandraConfig for use in serving module
    * if user registers CassandraConfig with a different table name other than "feature_store", this will throw an exception

* add cassandra serving service

  * Abstracted OnlineServingService for common implementation of online serving stores
  * Complete tests remain in RedisServingServiceTest while Cassandra tests only contain basic tests for writes, and some other implementation specific to Cassandra

* update documentation to reflect current API and add cassandra store to docs

* add default expiration to cassandra config for when featureset does not have max age

* docs update, spotless check, and bug fix on cassandra schema

* create cassandra store for registration and ingestion

  * Downgraded Guava to 25
    * Beam 2.16 uses Cassandra 3.4.0 (So we cannot use Cassandra 4.x which shades Guava)
    * Cassandra 3.4.0 uses Guava version 16.0 but has a compatibility check to use a different class when we use version > 19.0.
    * Guava version 26 (version previously used) has breaking change to method used in compatibility check in Cassandra's dependency, hence version 25
  * Using Cassandra's internal field 'writetime' to handle out of order arrivals. When older records where the primary key already exist in Cassandra are ingested, they are set as tombstones in Cassandra and ignored on retrieval.
    * Aware that this way of handling out of order arrival is specific to Cassandra, but until we have a general way to handle out of order arrivals we need to do it this way
  * Cassandra's object mapper requires stating table's name along with @table annotation
    * table_name is still part of CassandraConfig for use in serving module
    * if user registers CassandraConfig with a different table name other than "feature_store", this will throw an exception

* add cassandra serving service

  * Abstracted OnlineServingService for common implementation of online serving stores
  * Complete tests remain in RedisServingServiceTest while Cassandra tests only contain basic tests for writes, and some other implementation specific to Cassandra

* update documentation to reflect current API and add cassandra store to docs

* add default expiration to cassandra config for when featureset does not have max age

* docs update, spotless check, and bug fix on cassandra schema

Pr Comments and using ttl for cassandra job service

1. Still unresolved is why the CassandraServingServiceITTest is failing,
I'm thinking its because I'm not mocking the feature request correctly

* Ordering keys in ingestion and serving

* Notracing

* Versionless serving option

* Adding in tracing

* Make nulls not update feature values