Release 3.3.0

Hopsworks 3.3.0

Bug

HWORKS-224 hopsworks python SDK opensearch link broken

HWORKS-267 After fail checkout, branch is empty on pull

HWORKS-269 Don't show the same in file status and in UI table

HWORKS-309 Bugs in searching

HWORKS-319 User active projects not updated

HWORKS-331 Hopsworks metrics not showing after payara restart

HWORKS-374 opensearch_api.get_default_py_config() returns public IP of host, should return consul fqdn

HWORKS-379 LDAP users cannot download certificates

HWORKS-386 Spark job does not accept space in arguments

HWORKS-397 Links to Model Serving API reference are broken

HWORKS-459 Audit log caller should be consistent

HWORKS-469 Dependent conda commands not handled correctly in case of failure

HWORKS-475 hopsworks Python library can't connect in sklearn legacy deployments

HWORKS-492 Servings logs are not shown in Kibana

HWORKS-493 Request batching configuration is broken in the deployment creation form

HWORKS-496 Exception not raised if other than duplicated deployment entry

HWORKS-498 Transformer resources in dict format are not properly deserialized

HWORKS-499 Support for jupyter notebooks as predictor scripts

HWORKS-501 Collect logs in Flink UI and distribute custom libraries

HWORKS-521 Resolve ruby test failures

HWORKS-524 HSML ignores knative_domain parameter

HWORKS-525 Python resource monitor for K8s counts all nodes

HWORKS-528 Agent users should never be blocked

HWORKS-533 Nightly jenkins pipeline reporting 500 failures even though we have only 40 tests

HWORKS-539 Storage initializer broken after release of requests 2.29.0

HWORKS-540 Bump guava version to 27.0-jre

HWORKS-541 update CVE suppression - hops updated dependencies

HWORKS-558 Old UI getting 404 trying to get dela client

HWORKS-561 Hopsworks Kafka Authorizer does not allow IDEMPOTENT_WRITE operations

HWORKS-562 Serving hashcode method should handle null cases

HWORKS-563 Bump commons-configuration2 in expat

HWORKS-566 User account creation fails if the serving api key is not available within 3 seconds

HWORKS-568 Expat fails to start with logging class not found exception

HWORKS-569 Filebeat cannot list nodes which is required for scraping container logs

HWORKS-570 Flatbuffer class not found exception when running arrow internal spark

HWORKS-577 Upgrade 3.2 to 3.3 failing on kzookeeper::default

HWORKS-599 Execution status is not being updated

HWORKS-600 Python library installation failure on non DAS nodes

HWORKS-602 HA cluster could not connect to live logs, rm UI, spark UI, monitor, onlineFS

HWORKS-604 Collect glassfish node logs in logstash

HWORKS-605 GitTimeoutCommandsMonitor not working in HA

HWORKS-606 Spark and Flink configuration files are not present in Glassfish config nodes

HWORKS-608 Git commit message might be bigger than VARCHAR(1000)

HWORKS-609 Alertmanager config missing route.repeat_interval

HWORKS-610 HazelcastSerializationException on JupyterJWTManager

HWORKS-611 ApiKey filter should check if user is blocked

HWORKS-612 hops-hadoop-chef should automatically set HopsYARN HA attributes

HWORKS-614 hopsworks.login on a project with shared feature store

HWORKS-616 Upgrade fails on fuse_mnt

HWORKS-619 Python jobs are always set to FAILED even after completion

HWORKS-622 hsml connection fails to close

HWORKS-624 Race condition in LibraryInstaller CondaCommand execution

HWORKS-625 Kafka storage connector fails migration if inode is null

HWORKS-634 Revisit RonDB restore process in Chef recipes

HWORKS-638 Kafka 3 removed --zookeeper option

HWORKS-639 Template cert store passwords in onlinefs-kafka.properties

HWORKS-643 Revoke Glassfish internal certificate when restoring a backup

HWORKS-653 Publish hopsworks-service-discovery module on archiva

HWORKS-654 extra constraints migration does not respect the order of the columns

HWORKS-655 elkadmin certificate is not generated when restoring a backup

HWORKS-657 Setting current_version attribute with patch version will template all the DDL SQL files

HWORKS-660 Consul recipe should install bc package

HWORKS-661 Incorrect glassfish certificates after upgrade to HA environment

HWORKS-663 remote_file does not support not_if condition

HWORKS-666 Configuring HA Glassfish fails when importing certificates on worker nodes

HWORKS-670 Set Flink job staging and checkpoint directories per user

HWORKS-674 Feature store job migration doesn't work if the spark user is not spark

Task

HWORKS-217 Alerting needs to update alertmanager config in local file system

HWORKS-218 Fix upload for clustered payara

HWORKS-225 Create Deployment Groups in payara5 (clustering)

HWORKS-226 python api should throw informative error if user tries to download a folder

HWORKS-258 Add hopsworks python sdk dataset upload/download to workflow testing

HWORKS-266 Checking out a branch that exist on remote repo does not work

HWORKS-284 Documentation to export cluster logs

HWORKS-314 Add support for git fetch and reset

HWORKS-409 cleaner timer bean are too verbose and pollute logs

HWORKS-415 Add project name to the audit logs

HWORKS-422 Exclude old versions of the documentation from being indexed by search engines

HWORKS-428 Add sleep to loadtest to avoid running into issue due to parallelizing test

HWORKS-437 Upgrade Kafka to version 3.4.0

HWORKS-446 Fix programmatic timers for HA

HWORKS-447 Merge ObjectMapper singletons

HWORKS-448 Fix singletons with concurrency management

HWORKS-449 Fix cache singletons for HA

HWORKS-452 Fix persistent timers for HA

HWORKS-486 Remove hdfs_inodes foreign key from Project table

HWORKS-487 Remove hdfs_inodes foreign key from Dataset table

HWORKS-491 Replace hsml with hopsworks in the code snippets shown in the UI

HWORKS-494 Support local script path in model.deploy()

HWORKS-503 Remove inode foreign key from git repositories

HWORKS-508 Remove DML undo files

HWORKS-510 Archive hops-util Java

HWORKS-511 Add number of GPUs to Python resource usage

HWORKS-512 Docker cgroup parent should be configurable

HWORKS-515 Remove inode foreign key from feature store code

HWORKS-516 Integration test VMs not shut down properly when test is aborted or fails early

HWORKS-520 Make oauth claims configurable

HWORKS-523 Remove inode foreign key from transformation_function table

HWORKS-526 Remove inode foreign key from validation_report table

HWORKS-527 Fix Jupyter JWT manager for HA

HWORKS-529 Upgrade base environment to Python 3.10

HWORKS-530 Kafka advertise listeners should use consul domain name

HWORKS-531 Memory alerts for HMS and HS2

HWORKS-536 Move master encryption password to KMS

HWORKS-542 Docker Library Installer refactor

HWORKS-548 Remove inode foreign key from training datasets

HWORKS-551 UserDTO polluting logs

HWORKS-552 Add org.apache.commons:commons-pool2:jar:2.6.2 to Spark classpath

HWORKS-555 Jupyter on Hopsfs-mount

HWORKS-556 Remove bijection-avro_2.12 - CVE-2017-15288

HWORKS-557 Send alert through Alertmanager if project creation fails

HWORKS-560 Add an alert when approaching the maximum number of shards in a cluster

HWORKS-564 Bump Grafana version to address security vulnerabilities

HWORKS-565 Hopsworks api dataset upload should support parallel chunk upload

HWORKS-573 Only set python engine in project.get_feature_store() if connected to app

HWORKS-574 Remove hdfs_inodes FK from hdfs_command_execution

HWORKS-575 Remove reusing load/workflow test environment from run script

HWORKS-584 Metrics exception due to uninitialised executor pool

HWORKS-585 Change the download url for maven in karamel-chef

HWORKS-586 Upgrade great expectations to 0.14.13

HWORKS-587 Remove inode foreign key from feature views

HWORKS-588 Remove inode foreign key from external feature groups

HWORKS-592 Documentation for enabling hopsfs mount for Jupyter

HWORKS-603 Switch to using libmamba solver in Anaconda

HWORKS-607 Remove inode foreign key for statistics

HWORKS-620 Bump Hops version to 3.2.0.10-RC0

HWORKS-623 Bump hops-git dev version to 0.5.0

HWORKS-628 Skip project creation failure alert if a project with the same name exists

HWORKS-648 Change default ports for the alertmanager

HWORKS-649 consul should log to a file instead of stdout/systemd

HWORKS-650 onlinefs should not log to stdout

HWORKS-651 Airflow should not log to stdout

HWORKS-658 Hops certificate localization directory should not be at /tmp

HWORKS-659 PID files should not be under /tmp

HWORKS-664 Hive temporary directories should be under java.io.tmpdir and not under /tmp

HWORKS-667 MySQL X plugin socket should not be at /tmp

HWORKS-668 Release new version of Karamel

Feature Store

Bug

FSTORE-647 GCP connector path issue

FSTORE-683 Editing BigQuery (and maybe other connectors) It's not possible to set arguments field to ‘null’/‘undefined’/[]

FSTORE-719 Kafka connectors with plaintext as security protocol can't be retrieved by hsfs

FSTORE-752 Online type consistency issue

FSTORE-798 Beam integration tutorial improvement

FSTORE-800 Kafka Topic creation times out after 3 seconds

FSTORE-801 Onlinefs should not try and write rows to tables that have been deleted

FSTORE-812 Enable Kafka topic auto creation

FSTORE-814 StackOverflowException if you provide statistics columns in the statistics config

FSTORE-816 Add Kafka Datasource Jar file to spark deployment

FSTORE-843 Online-enabled External Feature Groups are offline after .save

FSTORE-844 Shade avro dependency in hsfs

FSTORE-846 Feature Group commit_details fails in certain cases

FSTORE-847 Spark Pandas UDFs not working on 3.1 and non-cloud installations

FSTORE-853 Flyingduck HSFS Client: Support Features without Featuregroup

FSTORE-854 Flyginduck HSFS Client: Support datetime objects for date filters

FSTORE-857 Appending to an online feature group fails after upgrade

FSTORE-863 Spark integration documentation doesn't have correct hive metastore jar option

FSTORE-864 Downgrade avro version in hsfs to 1.8.2

FSTORE-890 Statistics are not filtered properly for row percentages other than 1

FSTORE-894 mkdocs-minify-plugin missing from setup.py

FSTORE-902 Recomputing training dataset statistics should overwrite existing statistics

FSTORE-910 OnlineFS set NaN to null

FSTORE-912 Use a recursive approach to infer schema of pyarrow struct

FSTORE-917 Update metadata for expectation suite expect decamelize dict

FSTORE-918 Ensure alert are sent only once

FSTORE-919 Dataframe row upload to kafka progress bar is broken on hsfs master

FSTORE-922 Python join deserialization reads join_type instead of type

FSTORE-929 Enable self join in REST API

FSTORE-934 Flying Duck / HSFS - JVM Error Investigation

FSTORE-936 Canary NPE when ingesting

FSTORE-955 get_batch_data failed when there is no training dataset

FSTORE-962 Topics for feature store do not account for default replication configuration

FSTORE-963 feature group expectation_suite setter doesn't handle properly ExpectationSuite type objects

FSTORE-965 fastavro fails to build when installing HSFS

Subtask

FSTORE-808 Statistics API new schemas, CRUD and filtering

FSTORE-859 Loadtest and integration tests for Statistics API

FSTORE-868 Support for Feature View statistics based on both commit times and event times

Design Log

FSTORE-625 Design Log for FM persistence

FSTORE-733 Write initial design document for feature store REST API

Task

FSTORE-544 Flying Duck v1

FSTORE-574 streaming feature computation pipeline example using Flink

FSTORE-577 Add options to AWS S3 storage connector

FSTORE-626 Boilerplate FM persistence

FSTORE-674 Modify project and FG alert to add new status for FM

FSTORE-675 Hopsworks backend for FM alert

FSTORE-684 Job scheduling PoC

FSTORE-690 Setup Online Feature Store Benchmark with Locust

FSTORE-712 Add Feature Monitoring Job to FsJobManagerController

FSTORE-713 Add endpoint to get feature monitoring config by name + enable/disable

FSTORE-721 Sketch prototype run_feature_monitoring job

FSTORE-736 Export Consumer Group status from onlinefs to prometheus for onlinfs health monitoring

FSTORE-737 Show archived commits in Feature Group activity page

FSTORE-740 Support for Date type columns as primary key on online storage

FSTORE-744 Flying Duck - HSFS Integration

FSTORE-748 Flying duck: Monitoring

FSTORE-749 Move OnlineFS Kafka offsets in RonDB

FSTORE-750 Feature flag for Feature Monitoring

FSTORE-751 Flink support for HSFS java client

FSTORE-768 Integrate scheduler with Feature monitoring service

FSTORE-769 Input Validation and Exception in Feature Monitoring

FSTORE-771 UX proposal for python client feature monitoring

FSTORE-778 Add more type hint to hsfs

FSTORE-779 Simplify OnlineFS

FSTORE-802 Resolve OnlineFS memory consumption

FSTORE-815 Auto-reset Feature Group Kafka offset after topic recreation/upgrade/restore

FSTORE-817 Integrate alerts in feature monitoring

FSTORE-821 Add feature store name to feature store table for REST API

FSTORE-829 Test ASOF Joins in DuckDB

FSTORE-835 Investigation: HSFS Pandas 2.0 Compatability

FSTORE-836 HSFS "Querying..." Printline

FSTORE-837 Flyingduck Karamel Integration

FSTORE-838 Asnychronous fg.insert

FSTORE-841 Relax confluent-kafka dependency to support python 3.10

FSTORE-848 Add support for transformation function to Feature Monitoring

FSTORE-849 Support Spine Dataframes for batch data and training datasets

FSTORE-850 DBT tutorial with BigQuery

FSTORE-851 Increase Kafka List Topics Timeout in HSFS

FSTORE-852 Increase Flyingduck Python Version

FSTORE-855 HSFS Client: Query object .get_feature(), features that returns all features, not just the left side of join

FSTORE-858 Pandas 2.0 Compatibility

FSTORE-861 hsfs client should fetch external broker address for spark/pyspark

FSTORE-867 End to end workflows feature monitoring.

FSTORE-869 Fix typo data validation doc

FSTORE-871 Round event_time to hour in monitoring

FSTORE-873 Add fetch dataset stats support based on specific id + bug fix

FSTORE-874 Notebook feature monitoring demo

FSTORE-875 Design Log External FG Access in Python/Flyingduck

FSTORE-886 Persist feature view statistics computed on both transformed and untransformed data

FSTORE-889 Documentation for Flink/Beam compute engines and feature engineering

FSTORE-893 Get hopsworks certificates for external Flink client

FSTORE-911 Failed to ingest nullable int/long data

FSTORE-913 Add support for MEDIUM/LONG BLOB/TEXT

FSTORE-914 Test HA Hackathon Ticket

FSTORE-915 Flying duck not being used for non-admin users

FSTORE-931 Add support for creating feature group metadata through save method

FSTORE-935 add return type to get_feature_vector(s)

FSTORE-945 Bump OnlineFS version to 1.6

FSTORE-947 Tutorials Streamlit Fix

FSTORE-953 Add feature store REST API doc