Update dependency com.hazelcast:hazelcast to v5 #2448
Conversation
![renovate[bot]](https://avatars.githubusercontent.com/in/2740?s=60&v=4){kind=small}
|
Kudos, SonarCloud Quality Gate passed!
|
|
Code Climate has analyzed commit 8d2a3bd and detected 0 issues on this pull request. View more on Code Climate. |
|
Test summary
Run details
View run in Cypress Dashboard ➡️ This comment has been generated by cypress-bot as a result of this project's GitHub integration settings. You can manage this integration in this project's settings in the Cypress Dashboard |
|
Kudos, SonarCloud Quality Gate passed!
|
Codecov Report
@@ Coverage Diff @@
## master #2448 +/- ##
=========================================
Coverage 75.45% 75.45%
Complexity 428 428
=========================================
Files 122 122
Lines 1972 1972
Branches 60 60
=========================================
Hits 1488 1488
Misses 473 473
Partials 11 11 Continue to review full report at Codecov.
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
|
Kudos, SonarCloud Quality Gate passed!
|
This PR contains the following updates:
4.2.2->5.0Release Notes
hazelcast/hazelcast
v5.0Compare Source
This document lists the new features, enhancements, fixed issues and, removed
or deprecated features for Hazelcast 5.0 release. The numbers in the square
brackets refer to the issues in Hazelcast's GitHub repository.
Hazelcast 5.0 is a major release which unifies
the former Hazelcast IMDG and Jet products in a
single solution. The changes are summarized below.
For information about upgrading from previous IMDG and Jet
releases, see the Upgrade chapter in https://docs.hazelcast.com/hazelcast/5.0/.
To learn about the changes in previous IMDG and Jet releases, see https://docs.hazelcast.org/docs/rn/ and
https://jet-start.sh/blog/.
New Features
Jet Streaming Engine: Hazelcast now incorporates the streaming engine, which was formerly and separately known as Hazelcast Jet.
SQL Engine: We have added the support of following operators to our SQL engine:
** INSERT
** UPDATE
** DELETE
** ORDER BY
** GROUP BY
** JOIN
** CASE
See https://docs.hazelcast.com/hazelcast/5.0/sql/sql-overview.html.
Related issue and PRs: [#18940], [#18910], [#18898], [#18734],
[#18663], [#18569], [#18479], [#18483], [#18422], [#18067]
Persistence: This allows individual members and whole clusters to recover
faster by persisting map entries and JCache data on disk. Members can use persisted data to recover during restarts.
See https://docs.hazelcast.com/hazelcast/5.0/storage/persistence.html.
Compact Serialization Format (Preview Feature): We have introduced a new serialization format
(
CompactSerializer), which is in its beta stage currently. This format requires muchless storage space, provides better latency and throughput, and supports more extensive type sets:
** It supports schema evolution of a class and does not require any version to be specified explicitly.
** On common use cases where the field types of an object are natively supported via this compact format,
Java DTO classes are automatically serialized without requiring a serializer (
CompactSerializer) to be implemented.** Your classes do not need to implement an interface like
Portable. Once you enable the compact serialization,the classes not matching to any other serialization methods are serialized in this new format.
See the https://docs.hazelcast.com/hazelcast/5.0/serialization/compact-serialization.html.
Related PR: [#19017]
Security for Jet Engine Jobs: The Jet engine allows you to upload custom code as well as access data
outside of Hazelcast such as files on the device that's running a member. You can secure your Hazelcast member
against malicious uses of the Jet API. See https://docs.hazelcast.com/hazelcast/5.0/pipelines/job-security.html.
Breaking Changes
API Changes
For Google Cloud Storage (GCS) file sources, the
SecuredFunctioninterface now returns a list of permissions instead of a singlepermission. Previously, Hazelcast was returning only the permission for the bucket to be read, and this was causing a source to be
possibly exploited for gaining information whether some file exists in the system. Now, Hazelcast also returns a permission
for the keyfile that is required to read a bucket on GCS. [#19407]
The "Hot Restart Persistence" feature name has been changed to "Persistence"; with this change
we've performed various cleanups in its API:
** The
HotRestartServicehas been deprecated; see <<removed-deprecated-features, Deprecated Features>>.** Renamed
com.hazelcast.hotrestart.BackupTaskascom.hazelcast.persistence.BackupTask.** Renamed
com.hazelcast.hotrestart.BackupTaskStateascom.hazelcast.persistence.BackupTaskState.** Added the
isBackupEnabled()toPersistenceService/HotRestartServicethat doesnot have a default implementation.
[#19404]
After the unification of cloud discovery plugins (see the Cloud Discovery Plugins section below),
the exception class for the failure in generating
SSLSocketFactoryis changed fromKubernetesClientExceptiontoRestClientExceptionfor the Kubernetes plugin. [#19132]Changed the type of
YEARfield in the SQL client protocol fromshorttoint. [#18984]Since the
HOSTandRACKmetadata were deprecated in the previous Hazelcastrelease, these information are removed from the
ZONE_AWAREpartition groupingconfiguration. [#18780]
Since DAG printing was not reflecting the real queue size, the
DAG.toDotString(int defaultParallelism)methodsignature is changed as
DAG.toDotString(int defaultLocalParallelism, int defaultQueueSize). You, now, need to supply thequeue size that will be shown if it is not overriden on the edge. [#18475]
Configuration Changes
is now part of Hazelcast 5.0:
**
jet.home**
jet.imdg.version.mismatch.check.disabled[#18999]
Enhancements
Distribution
While the full one includes Hazelcast and all of its modules along with SQL,
Jet engine extensions and Management Center, the slim one only includes the Hazelcast
runtime, default configurations and scripts.
See https://docs.hazelcast.com/hazelcast/5.0/migrate/upgrading-from-imdg-4.html#changes-in-distribution-packaging
for details. [#18990], [#18989]
See https://docs.hazelcast.com/hazelcast/5.0/migrate/upgrading-from-imdg-4.html#changes-in-distribution-packaging. [#18999]
hazelcast-sql,hazelcast-sql-coreandhazelcast-jet-sqlMaven modules in the distributionshave been merged into a single
hazelcast-sqlmodule as a part of the Hazelcast 5.0 distribution. [#18584]jackson-coredependency frompom.xmlsince it wasbreaking the extensions that depend on
jackson. [#18665]https://github.com/hazelcast/hazelcast-code-samples repository. [#475]
The Jet and SQL Engine
mappingininformation_schema.mappingsandtableininformation_schema.columns.This inconsistency has been fixed by renaming
mappingastable. [#19210]CONCAT_WSfunction, which takes variable sizedVARCHAR(minimum 3) and returns aVARCHARthat consists of theconcatenation of the arguments except the first one using the first argument as a separator. [#19094]
FROMclauseso that you can submit queries like
SELECT rand()without this clause. [#19030]when you upgrade your Hazelcast version since the Jet engine doesn't provide backwards compatibility. [#19012]
during a partition migration, this processor searches all the migrated partitions on all
available cluster members. [#18968]
putIfAbsentAsync()method for maps on the member side;which is required for the usage of
INSERT INTOstatements in SQL queries. [#18946]deserialize them, which enables you to use
Portablein client/serverdeployments without touching the server side; for example, SQL queries
can now return columns without having the class on the server-side classpath. [#18922]
TIMEandTIMESTAMPtemporal formats for theSQL engine: You can now use
TIMEwithout leading zeroes andTIMESTAMPwith space instead of the
Tsymbol. Also added support of leading non-zerocharacters for the
DATEformats. [#18881], [#18842]OFFSETfor SQL queries. [#18866]IdentifiedDataSerializablefor SQL schema objects. [#18851]sincetags in Jet engine API and its extension modulesfrom
@since x.yto@since Jet x.y. [#18832]OnHeapMapScanPclass to read the Hazelcast maps directlyby the SQL engine. [#18685]
so that number of records accumulated by it can be limited
to avoid out of memory failures. You can use the
max-processor-accumulated-recordsconfiguration element for this purpose. [#18671]
QueryDataType.MAPandQueryDataTypeFamily.MAPto support map operand checksfor file table functions. [#18602]
EXTRACT(field FROM source)for the SQL engine.The function computes date parts from the source field. The supported types for
sourceargument are as follow:** Date
** Time
** Timestamp
** Timestamp With Time Zone
[#18570]
LIMIT <n>andORDER BYclauses for the streaming engine. [#18479]**
REPLACE**
ATAN2**
POWER**
SQUARE**
SQRT**
CBRT**
POSITION**
COALESCE**
NULLIF**
TO_EPOCH_MILLIS**
TO_TIMESTAMP_TZ[#18900], [#18856], [#18510], [#18487], [#18450], [#18424], [#18405]
the SQL engine. [#18390]
Portabletypes for the SQL engine. [#18115]INandBETWEENoperators for the SQL queries. [#18483], [#18422], [#18067]Data Structures
while calculating the size of map; this was causing latencies and performance issues
as the entries in a Replicated Map grows. The related
size()methodhas been refactored to eliminate the aforementioned situation. [#19005]
Cloud Discovery Plugins
that points to the given Hazelcast pod. If there are multiple services pointing to one pod,
then the discovery could not work or might have chosen the wrong service. The following changes
have been made to address this:
** Added label-based filtering for the Kubernetes Service per pod.
** Added matching service and pod by name (if there are multiple services per pod is configured,
the priority takes a service with the same name as the pod, before it was a random service.
** Added resolving load balancer service if "hostname" is defined.
[#19168]
plugins' in their own Github repos have been moved into the
hazelcast/hazelcastrepo. Their documentation also has been merged and unified into https://docs.hazelcast.com/hazelcast/5.0/deploy/deploying-in-cloud.html. [#19132]
into the
hazelcast/hazelcastGithub repository askubernetes-rbac.yaml. [#19093]Serialization
which has been necessary for non-Java clients to use these:
**
LocalDate**
LocalTime**
LocalDateTime**
OffsetDatetime[#18983]
Security
hazelcast-security-hardened.yaml) focusedon hardened security to the distribution packages; it lists configuration options with their
descriptions which may help securing your Hazelcast deployment. [#18843]
their assigned roles stored together with other Hazelcast configurations. [#18948]
See the YAML example:
hazelcast:
security:
enabled: true
realms:
- name: simpleRealm
authentication:
simple:
users:
- username: test
password: 'a1234'
roles:
- monitor
- hazelcast
- username: root
password: 'secret'
roles:
- admin
Configuration
when your cluster has a map or cache whose persistence is enabled; this is to improve
a single member recovery from a crash, and it does not have a high memory overhead. [#19502]
expose-externallyconfiguration parameter for objects that expose an external IP addressIn. Kubernetes. See https://docs.hazelcast.com/hazelcast/5.0/deploy/configuring-kubernetes.html
for its description.
JetPropertiesare now merged intoClusterProperty.Also added the
hazelcastprefix to the former Jet property names, e.g.,jet.job.scan.periodhas becomehazelcast.jet.job.scan.periodand the former one is deprecated. [#19146]Jet engine jobs. See https://docs.hazelcast.com/hazelcast/5.0/configuration/jet-configuration.html#enabling-resource-uploads for details.
the map is configured to have the the in-memory format as
OBJECT, Hazelcast now can storeportables as
PortableGenericRecordand still query themwithout needing to convert them to Object/Data. [#18891]
**
hazelcast.partition.rebalance.mode: It determines whether clustermembership change triggers partition rebalancing automatically (
auto) orexplicit action is required for rebalancing to occur (
manual). Its default isauto.**
hazelcast.partition.rebalance.delay.seconds: it specifies the time in secondsto wait before triggering automatic partition
rebalancing after a member leaves the cluster unexpectedly. Unexpectedly in this
context means that a member leaves the cluster by programmatic termination, a
process crash or network partition. Its default is 0, which means rebalancing is
triggered immediately.
[#18425]
Other Enhancements
hz-cliand Hazelcast detectshazelcastor
hazelcast-enterpriseis packaged in it. [#19512]queueFillPercentmetric to show how full the WAN replication queue is, in percentage. [#19431]hazelcast/hazelcastGitHub repository has been completelyrewritten to reflect the unification of former Hazelcast IMDG and Jet products. [#19061]
hazelcast-sqlmodule is now covered by the Hazelcast Community License; before,it was Apache License, Version 2. [#18957]
interface; you can now use the
hazelcast consolecommand to startthe client console application. [#18857]
getPartitionGroupStrategy()method to have cluster membersas arguments so that useful partitioning strategies can be implemented by accessing
the members using this method. [#18794]
Previously, it was represented as the value of
Long.MAX_VALUE. [#18642]now a thread local array controls the allocations for these tasks otherwise
which may cause increased garbage collection pressure and CPU usage spikes when
you use aggressive expiration configurations, e.g., low time-to-live values. 18633[#18633]
for maps and caches to prevent out of memory failures. [#18499]
See https://docs.hazelcast.com/hazelcast/5.0/list-of-metrics.html
for the full list of metrics with their descriptions. [#17880]
removing the unnecessary
sleepstatements in the code. [#17428]Fixes
Fixed a possible serialization error while submitting a Jet engine job when the member and client sides
have different Hazelcast versions with compatible APIs. [#19534]
Fixed an issue where running the
map.clear/cache.clearmethods was evictingall entries in all Near Caches of all maps, not only in the requested map/cache. [#19523]
Fixed an issue where the wildcard configuration mechanism was not working
correctly if a matching pattern has the same prefix and suffix. [#19357]
If the connector permission for file includes wildcard (*) then any file in the system could be
read by using
..in the path in connector. See the below example:Then one can read a file like
readFrom(FileSources.files("/home/user/workspace/../some_secure_file").This has been fixed by converting the file path to canonical path for file permissions.
If two clusters with different cluster names run locally and both of them has enabled security,
then a Hazelcast client was ignoring the configured cluster names and connecting to any of them;
a check has been put to eliminate this issue. [#19344]
Fixed an issue where a high amount of garbage collection pressure was occurring
during repartitioning especially when having a high partition count. [#19312]
Fixed an issue where the MultiMap operation statistics were not being
updated after these operations are called from client. [#19296]
Fixed an issue where the
hz-cli submitscript was not working properly withrelative path: if the script is called from a different directory (like
./bin/hz-cli), thebindirectory wastaken as root for the relative path instead of the directory from where the script is called. [#19204]
Fixed an issue where
ElasticSearchdid not have a client method that allowsHTTPS connections; added a new client with HTTP and HTTPS schemes. [#19139]
SQL expressions now does not fail when used with trailing semicolons. [18976]
Fixed an issue where the health monitor was incorrectly showing the value for
free metadata memory. [#18951]
Some merge policies like
LatestUpdateMergePolicyfor the map and WAN replicationconfigurations require the per-entry statistics to be enabled. Previously, this
configuration inconsistency was causing the related member to fail at runtime.
Now, the Hazelcast member fails to start, i.e., fast fails, in such a case. [#18928]
Fixed an issue where the maximum size policy for a map was being ignored
when the policy is
PER_NODEand the cluster is scaled down (due to losing or killing a member). [#18927]The LRU eviction policy now takes last access time value into account to
prevent premature removal of the lately added but not yet accessed map entries.[#18909]
Fixed an issue where the map���s Near Cache was setting its maximum
size as 10.000 even if the configured eviction policy is
NONE. [#18835]Fixed a regression issue where a job using map reader/writer could not be completed
when the target map has a configured Near Cache. [#18696]
Fixed an issue where the updates made to a persistent map store might be lost when the
write coalescing is enabled. [#18686]
Fixed a reconnection flood when members are separated by a proxy: When a member is disconnected
from the cluster, the alive cluster members still try to reconnect to it if the dying member
connection is not closed explicitly. In the cases where the connection is explicitly closed with a cause
(such as
Connection reset by peerorRemote socket closed!), a new connection was being establishedif the member is placed behind a proxy. This scenario was end causing opening and closing connections continuously.
This issue has been fixed. [#18673]
Fixed an issue where the multicast discovery was not working between the members
when the loopback mode is enabled. [#18669]
The
HazelcastInstance.shutdown()method now gracefully terminate Jet engine jobs, too.After the merge of IMDG and Jet, it was failing. [#18625]
Replicated Map does not fail to publish events anymore, from an entry listener with a predicate
which has an attribute path. [#18623]
Fixed a possible performance regression by not starting the cooperative threads
until a job is submitted; otherwise the Jet engine was consuming system resources. [#18574]
Fixed an issue where running SQL statements was fetching results incorrectly
(from an unexpected mapping) when there are different user-provided schemas for
data structures and mappings. [#18428]
Fixed an issue where the client state listener was not properly working
with failover clients (in blue-green deployments); it was failing with
invalid configuration exception. [#18351]
Fixed an issue where there might be continuous reconnection attempts by the
cluster members to a failed member, even its connection is explicitly closed
and when Hazelcast is placed behind a proxy. [#18320]
Hazelcast now properly works on hosts with multiple NICs. [#17834]
Removed/Deprecated Features
**
hazelcast.client.statistics.enabled**
hazelcast.client.statistics.period.seconds[#19219]
HotRestartServiceclass has been deprecated; usePersistenceServiceinstead. [#19404]**
hazelcast.hotrestart.free.native.memory.percentage[#19404]
Jet,JetInstanceandJetCacheManagerclasses have been deprecated.See https://docs.hazelcast.com/hazelcast/5.0/migrate/upgrading-from-jet.html
for details. Accordingly
JetInstancehas been removed from Hazelcast's command lineinterface (CLI) and Jet engine tests (also the name of CLI has been changed to
HazelcastCommandLine). [#18829], [#18775], [#18667]bootstrappedInstance()has been deprecated. Instead, you can useHazelcast.bootstrappedInstance().See https://docs.hazelcast.com/hazelcast/5.0/migrate/upgrading-from-jet.html for details.
NULLS FIRSTandNULLS LASThas been removed from the SQL engine;the indices treat
NULLas the smallest value in ordering, therefore we needed to disable temporarily these constructs. [#19031]hot-restart-persistencehas been deprecated.You can use
persistenceinstead, which is the successor ofhot-restart-persistence.If both are enabled, Hazelcast uses the
persistenceconfiguration.The
hot-restart-persistenceelement will be removed in a future release. [#19004]hazelcast-allmodule has been removed from the Hazelcast distribution after the merge offormer IMDG and Jet products.
Contributors
We would like to thank the contributors from our open source community
who worked on this release:
Configuration
📅 Schedule: At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR has been generated by WhiteSource Renovate. View repository job log here.