7.11.0 Meta Ticket

[stevejgordon]

Breaking changes

Machine Learning

• Add new flag exclude_generated that removes generated fields in GET config APIs #63899 (issue: #63055)

Deprecations

Features/CAT APIs

• Deprecate the 'local' parameter of /_cat/indices #62198 (issue: #60718)
• Deprecate the 'local' parameter of /_cat/shards #62197 (issue: #60718)

MULTIPLE AREA LABELS

• Deprecates _upgrade API (#47678) #50484 (issue: #47678)

Mapping

• Emit a warning when an unknown geo strategy is passed in #63975

New features

Aggregations

• Support global ords in top_metrics #64967 (issue: #64774)

Features/Ingest

• URI parts ingest processor #65150 (issue: #57481) - PR Implement URI parts ingest processor #5304

Geo

• Add geo_line aggregation #41612 (issue: #41649)

MULTIPLE AREA LABELS

• Add doc_count field mapper #64503 (issue: #58339)

Search

• Async search status API #62947 (issue: #57537) - PR Implement AsyncSearch Status API #5239

Enhancements

Aggregations

• Add hard_bounds support for histogram field-based histograms #64246 (issue: #62124)
• Add support for histogram fields to rate aggregation #63289 (issue: #62939) - PR Add support for rate aggregation #5206
• Add supports for upper and lower values on boxplot based on the IQR value #63617 (issue: #60466)
• Add value_count mode to rate agg #63687 (issue: #63575) - PR Add support for rate aggregation #5206
• Allow mixing set-based and regexp-based include and exclude #63325 (issue: #62246)

Authentication

• Adding API for generating SAML SP metadata #64517 (issue: #49018)
• Adds realm name OIDC _security/oidc/prepare and _security/oidc/authenticate APIs responses #64966 (issue: #53161)

Client

• Adding authentication information to access token create APIs #62490 (issue: #59685)

Features/Data streams

• Add custom metadata support to data steams. #63991 (issue: #59195)
• Hidden data streams #63987 (issue: #63889)
• REST endpoint and tests for data stream migration #64415 (issue: #61046)

Features/Indices APIs

• Add a template parameter to override auto_create_index value #61858 (issue: #20640)
• Mark component and composable index template APIs as stable #65013

Features/Ingest

• Add copy_from parameter for set ingest processor #63540 (issues: #51046, #55682, #56985)
• Attachment ingest processor: add resource_name field #64389 - PR AWAITING APPROVAL Support resource_name on AttachmentProcessor #5205

Geo

• Enable geo_distance and geo_bounding_box queries on geo_shape field type #64224 (issue: #54628)

License

• Adding a warning header when a license is about to expire #64948 (issue: #60562)

MULTIPLE AREA LABELS

• Do not skip not available shard exception in search response #64337 (issue: #47700)
• Check Security Roles in Deprecation Info API #49212 (issues: #47333, #47714)
• Geo_point runtime field implementation #63164

Machine Learning

• Add new snapshot upgrader API for upgrading older snapshots #64665 (issue: #64154)
• Add total ML memory to ML info #65195 (issue: #64225)
• Adding result_type and mlcategory fields to category definitions #63326 (issue: #60108)
• Provide a way to revert an AD job to an empty snapshot #65431

Mapping

• Support unmapped fields in search 'fields' option #65386 (issue: #63690)
• Allow MappedFieldType impls to hide themselves from field caps #63547 (issue: #63446)
• Enable collapse on unsigned_long field #63495 (issue: #60050)
• Introduce runtime section in mappings #62906 (issue: #59332)

Search

• Add runtime_mappings to search request #64374 (issue: #59332) PR Implement runtime fields on searches #5259
• Add support for runtime fields #61776 (issue: #59332)
• Automatic tie-breaking for sorted queries within a PIT #65450 (issue: #56828)

Transform

• Add new exclude_generated flag to GET transform #63093 (issue: #63055)
• Add support for "missing" aggregation #63651
• Add support for median absolute deviation #64634
• [Transform] use ISO dates in output instead of epoch millis #65584 - Introduces changing behaviour and new dates_as_epoch_millis setting.

Bug fixes

Features/Indices APIs

• Fix remove alias with must_exist #65141 (issues: #58100, #62642)

Search

• Add explanation to runtime field query #63429
• Drop boost from runtime distance feature query #63949 (issue: #63767)

7.10 Remaining Items

New features

Aggregations

• Add rate aggregation #61369 (issue: #60674) - PR Add support for rate aggregation #5206

Features/Features

• Add data tiers (hot, warm, cold, frozen) as custom node roles #60994 (issue: #60848)

Features/ILM+SLM

• ILM migrate data between tiers #61377 (issue: #60848)
• ILM: add force-merge step to searchable snapshots action #60819 (issues: #53488, #56215)

Machine Learning

• Implement AucRoc metric for classification #60502 (issue: #62160)

Mapping

• [ x] Introduce 64-bit unsigned long field type #60050 (issue: #32434) PR Implement unsigned long numeric field #5275

Search

• Add search fields option to support high-level field retrieval #60100 (issues: #49028, #55363) - SG WIP (Required for Runtime Fields)

Enhancements

• Oidc additional client auth types #58708

Authorization

• Add more context to index access denied errors #60357 (issue: #42166)

CRUD

• Dedicated threadpool for system index writes #61655

Cluster Coordination

• Provide option to allow writes when master is down #60605

Distributed

• Detect noop of update index settings #61348

Features/Features

• Add aggregation list to node info #60074 (issue: #52057)
• Adding new require_alias option to indexing requests #58917 (issue: #55267)

Features/Ingest

• Add network from MaxMind Geo ASN database #61676
• Allow_duplicates option for append processor #61916 (issue: #57543) - PR Support allow_duplicates on append processor #5208
• Configurable output format for date processor #61324 (issue: #42523)
• Enhance the ingest node simulate verbose output #60433 (issue: #56004)
• Per processor description for verbose simulate #58207 (issue: #57906)
• Preserve grok pattern ordering and add sort option #61671 (issue: #40819)

Infra/Core

• Add logstash system index APIs #53350
• Deprecate REST access to System Indices #60945

Infra/Logging

• Write deprecation logs to a data stream #61484 (issues: #46106, #61474)

Machine Learning

• Add a "verbose" option to the data frame analytics stats endpoint #59589 (issue: #59125)
• Add new include flag to get trained models API for model training metadata #61922
• Add new feature_processors field for data frame analytics #60528 (issue: #59327)
• Add new n_gram_encoding custom processor #61578
• Calculate total feature importance to store with model metadata #1387
• Change outlier detection feature_influence format to array with nested objects #1475, #62068

Mapping

• Add field type for version strings #59773 (issue: #48878)
• Allow [null] values in [null_value] #61798 (issues: #7978, #58823)
• Allow metadata fields in the _source #61590 (issue: #58339)

Search

• Add case insensitive flag for "term" family of queries #61596 (issue: #61546)
• Add case insensitive support for regex queries #59441
• Tweak toXContent implementation of ParametrizedFieldMapper #59968
• Implement fields value fetching for the text, search_as_you_type and token_count field types #63515

Snapshot/Restore

• Add repositories metering API #60371
• Clone Snapshot API #61839 - SG WIP
• Introduce index based snapshot blob cache for Searchable Snapshots #60522

Store

• Report more details of unobtainable ShardLock #61255 (issue: #38807)

Transform

• Add support for missing bucket #59591 (issues: #42941, #55102)

Bug fixes

Aggregations

• Fix AOOBE when setting min_doc_count to 0 in significant_terms #60823 (issues: #60683, #60824)
• Make sure non-collecting aggs include sub-aggs #64214 (issue: #64142)
• Fix broken parent and child aggregator #63811

CRUD

• Propagate forceExecution when acquiring permit #60634 (issue: #60359)

Distributed

• Fix cluster health when closing #61709

Features/Stats

• Remove sporadic min/max usage estimates from stats #59755

Features/Watcher

• Correct the query dsl for watching elasticsearch version #58321 (issue: #58261)
• Fix passing params to template or script failed in watcher #58559 (issue: #57625)

Machine Learning

• Get data frame analytics jobs stats API can return multiple responses if more than one error #60900 (issue: #60876)
• Correct the anomaly detection job model state min_version #1546

Mapping

• Improve ignore_malformed handling for dates #60211 (issue: #52634)

Network

• Suppress noisy SSL exceptions #61359

Search

• Allows nanosecond resolution in search_after #60328 (issue: #52424)