[DOCS] Add overview for rollup refactor (#70001) (#70233)

docs/reference/data-rollup-transform.asciidoc

@@ -6,9 +6,23 @@
 
 {es} offers the following methods for manipulating your data:
 
+ifdef::permanently-unreleased-branch[]
+
+* <<xpack-rollup>>
++
+A rollup aggregates an index's time series data and stores the results in new
+read-only index. For example, you can roll up hourly data into daily or weekly
+summaries.
+
+endif::[]
+ifndef::permanently-unreleased-branch[]
+
 * <<xpack-rollup,Rolling up your historical data>>
 +
 include::rollup/index.asciidoc[tag=rollup-intro]
+
+endif::[]
+
 * <<transforms,Transforming your data>>
 +
 include::transform/transforms.asciidoc[tag=transform-intro]

docs/reference/redirects.asciidoc

@@ -1337,3 +1337,39 @@ See <<search-aggregations-matrix-stats-aggregation>>.
 === `fielddata` mapping parameter
 
 See <<fielddata-mapping-param>>.
+
+ifdef::permanently-unreleased-branch[]
+[role="exclude",id="rollup-overview"]
+=== {rollup-cap} overview
+
+// tag::legacy-rollup-redirect[]
+Legacy rollups are deprecated and will be replaced by <<xpack-rollup,new rollup
+functionality>> introduced in Elasticsearch 7.x. For legacy rollup API
+documentation, see <<legacy-rollup-apis>>.
+// end::legacy-rollup-redirect[]
+
+[role="exclude",id="rollup-getting-started"]
+=== Getting started with {rollups}
+
+include::redirects.asciidoc[tag=legacy-rollup-redirect]
+
+[role="exclude",id="rollup-api-quickref"]
+=== {rollup-cap} API quick reference
+
+include::redirects.asciidoc[tag=legacy-rollup-redirect]
+
+[role="exclude",id="rollup-understanding-groups"]
+=== Understanding rollup groups
+
+include::redirects.asciidoc[tag=legacy-rollup-redirect]
+
+[role="exclude",id="rollup-agg-limitations"]
+=== {rollup-cap} aggregation limitations
+
+include::redirects.asciidoc[tag=legacy-rollup-redirect]
+
+[role="exclude",id="rollup-search-limitations"]
+=== {rollup-cap} search limitations
+
+include::redirects.asciidoc[tag=legacy-rollup-redirect]
+endif::[]

docs/reference/rollup/index.asciidoc

@@ -1,3 +1,109 @@
+ifdef::permanently-unreleased-branch[]
+[role="xpack"]
+[testenv="basic"]
+[[xpack-rollup]]
+== Rollups
+
+A rollup aggregates an index's time series data and stores the results in a new
+read-only index, called a rollup index. For example, you can roll up hourly
+metric data into daily or weekly summaries.
+
+A rollup index only contains aggregated data for the fields you choose. You can
+search, visualize, and aggregate a rollup index like a regular index. While its
+data is less granular, a rollup index contains fewer fields and documents,
+making these operations faster.
+
+Use rollups to:
+
+* Reduce storage costs by deleting or archiving your original indices
+* Speed up searches and visualizations
+* Compactly store historical metric data
+
+image::images/rollups/rollups.gif[Use rollups to reduce the size of your data]
+
+[discrete]
+[[roll-up-your-data]]
+=== Roll up your data
+
+You typically perform rollups using the {ilm-init} <<ilm-rollup,`rollup`>>
+action. You configure the `rollup` action with the aggregations and metrics
+you want to store in the rollup index.
+
+The `rollup` action also lets you specify an {ilm-init} policy for the resulting
+rollup index. If you don't specify this policy, {ilm-init} will not manage the
+rollup index.
+
+[TIP]
+====
+In most cases, the {ilm-init} policy for a rollup index should differ from the
+policy for its source index. To prevent <<size-your-shards,oversharding>>, we
+also recommend using the {ilm-init} <<ilm-shrink,`shrink`>> action to reduce the
+rollup index's primary shard count.
+====
+
+You can also manually roll up an index using the <<rollup-api,rollup API>>.
+
+[discrete]
+[[search-rollups]]
+=== Rollups and data streams
+
+Rollups are designed to work with the backing indices of data streams. If you
+roll up a backing index for a data stream, the resulting rollup index is a
+backing index for the same stream.
+
+If you search a data stream containing both a rollup index and its source index,
+{es} automatically resolves searches without duplicate results.
+
+If results from the rollup index and the source index would be the same, {es}
+uses the rollup for faster results. If the results differ, {es} uses the source
+index's data for better accuracy. If you've replaced the source index with a
+searchable snapshot, {es} uses the searchable snapshot as the source index.
+
+This automatic search resolution lets you use rollups as a caching layer for
+your data stream. You can keep rollup indices in a hot or warm <<data-tiers,data
+tier>> and archive your original indices in a cold or frozen tier. 
+
+NOTE: Only data streams support automatic search resolution for rollups. Searches
+directly targeting a rollup index and its source index may return duplicate
+results.
+
+[discrete]
+[[legacy-rollups]]
+=== Legacy rollups
+
+// tag::legacy-rollups[]
+Before {es} 7.x, you could only create rollups using periodic cron jobs. Special
+APIs were required to manage these jobs and search the resulting rollup indices.
+These rollup APIs are now deprecated and will be removed in a future release.
+// end::legacy-rollups[]
+
+See <<legacy-rollup-apis>>.
+
+[discrete]
+[[differences-with-legacy-rollups]]
+==== Differences from legacy rollups
+
+The new rollup functionality differs from legacy rollups as follows:
+
+* Rollups no longer require a cron job. You can perform rollups using the
+{ilm-init} <<ilm-rollup,`rollup`>> action or <<rollup-api,rollup API>>.
+
+* Rollup indices are read-only and only contain aggregated data from one source
+index. Previously, multiple legacy rollup jobs could index into a single rollup
+index.
+
+* You can now search rollup indices like a regular index. Legacy rollup indices
+required a special rollup search API, which could only search one rollup index
+at a time.
+
+* While still approximate, `terms` aggregations for rollups are now more
+accurate. Previously, `terms` aggregations for legacy rollup jobs could provide
+inaccurate document counts due to differences between shards for the source
+index.
+
+endif::[]
+ifndef::permanently-unreleased-branch[]
+
 [role="xpack"]
 [testenv="basic"]
 [[xpack-rollup]]
@@ -28,4 +134,5 @@ include::api-quickref.asciidoc[]
 include::rollup-getting-started.asciidoc[]
 include::understanding-groups.asciidoc[]
 include::rollup-agg-limitations.asciidoc[]
-include::rollup-search-limitations.asciidoc[]
+include::rollup-search-limitations.asciidoc[]
+endif::[]

docs/reference/rollup/rollup-apis.asciidoc

@@ -15,9 +15,7 @@ summaries.
 [[legacy-rollup-apis]]
 === Legacy rollup APIs
 
-Before {es} 7.x, you could only create rollups using periodic cron jobs. Special
-APIs were required to manage these jobs and search the resulting rollup indices.
-These rollup APIs are now deprecated and will be removed in a future release.
+include::{es-repo-dir}/rollup/index.asciidoc[tag=legacy-rollups]
 
 [discrete]
 [[rollup-jobs-endpoint]]