Add synthetic mode as default when creating a datastream

[mrodm]

Closes #1026

This PR adds into the template the needed fields to enable synthetic mode when creating a new metrics data stream. New questions are shown to the user.

• No question is shown if a logs data stream is created/requested:
  • Nothing added to the manifest neither

  Create a new data stream
  ? Data stream name: foo
  ? Data stream title: Foo
  ? Type: logs
  New data stream has been created: foo
  Done
  

• Example adding just synthetic source:
  • Output

     $ elastic-package create data-stream
    Create a new data stream
    ? Data stream name: foo
    ? Data stream title: Foo
    ? Type: metrics
    ? Enable time series and synthetic source? No
    ? Enable synthetic source? Yes
    New data stream has been created: foo
    Done
    

  • Manifest file generated:

    title: "Foo"
    type: metrics
    streams:
      - input: sample/metrics
        title: Sample metrics
        description: Collect sample metrics
        vars:
          - name: period
            type: text
            title: Period
            default: 10s
    elasticsearch:
      source_mode: synthetic

• Example adding both synthetic source and time series:
  • Output :

    Create a new data stream
    ? Data stream name: foo
    ? Data stream title: Foo
    ? Type: metrics
    ? Enable time series and synthetic source? Yes
    ? Enable synthetic source? Yes
    New data stream has been created: foo
    Done
    

  • Manifest file generated:

    title: "Foo"
    type: metrics
    streams:
      - input: sample/metrics
        title: Sample metrics
        description: Collect sample metrics
        vars:
          - name: period
            type: text
            title: Period
            default: 10s
    elasticsearch:
      source_mode: synthetic
      index_mode: time_series

[mrodm]

Should it be included also time_series setting as synthetic _source is just GA for index.mode set to time_series ?
https://www.elastic.co/guide/en/elasticsearch/reference/8.7/mapping-source-field.html#synthetic-source

elasticsearch:
  source_mode: synthetic
  index_mode: time_series

[jsoriano]

This would look like a sane default, yes. But then I wonder if this should be enabled by default in logs data streams, not sure if we want to enable time series mode there.

[mrodm]

I've introduced a new question to ask for enabling time_series

 $ rm -rf data_stream/foo ; elastic-package create data-stream ; cat data_stream/foo/manifest.yml 
Create a new data stream
? Data stream name: foo
? Data stream title: Foo
? Type: metrics
? Enable synthetic source? Yes
? Enable time series? Yes
New data stream has been created: foo
Done
title: "Foo"
type: metrics
streams:
  - input: sample/metrics
    title: Sample metrics
    description: Collect sample metrics
    vars:
      - name: period
        type: text
        title: Period
        default: 10s
elasticsearch:
  source_mode: synthetic
  index_mode: time_series

This last question just is shown when the user selects Yes to "Enable synthetic source?" question.

So, the user could choose:

• not to use synthetic source
• use just synthetic source
• use both synthetic source and time_series

Maybe it's not needed and if so, I could revert the latest changes so this PR just introduces source_mode key. WDYT @jsoriano ?

[jsoriano]

Looks good, but maybe another option would be to make it more explicit, asking first if they want to enable time series and synthetic source, something like this:

? Enable time series and synthetic source? No
? Enable synthetic source? Yes
New data stream has been created: foo

If the first answer is affirmative, the second question is not done:

? Enable time series and synthetic source? Yes
New data stream has been created: foo

[mrodm]

Ok, I'll change it 👍
I'll add both question that by default they are true (yes answer).

[mrodm]

Examples:

• Just synthetic source:

Create a new data stream
? Data stream name: foo
? Data stream title: Foo
? Type: metrics
? Enable time series and synthetic source? No
? Enable synthetic source? Yes
New data stream has been created: foo
Done
title: "Foo"
type: metrics
streams:
  - input: sample/metrics
    title: Sample metrics
    description: Collect sample metrics
    vars:
      - name: period
        type: text
        title: Period
        default: 10s
elasticsearch:
  source_mode: synthetic

• Both synthetic and time_series:

Create a new data stream
? Data stream name: foo
? Data stream title: Foo
? Type: metrics
? Enable time series and synthetic source? Yes
New data stream has been created: foo
Done
title: "Foo"
type: metrics
streams:
  - input: sample/metrics
    title: Sample metrics
    description: Collect sample metrics
    vars:
      - name: period
        type: text
        title: Period
        default: 10s
elasticsearch:
  source_mode: synthetic
  index_mode: time_series

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 85ad5b1

History

• 💚 Build #917 succeeded 68fb6c4
• 💚 Build #880 succeeded f3a68b1
• 💚 Build #747 succeeded 41f308f
• 💚 Build #739 succeeded d5adcf3

[mrodm]

buildkite test integrations

[mrodm]

buildkite test integrations