[ingest] Default ingest node pipeline via index template fails to apply for first document.

[jakelandis]

A ingest pipeline defined as a default from an index template will not apply the pipeline to the very first document indexed.

curl -XDELETE "http://localhost:9200/test"
curl -XPUT "http://localhost:9200/_ingest/pipeline/test-pipeline" -H 'Content-Type: application/json' -d'
{
  "description" : "test pipeline",
  "processors" : [
    {
      "set" : {
        "field" : "foo",
        "value" : "bar"
      }
    }
  ]
}'
curl -XPUT "http://localhost:9200/_template/template_test" -H 'Content-Type: application/json' -d'
{
    "index_patterns": ["test"],
    "settings": {
        "index.default_pipeline": "test-pipeline"
    }
}'
curl -XPOST "http://localhost:9200/test/_doc/1" -H 'Content-Type: application/json' -d'
{
  "a" : "b"
}'
curl -XPOST "http://localhost:9200/test/_refresh"
curl -XGET "http://localhost:9200/test/_search?pretty"
curl -XPOST "http://localhost:9200/test/_doc/2" -H 'Content-Type: application/json' -d'
{
  "a" : "b"
}'
curl -XPOST "http://localhost:9200/test/_refresh"
curl -XGET "http://localhost:9200/test/_search?pretty"

results in

"hits" : {
    "total" : 2,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "a" : "b"
        }
      },
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "2",
        "_score" : 1.0,
        "_source" : {
          "a" : "b",
          "foo" : "bar"
        }
      }
    ]
  }

^^ note the the set pipeline did not take effect on the first document.

It works fine when creating the index without without a template

curl -XDELETE "http://localhost:9200/test"
curl -XPUT "http://localhost:9200/_ingest/pipeline/test-pipeline" -H 'Content-Type: application/json' -d'
{
  "description" : "test pipeline",
  "processors" : [
    {
      "set" : {
        "field" : "foo",
        "value" : "bar"
      }
    }
  ]
}'
curl -XPUT "http://localhost:9200/test" -H 'Content-Type: application/json' -d'
{
    "settings" : {
        "index" : {
            "default_pipeline" : "test-pipeline" 
        }
    }
}'
curl -XPOST "http://localhost:9200/test/_doc/1" -H 'Content-Type: application/json' -d'
{
  "a" : "b"
}'
curl -XPOST "http://localhost:9200/test/_refresh"
curl -XGET "http://localhost:9200/test/_search?pretty"
curl -XPOST "http://localhost:9200/test/_doc/2" -H 'Content-Type: application/json' -d'
{
  "a" : "b"
}'
curl -XPOST "http://localhost:9200/test/_refresh"
curl -XGET "http://localhost:9200/test/_search?pretty"

output is as expected:

"hits" : {
    "total" : 2,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "a" : "b",
          "foo" : "bar"
        }
      },
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "2",
        "_score" : 1.0,
        "_source" : {
          "a" : "b",
          "foo" : "bar"
        }
      }
    ]
  }

cc: @original-brownbear

[elasticmachine]

Pinging @elastic/es-core-infra