[CI] Rest text p0=snapshot/10_basic/Create a source only snapshot and then restore it failure

[jrodewig]

Received this build failure during the push of an unrelated docs change (a311018)

Failure: https://gradle-enterprise.elastic.co/s/v5h5hntmwkiqy/tests/fcijb24gqtmnc-cqpxjyuvwnxpc

Based on the results, it looks like two snapshots were restored instead of one as expected.

Caused by: java.lang.AssertionError: 
snapshot.shards.successful didn't match expected value:
    snapshot.shards.successful: expected Integer [1] but was Integer [2]

  "stash" : {
    "body" : {
      "snapshot" : {
        "snapshot" : "test_snapshot",
        "uuid" : "1pr60of1RIWn1w2MAjRmRw",
        "version_id" : 8000099,
        "version" : "8.0.0",
        "indices" : [
          "test_index",
          "ilm-history-1-000001"
        ],
        "include_global_state" : true,
        "state" : "SUCCESS",
        "start_time" : "2019-12-20T19:16:11.335Z",
        "start_time_in_millis" : 1576869371335,
        "end_time" : "2019-12-20T19:16:11.535Z",
        "end_time_in_millis" : 1576869371535,
        "duration_in_millis" : 200,
        "failures" : [ ],
        "shards" : {
          "total" : 2,
          "failed" : 0,
          "successful" : 2
        }
      }
    }
  }
}]

[elasticmachine]

Pinging @elastic/es-distributed (:Distributed/Snapshot/Restore)

[mayya-sharipova]

The test failed on 7.x, but it may be just a simple timeout.

Log: https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+7.x+matrix-java-periodic/ES_BUILD_JAVA=openjdk13,ES_RUNTIME_JAVA=openjdk13,nodes=general-purpose/484/console
Build scan: https://gradle-enterprise.elastic.co/s/uqzlfrbi4wq7u

Relevant stacktrace:

org.elasticsearch.xpack.test.rest.XPackRestIT > test {p0=snapshot/10_basic/Create a source only snapshot and then restore it} FAILED |  
-- | --
  | java.lang.AssertionError: Failure at [snapshot/10_basic:21]: expected [2xx] status code but api [cluster.health] returned [408 Request Timeout] [{"cluster_name":"integTest","status":"yellow","timed_out":true,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":2,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":1,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":66.66666666666666}] |  
  | at __randomizedtesting.SeedInfo.seed([D3B89AEE43928B07:5BECA534ED6EE6FF]:0)


Caused by: |  
-- | --
  | java.lang.AssertionError: expected [2xx] status code but api [cluster.health] returned [408 Request Timeout] [{"cluster_name":"integTest","status":"yellow","timed_out":true,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":2,"active_shards":2,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":1,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":66.66666666666666}] |  
  | at org.junit.Assert.fail(Assert.java:88)