Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[OSquery Integration] Live query search doesn't return relevant results for agents #115069

Closed
TheRiffRafi opened this issue Oct 14, 2021 · 1 comment · Fixed by #116332
Closed

[OSquery Integration] Live query search doesn't return relevant results for agents #115069

TheRiffRafi opened this issue Oct 14, 2021 · 1 comment · Fixed by #116332
Assignees
@patrykkopycinski
Labels
bug Fixes for quality problems that affect the customer experience Team:Asset Management Security Asset Management Team

Comments

@TheRiffRafi
Copy link

TheRiffRafi commented Oct 14, 2021 •
edited
Loading

Kibana version: 7.15.0

Elasticsearch version: 7.15.0

Describe the bug:
The search bar to find agents in the live query screen doesn't return relevant results when there are more than 10 agents using the integration.

Steps to reproduce:

  1. Add more than 10 agents with different policies all having the OS query manager integration.
  2. Perform a lookup of the oldest created agent.
  3. Agent doesn't come up on the list for selection
  4. (bonus problem) trying to specify the agent within quotes generated an error:
HttpFetchError@https://testmonitorfrozen.kb.us-central1.gcp.cloud.es.io:9243/44040/bundles/core/core.entry.js:6:2214
fetchResponse@https://testmonitorfrozen.kb.us-central1.gcp.cloud.es.io:9243/44040/bundles/core/core.entry.js:6:27830

Screen Shot 2021-10-14 at 12 23 42

Expected behavior:
The search should return the relevant agent specified in the lookup.

Any additional context:

This is an example query being performed by the integration as tested in my lab (grabbed by enabling slowlogs):

query 
GET .fleet-agents/_search
{
  "from": 0,
  "size": 10,
  "query": {
    "bool": {
      "filter": [
        {
          "bool": {
            "should": [
              {
                "bool": {
                  "should": [
                    {
                      "match": {
                        "policy_id": {
                          "query": "e9cb3fd0-0565-11ec-9dfc-939b7df9c76c",
                          "operator": "OR",
                          "prefix_length": 0,
                          "max_expansions": 50,
                          "fuzzy_transpositions": true,
                          "lenient": false,
                          "zero_terms_query": "NONE",
                          "auto_generate_synonyms_phrase_query": true,
                          "boost": 1
                        }
                      }
                    }
                  ],
                  "adjust_pure_negative": true,
                  "minimum_should_match": "1",
                  "boost": 1
                }
              },
              {
                "bool": {
                  "should": [
                    {
                      "match": {
                        "policy_id": {
                          "query": "0f05ffb0-0039-11ec-abcf-d1069794f7a2",
                          "operator": "OR",
                          "prefix_length": 0,
                          "max_expansions": 50,
                          "fuzzy_transpositions": true,
                          "lenient": false,
                          "zero_terms_query": "NONE",
                          "auto_generate_synonyms_phrase_query": true,
                          "boost": 1
                        }
                      }
                    }
                  ],
                  "adjust_pure_negative": true,
                  "minimum_should_match": "1",
                  "boost": 1
                }
              },
              {
                "bool": {
                  "should": [
                    {
                      "match": {
                        "policy_id": {
                          "query": "6acc0290-003e-11ec-abcf-d1069794f7a2",
                          "operator": "OR",
                          "prefix_length": 0,
                          "max_expansions": 50,
                          "fuzzy_transpositions": true,
                          "lenient": false,
                          "zero_terms_query": "NONE",
                          "auto_generate_synonyms_phrase_query": true,
                          "boost": 1
                        }
                      }
                    }
                  ],
                  "adjust_pure_negative": true,
                  "minimum_should_match": "1",
                  "boost": 1
                }
              },
              {
                "bool": {
                  "should": [
                    {
                      "match": {
                        "policy_id": {
                          "query": "f14fcf60-003c-11ec-abcf-d1069794f7a2",
                          "operator": "OR",
                          "prefix_length": 0,
                          "max_expansions": 50,
                          "fuzzy_transpositions": true,
                          "lenient": false,
                          "zero_terms_query": "NONE",
                          "auto_generate_synonyms_phrase_query": true,
                          "boost": 1
                        }
                      }
                    }
                  ],
                  "adjust_pure_negative": true,
                  "minimum_should_match": "1",
                  "boost": 1
                }
              },
              {
                "bool": {
                  "filter": [
                    {
                      "bool": {
                        "should": [
                          {
                            "match": {
                              "policy_id": {
                                "query": "6c7e63c0-1fca-11ec-9197-6ffc15e5eb0d",
                                "operator": "OR",
                                "prefix_length": 0,
                                "max_expansions": 50,
                                "fuzzy_transpositions": true,
                                "lenient": false,
                                "zero_terms_query": "NONE",
                                "auto_generate_synonyms_phrase_query": true,
                                "boost": 1
                              }
                            }
                          }
                        ],
                        "adjust_pure_negative": true,
                        "minimum_should_match": "1",
                        "boost": 1
                      }
                    },
                    {
                      "bool": {
                        "should": [
                          {
                            "bool": {
                              "must": [
                                {
                                  "query_string": {
                                    "query": "*testhost*",
                                    "fields": [
                                      "local_metadata.host.hostname^1.0"
                                    ],
                                    "type": "best_fields",
                                    "default_operator": "or",
                                    "max_determinized_states": 10000,
                                    "enable_position_increments": true,
                                    "fuzziness": "AUTO",
                                    "fuzzy_prefix_length": 0,
                                    "fuzzy_max_expansions": 50,
                                    "phrase_slop": 0,
                                    "escape": false,
                                    "auto_generate_synonyms_phrase_query": true,
                                    "fuzzy_transpositions": true,
                                    "boost": 1
                                  }
                                }
                              ],
                              "adjust_pure_negative": true,
                              "minimum_should_match": "1",
                              "boost": 1
                            }
                          },
                          {
                            "bool": {
                              "should": [
                                {
                                  "query_string": {
                                    "query": "*testhost*",
                                    "fields": [
                                      "local_metadata.elastic.agent.id^1.0"
                                    ],
                                    "type": "best_fields",
                                    "default_operator": "or",
                                    "max_determinized_states": 10000,
                                    "enable_position_increments": true,
                                    "fuzziness": "AUTO",
                                    "fuzzy_prefix_length": 0,
                                    "fuzzy_max_expansions": 50,
                                    "phrase_slop": 0,
                                    "escape": false,
                                    "auto_generate_synonyms_phrase_query": true,
                                    "fuzzy_transpositions": true,
                                    "boost": 1
                                  }
                                }
                              ],
                              "adjust_pure_negative": true,
                              "minimum_should_match": "1",
                              "boost": 1
                            }
                          }
                        ],
                        "adjust_pure_negative": true,
                        "minimum_should_match": "1",
                        "boost": 1
                      }
                    }
                  ],
                  "adjust_pure_negative": true,
                  "boost": 1
                }
              }
            ],
            "adjust_pure_negative": true,
            "minimum_should_match": "1",
            "boost": 1
          }
        },
        {
          "bool": {
            "should": [
              {
                "match": {
                  "active": {
                    "query": true,
                    "operator": "OR",
                    "prefix_length": 0,
                    "max_expansions": 50,
                    "fuzzy_transpositions": true,
                    "lenient": false,
                    "zero_terms_query": "NONE",
                    "auto_generate_synonyms_phrase_query": true,
                    "boost": 1
                  }
                }
              }
            ],
            "adjust_pure_negative": true,
            "minimum_should_match": "1",
            "boost": 1
          }
        }
      ],
      "adjust_pure_negative": true,
      "boost": 1
    }
  },
 "sort": [
    {
      "enrolled_at": {
        "order": "desc"
      }
    }
  ],
  "track_total_hits": 2147483647
}

And it returns these values:

results 
{
  "took" : 801,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [
      {
        "_index" : ".fleet-agents-7",
        "_type" : "_doc",
        "_id" : "a0b21013-c85f-4536-9acb-b4fdc1c072f0",
        "_score" : null,
        "_source" : {
          "access_api_key_id" : "zubVW3wBK1tuY2ttTtZb",
          "action_seq_no" : [
            -1
          ],
          "active" : true,
          "agent" : {
            "id" : "a0b21013-c85f-4536-9acb-b4fdc1c072f0",
            "version" : "7.15.0"
          },
          "enrolled_at" : "2021-10-07T17:37:58Z",
          "local_metadata" : {
            "elastic" : {
              "agent" : {
                "build.original" : "7.15.0 (build: 9023152025ec6251bc6b6c38009b309157f10f17 at 2021-09-16 06:31:45 +0000 UTC)",
                "id" : "a0b21013-c85f-4536-9acb-b4fdc1c072f0",
                "log_level" : "info",
                "snapshot" : false,
                "upgradeable" : true,
                "version" : "7.15.0"
              }
            },
            "host" : {
              "architecture" : "x86_64",
              "hostname" : "fjdkfne.domanin.net",
              "id" : "3ea6f2e8be0249c297b52db0e47ca8da",
              "ip" : [
                "127.0.0.1/8",
                "10.10.123.23/24"
              ],
              "mac" : [
                "00:25:b1:58:a4:6e",
                "00:21:b5:55:b5:1e"
              ],
              "name" : "gd23ef.domanin.net"
            },
            "os" : {
              "family" : "redhat",
              "full" : "Red Hat Enterprise Linux Server Maipo(7.9 (Maipo))",
              "kernel" : "3.10.0-1160.11.1.el7.x86_64",
              "name" : "Red Hat Enterprise Linux Server",
              "platform" : "rhel",
              "version" : "7.9 (Maipo)"
            }
          },
          "policy_id" : "6acc0290-003e-11ec-abcf-d1069794f7a2",
          "type" : "PERMANENT",
          "default_api_key" : "DF7ZW3wBN6QoF0S9O350:oa2FPHo2TUGTkMgS-t8x5Q",
          "policy_output_permissions_hash" : "dcb55dfca4c44666c4a2d0b6f2d851fbbc5d34d70dabdb95802379c204aad67d",
          "default_api_key_id" : "DF7ZW3wBN6QoF0S9O350",
          "updated_at" : "2021-10-08T22:23:37Z",
          "last_checkin_status" : "online",
          "last_checkin" : "2021-10-08T22:23:30Z",
          "policy_revision_idx" : 24,
          "policy_coordinator_idx" : 1
        },
        "sort" : [
          1633628278000
        ]
      },
      {
        "_index" : ".fleet-agents-7",
        "_type" : "_doc",
        "_id" : "a0b21013-c85f-4536-9acb-b4fdc1c072f1",
        "_score" : null,
        "_source" : {
          "access_api_key_id" : "zubVW3wBK1tuY2ttTtZb",
          "action_seq_no" : [
            -1
          ],
          "active" : true,
          "agent" : {
            "id" : "a0b21013-c85f-4536-9acb-b4fdc1c072f1",
            "version" : "7.15.0"
          },
          "enrolled_at" : "2021-09-07T17:37:58Z",
          "local_metadata" : {
            "elastic" : {
              "agent" : {
                "build.original" : "7.15.0 (build: 9023152025ec6251bc6b6c38009b309157f10f17 at 2021-09-16 06:31:45 +0000 UTC)",
                "id" : "a0b21013-c85f-4536-9acb-b4fdc1c072f0",
                "log_level" : "info",
                "snapshot" : false,
                "upgradeable" : true,
                "version" : "7.15.0"
              }
            },
            "host" : {
              "architecture" : "x86_64",
              "hostname" : "somehost.domanin.net",
              "id" : "3ea6f2e8be0249c297b52db0e47ca8da",
              "ip" : [
                "127.0.0.1/8",
                "10.10.123.23/24"
              ],
              "mac" : [
                "00:24:b5:18:a4:6e",
                "00:25:b5:58:b5:6e"
              ],
              "name" : "somehost.domanin.net"
            },
            "os" : {
              "family" : "redhat",
              "full" : "Red Hat Enterprise Linux Server Maipo(7.9 (Maipo))",
              "kernel" : "3.10.0-1160.11.1.el7.x86_64",
              "name" : "Red Hat Enterprise Linux Server",
              "platform" : "rhel",
              "version" : "7.9 (Maipo)"
            }
          },
          "policy_id" : "6acc0290-003e-11ec-abcf-d1069794f7a3",
          "type" : "PERMANENT",
          "default_api_key" : "DF7ZW3wBN6QoF0S9O350:oa2FPHo2TUGTkMgS-t8x5Q",
          "policy_output_permissions_hash" : "dcb55dfca4c44666c4a2d0b6f2d851fbbc5d34d70dabdb95802379c204aad67d",
          "default_api_key_id" : "DF7ZW3wBN6QoF0S9O350",
          "updated_at" : "2021-10-08T22:23:37Z",
          "last_checkin_status" : "online",
          "last_checkin" : "2021-10-08T22:23:30Z",
          "policy_revision_idx" : 24,
          "policy_coordinator_idx" : 1
        },
        "sort" : [
          1631036278000
        ]
      },
      {
        "_index" : ".fleet-agents-7",
        "_type" : "_doc",
        "_id" : "a0b21013-c85f-4536-9acb-b4fdc1c072f2",
        "_score" : null,
        "_source" : {
          "access_api_key_id" : "zubVW3wBK1tuY2ttTtZb",
          "action_seq_no" : [
            -1
          ],
          "active" : true,
          "agent" : {
            "id" : "a0b21013-c85f-4536-9acb-b4fdc1c072f2",
            "version" : "7.15.0"
          },
          "enrolled_at" : "2021-09-07T17:37:58Z",
          "local_metadata" : {
            "elastic" : {
              "agent" : {
                "build.original" : "7.15.0 (build: 9023152025ec6251bc6b6c38009b309157f10f17 at 2021-09-16 06:31:45 +0000 UTC)",
                "id" : "a0b21013-c85f-4536-9acb-b4fdc1c072f2",
                "log_level" : "info",
                "snapshot" : false,
                "upgradeable" : true,
                "version" : "7.15.0"
              }
            },
            "host" : {
              "architecture" : "x86_64",
              "hostname" : "gadfdsgf2.domanin.net",
              "id" : "3ea6f2e8be0249c297b52db0e47ca8da",
              "ip" : [
                "127.0.0.1/8",
                "10.10.123.23/24"
              ],
              "mac" : [
                "01:25:c5:53:a4:6e",
                "01:25:a5:53:b5:6e"
              ],
              "name" : "somehost.domanin.net"
            },
            "os" : {
              "family" : "redhat",
              "full" : "Red Hat Enterprise Linux Server Maipo(7.9 (Maipo))",
              "kernel" : "3.10.0-1160.11.1.el7.x86_64",
              "name" : "Red Hat Enterprise Linux Server",
              "platform" : "rhel",
              "version" : "7.9 (Maipo)"
            }
          },
          "policy_id" : "6acc0290-003e-11ec-abcf-d1069794f7a3",
          "type" : "PERMANENT",
          "default_api_key" : "DF7ZW3wBN6QoF0S9O350:oa2FPHo2TUGTkMgS-t8x5Q",
          "policy_output_permissions_hash" : "dcb55dfca4c44666c4a2d0b6f2d851fbbc5d34d70dabdb95802379c204aad67d",
          "default_api_key_id" : "DF7ZW3wBN6QoF0S9O350",
          "updated_at" : "2021-10-08T22:23:37Z",
          "last_checkin_status" : "online",
          "last_checkin" : "2021-10-08T22:23:30Z",
          "policy_revision_idx" : 24,
          "policy_coordinator_idx" : 1
        },
        "sort" : [
          1631036278000
        ]
      },
      {
        "_index" : ".fleet-agents-7",
        "_type" : "_doc",
        "_id" : "a0b21013-c85f-4536-9acb-b4fdc1c072f4",
        "_score" : null,
        "_source" : {
          "access_api_key_id" : "zubVW3wBK1tuY2ttTtZb",
          "action_seq_no" : [
            -1
          ],
          "active" : true,
          "agent" : {
            "id" : "a0b21013-c85f-4536-9acb-b4fdc1c072f4",
            "version" : "7.15.0"
          },
          "enrolled_at" : "2021-09-07T17:37:58Z",
          "local_metadata" : {
            "elastic" : {
              "agent" : {
                "build.original" : "7.15.0 (build: 9023152025ec6251bc6b6c38009b309157f10f17 at 2021-09-16 06:31:45 +0000 UTC)",
                "id" : "a0b21013-c85f-4536-9acb-b4fdc1c072f2",
                "log_level" : "info",
                "snapshot" : false,
                "upgradeable" : true,
                "version" : "7.15.0"
              }
            },
            "host" : {
              "architecture" : "x86_64",
              "hostname" : "fsdfsdfs.domanin.net",
              "id" : "3ea6f2e8be0249c297b52db0e47ca8da",
              "ip" : [
                "127.0.0.1/8",
                "10.10.123.23/24"
              ],
              "mac" : [
                "01:25:b5:51:a4:6e",
                "10:25:b2:58:b5:7e"
              ],
              "name" : "somehost.domanin.net"
            },
            "os" : {
              "family" : "redhat",
              "full" : "Red Hat Enterprise Linux Server Maipo(7.9 (Maipo))",
              "kernel" : "3.10.0-1160.11.1.el7.x86_64",
              "name" : "Red Hat Enterprise Linux Server",
              "platform" : "rhel",
              "version" : "7.9 (Maipo)"
            }
          },
          "policy_id" : "6acc0290-003e-11ec-abcf-d1069794f7a3",
          "type" : "PERMANENT",
          "default_api_key" : "DF7ZW3wBN6QoF0S9O350:oa2FPHo2TUGTkMgS-t8x5Q",
          "policy_output_permissions_hash" : "dcb55dfca4c44666c4a2d0b6f2d851fbbc5d34d70dabdb95802379c204aad67d",
          "default_api_key_id" : "DF7ZW3wBN6QoF0S9O350",
          "updated_at" : "2021-10-08T22:23:37Z",
          "last_checkin_status" : "online",
          "last_checkin" : "2021-10-08T22:23:30Z",
          "policy_revision_idx" : 24,
          "policy_coordinator_idx" : 1
        },
        "sort" : [
          1631036278000
        ]
      },
      {
        "_index" : ".fleet-agents-7",
        "_type" : "_doc",
        "_id" : "a0b21013-c85f-4536-9acb-b4fdc1f4",
        "_score" : null,
        "_source" : {
          "access_api_key_id" : "zubVW3wBK1tuY2ttTtZb",
          "action_seq_no" : [
            -1
          ],
          "active" : true,
          "agent" : {
            "id" : "a0b21013-c85f-4536-9acb-b4fdc1f4",
            "version" : "7.15.0"
          },
          "enrolled_at" : "2021-09-07T17:37:58Z",
          "local_metadata" : {
            "elastic" : {
              "agent" : {
                "build.original" : "7.15.0 (build: 9023152025ec6251bc6b6c38009b309157f10f17 at 2021-09-16 06:31:45 +0000 UTC)",
                "id" : "a0b21013-c85f-4536-9acb-b4fdc1c072f2",
                "log_level" : "info",
                "snapshot" : false,
                "upgradeable" : true,
                "version" : "7.15.0"
              }
            },
            "host" : {
              "architecture" : "x86_64",
              "hostname" : "hostnamene.domanin.net",
              "id" : "3ea6f2e8be0249c297b52db0e47ca8da",
              "ip" : [
                "127.0.0.1/8",
                "10.10.123.23/24"
              ],
              "mac" : [
                "00:25:15:58:a4:2e",
                "20:25:b5:51:b5:2e"
              ],
              "name" : "what.domanin.net"
            },
            "os" : {
              "family" : "redhat",
              "full" : "Red Hat Enterprise Linux Server Maipo(7.9 (Maipo))",
              "kernel" : "3.10.0-1160.11.1.el7.x86_64",
              "name" : "Red Hat Enterprise Linux Server",
              "platform" : "rhel",
              "version" : "7.9 (Maipo)"
            }
          },
          "policy_id" : "6acc0290-003e-11ec-abcf-d1069794f7a3",
          "type" : "PERMANENT",
          "default_api_key" : "DF7ZW3wBN6QoF0S9O350:oa2FPHo2TUGTkMgS-t8x5Q",
          "policy_output_permissions_hash" : "dcb55dfca4c44666c4a2d0b6f2d851fbbc5d34d70dabdb95802379c204aad67d",
          "default_api_key_id" : "DF7ZW3wBN6QoF0S9O350",
          "updated_at" : "2021-10-08T22:23:37Z",
          "last_checkin_status" : "online",
          "last_checkin" : "2021-10-08T22:23:30Z",
          "policy_revision_idx" : 24,
          "policy_coordinator_idx" : 1
        },
        "sort" : [
          1631036278000
        ]
      },
      {
        "_index" : ".fleet-agents-7",
        "_type" : "_doc",
        "_id" : "a0b21013-c85f-4536-9acb-snf231s",
        "_score" : null,
        "_source" : {
          "access_api_key_id" : "zubVW3wBK1tuY2ttTtZb",
          "action_seq_no" : [
            -1
          ],
          "active" : true,
          "agent" : {
            "id" : "a0b21013-c85f-4536-9acb-dsame123s",
            "version" : "7.15.0"
          },
          "enrolled_at" : "2021-09-07T11:37:58Z",
          "local_metadata" : {
            "elastic" : {
              "agent" : {
                "build.original" : "7.15.0 (build: 9023152025ec6251bc6b6c38009b309157f10f17 at 2021-09-16 06:31:45 +0000 UTC)",
                "id" : "a0b21013-c85f-4536-9acb-b4fdc1c072f2",
                "log_level" : "info",
                "snapshot" : false,
                "upgradeable" : true,
                "version" : "7.15.0"
              }
            },
            "host" : {
              "architecture" : "x86_64",
              "hostname" : "hostnsdffamene.domanin.net",
              "id" : "3ea6f2e8be0249c297b52db0e47ca8da",
              "ip" : [
                "127.0.0.1/8",
                "10.10.123.23/24"
              ],
              "mac" : [
                "01:56:c1:58:b1:6e",
                "10:35:c5:59:c5:1e"
              ],
              "name" : "whaasdat.domanin.net"
            },
            "os" : {
              "family" : "redhat",
              "full" : "Red Hat Enterprise Linux Server Maipo(7.9 (Maipo))",
              "kernel" : "3.10.0-1160.11.1.el7.x86_64",
              "name" : "Red Hat Enterprise Linux Server",
              "platform" : "rhel",
              "version" : "7.9 (Maipo)"
            }
          },
          "policy_id" : "6acc0290-003e-11ec-abcf-d1069794f7a3",
          "type" : "PERMANENT",
          "default_api_key" : "DF7ZW3wBN6QoF0S9O350:oa2FPHo2TUGTkMgS-t8x5Q",
          "policy_output_permissions_hash" : "dcb55dfca4c44666c4a2d0b6f2d851fbbc5d34d70dabdb95802379c204aad67d",
          "default_api_key_id" : "DF7ZW3wBN6QoF0S9O350",
          "updated_at" : "2021-10-08T22:23:37Z",
          "last_checkin_status" : "online",
          "last_checkin" : "2021-10-08T22:23:30Z",
          "policy_revision_idx" : 24,
          "policy_coordinator_idx" : 1
        },
        "sort" : [
          1631014678000
        ]
      },
      {
        "_index" : ".fleet-agents-7",
        "_type" : "_doc",
        "_id" : "a0b21013-c85f-4536-nd2e-snf231s",
        "_score" : null,
        "_source" : {
          "access_api_key_id" : "zubVW3wBK1tuY2ttTtZb",
          "action_seq_no" : [
            -1
          ],
          "active" : true,
          "agent" : {
            "id" : "a0b21013-scase-4536-9acb-dsame123s",
            "version" : "7.15.0"
          },
          "enrolled_at" : "2021-09-07T11:27:58Z",
          "local_metadata" : {
            "elastic" : {
              "agent" : {
                "build.original" : "7.15.0 (build: 9023152025ec6251bc6b6c38009b309157f10f17 at 2021-09-16 06:31:45 +0000 UTC)",
                "id" : "a0b21013-c85f-4536-9acb-b4fdc1c072f2",
                "log_level" : "info",
                "snapshot" : false,
                "upgradeable" : true,
                "version" : "7.15.0"
              }
            },
            "host" : {
              "architecture" : "x86_64",
              "hostname" : "hosaaaaeene.domanin.net",
              "id" : "3ea6f2e8be0249c297b52db0e47ca8da",
              "ip" : [
                "127.0.0.1/8",
                "10.10.123.23/24"
              ],
              "mac" : [
                "d0:25:b5:52:b4:de",
                "40:35:25:52:b5:6e"
              ],
              "name" : "whaasdrwfat.domanin.net"
            },
            "os" : {
              "family" : "redhat",
              "full" : "Red Hat Enterprise Linux Server Maipo(7.9 (Maipo))",
              "kernel" : "3.10.0-1160.11.1.el7.x86_64",
              "name" : "Red Hat Enterprise Linux Server",
              "platform" : "rhel",
              "version" : "7.9 (Maipo)"
            }
          },
          "policy_id" : "6acc0290-003e-11ec-abcf-d1069794f7a3",
          "type" : "PERMANENT",
          "default_api_key" : "DF7ZW3wBN6QoF0S9O350:oa2FPHo2TUGTkMgS-t8x5Q",
          "policy_output_permissions_hash" : "dcb55dfca4c44666c4a2d0b6f2d851fbbc5d34d70dabdb95802379c204aad67d",
          "default_api_key_id" : "DF7ZW3wBN6QoF0S9O350",
          "updated_at" : "2021-10-08T22:23:37Z",
          "last_checkin_status" : "online",
          "last_checkin" : "2021-10-08T22:23:30Z",
          "policy_revision_idx" : 24,
          "policy_coordinator_idx" : 1
        },
        "sort" : [
          1631014078000
        ]
      },
      {
        "_index" : ".fleet-agents-7",
        "_type" : "_doc",
        "_id" : "a0b21013-c85f-4536-nd2e-sdasdvv",
        "_score" : null,
        "_source" : {
          "access_api_key_id" : "zubVW3wBK1tuY2ttTtZb",
          "action_seq_no" : [
            -1
          ],
          "active" : true,
          "agent" : {
            "id" : "a0b21013-scase-45sda36-9acb-dsame123s",
            "version" : "7.15.0"
          },
          "enrolled_at" : "2021-09-07T11:27:58Z",
          "local_metadata" : {
            "elastic" : {
              "agent" : {
                "build.original" : "7.15.0 (build: 9023152025ec6251bc6b6c38009b309157f10f17 at 2021-09-16 06:31:45 +0000 UTC)",
                "id" : "a0b21013-c85f-4536-9acb-b4fdc1c072f2",
                "log_level" : "info",
                "snapshot" : false,
                "upgradeable" : true,
                "version" : "7.15.0"
              }
            },
            "host" : {
              "architecture" : "x86_64",
              "hostname" : "hmmdweaaeene.domanin.net",
              "id" : "3ea6f2e8be0249c297b52db0e47ca8da",
              "ip" : [
                "127.0.0.1/8",
                "10.10.123.23/24"
              ],
              "mac" : [
                "bb:2b:b5:5b:b4:6e",
                "01:15:b8:58:f5:6e"
              ],
              "name" : "whaasdrwfat.domanin.net"
            },
            "os" : {
              "family" : "redhat",
              "full" : "Red Hat Enterprise Linux Server Maipo(7.9 (Maipo))",
              "kernel" : "3.10.0-1160.11.1.el7.x86_64",
              "name" : "Red Hat Enterprise Linux Server",
              "platform" : "rhel",
              "version" : "7.9 (Maipo)"
            }
          },
          "policy_id" : "6acc0290-003e-11ec-abcf-d1069794f7a3",
          "type" : "PERMANENT",
          "default_api_key" : "DF7ZW3wBN6QoF0S9O350:oa2FPHo2TUGTkMgS-t8x5Q",
          "policy_output_permissions_hash" : "dcb55dfca4c44666c4a2d0b6f2d851fbbc5d34d70dabdb95802379c204aad67d",
          "default_api_key_id" : "DF7ZW3wBN6QoF0S9O350",
          "updated_at" : "2021-10-08T22:23:37Z",
          "last_checkin_status" : "online",
          "last_checkin" : "2021-10-08T22:23:30Z",
          "policy_revision_idx" : 24,
          "policy_coordinator_idx" : 1
        },
        "sort" : [
          1631014078000
        ]
      },
      {
        "_index" : ".fleet-agents-7",
        "_type" : "_doc",
        "_id" : "a0b21013-cds25f-4536-nd2e-sdasdvv",
        "_score" : null,
        "_source" : {
          "access_api_key_id" : "zubVW3wBK1tuY2ttTtZb",
          "action_seq_no" : [
            -1
          ],
          "active" : true,
          "agent" : {
            "id" : "a0b21013-scase-45sd-9acb-dsame123s",
            "version" : "7.15.0"
          },
          "enrolled_at" : "2021-09-07T11:27:58Z",
          "local_metadata" : {
            "elastic" : {
              "agent" : {
                "build.original" : "7.15.0 (build: 9023152025ec6251bc6b6c38009b309157f10f17 at 2021-09-16 06:31:45 +0000 UTC)",
                "id" : "a0b21013-c85f-4536-9acb-b4fdc1c072f2",
                "log_level" : "info",
                "snapshot" : false,
                "upgradeable" : true,
                "version" : "7.15.0"
              }
            },
            "host" : {
              "architecture" : "x86_64",
              "hostname" : "hmmdwesdaaaaeene.domanin.net",
              "id" : "3ea6f2e8be0249c297b52db0e47ca8da",
              "ip" : [
                "127.0.0.1/8",
                "10.10.123.23/24"
              ],
              "mac" : [
                "bb:25:b1:58:a4:d3",
                "cc:25:f5:58:b5:6e"
              ],
              "name" : "whaasdrwfat.domanin.net"
            },
            "os" : {
              "family" : "redhat",
              "full" : "Red Hat Enterprise Linux Server Maipo(7.9 (Maipo))",
              "kernel" : "3.10.0-1160.11.1.el7.x86_64",
              "name" : "Red Hat Enterprise Linux Server",
              "platform" : "rhel",
              "version" : "7.9 (Maipo)"
            }
          },
          "policy_id" : "6acc0290-003e-11ec-abcf-d1069794f7a3",
          "type" : "PERMANENT",
          "default_api_key" : "DF7ZW3wBN6QoF0S9O350:oa2FPHo2TUGTkMgS-t8x5Q",
          "policy_output_permissions_hash" : "dcb55dfca4c44666c4a2d0b6f2d851fbbc5d34d70dabdb95802379c204aad67d",
          "default_api_key_id" : "DF7ZW3wBN6QoF0S9O350",
          "updated_at" : "2021-10-08T22:23:37Z",
          "last_checkin_status" : "online",
          "last_checkin" : "2021-10-08T22:23:30Z",
          "policy_revision_idx" : 24,
          "policy_coordinator_idx" : 1
        },
        "sort" : [
          1631014078000
        ]
      },
      {
        "_index" : ".fleet-agents-7",
        "_type" : "_doc",
        "_id" : "a0dsfa1013-cds25f-4536-nd2e-sdasdvv",
        "_score" : null,
        "_source" : {
          "access_api_key_id" : "zubVW3wBK1tuY2ttTtZb",
          "action_seq_no" : [
            -1
          ],
          "active" : true,
          "agent" : {
            "id" : "a0bdsw21013-scase-45sd-9acb-dsame123s",
            "version" : "7.15.0"
          },
          "enrolled_at" : "2021-09-07T11:27:58Z",
          "local_metadata" : {
            "elastic" : {
              "agent" : {
                "build.original" : "7.15.0 (build: 9023152025ec6251bc6b6c38009b309157f10f17 at 2021-09-16 06:31:45 +0000 UTC)",
                "id" : "a0b21013-c85f-4536-9acb-b4fdc1c072f2",
                "log_level" : "info",
                "snapshot" : false,
                "upgradeable" : true,
                "version" : "7.15.0"
              }
            },
            "host" : {
              "architecture" : "x86_64",
              "hostname" : "aaeene.domanin.net",
              "id" : "3ea6f2e8be0249c297b52db0e47ca8da",
              "ip" : [
                "127.0.0.1/8",
                "10.10.123.23/24"
              ],
              "mac" : [
                "aa:25:b5:58:bb:bb",
                "dd:2d:b5:48:bb:bb"
              ],
              "name" : "whaasdafrwfat.domanin.net"
            },
            "os" : {
              "family" : "redhat",
              "full" : "Red Hat Enterprise Linux Server Maipo(7.9 (Maipo))",
              "kernel" : "3.10.0-1160.11.1.el7.x86_64",
              "name" : "Red Hat Enterprise Linux Server",
              "platform" : "rhel",
              "version" : "7.9 (Maipo)"
            }
          },
          "policy_id" : "6acc0290-003e-11ec-abcf-d1069794f7a3",
          "type" : "PERMANENT",
          "default_api_key" : "DF7ZW3wBN6QoF0S9O350:oa2FPHo2TUGTkMgS-t8x5Q",
          "policy_output_permissions_hash" : "dcb55dfca4c44666c4a2d0b6f2d851fbbc5d34d70dabdb95802379c204aad67d",
          "default_api_key_id" : "DF7ZW3wBN6QoF0S9O350",
          "updated_at" : "2021-10-08T22:23:37Z",
          "last_checkin_status" : "online",
          "last_checkin" : "2021-10-08T22:23:30Z",
          "policy_revision_idx" : 24,
          "policy_coordinator_idx" : 1
        },
        "sort" : [
          1631014078000
        ]
      }
    ]
  }
}

Doing an explicit lookup for the hostname:

hostname testhost 
GET .fleet-agents/_search
{
  "size": 20,
  "query": {
  "bool": {
                              "must": [
                                {
                                  "query_string": {
                                    "query": "*testhost*",
                                    "fields": [
                                      "local_metadata.host.hostname^1.0"
                                    ],
                                    "type": "best_fields",
                                    "default_operator": "or",
                                    "max_determinized_states": 10000,
                                    "enable_position_increments": true,
                                    "fuzziness": "AUTO",
                                    "fuzzy_prefix_length": 0,
                                    "fuzzy_max_expansions": 50,
                                    "phrase_slop": 0,
                                    "escape": false,
                                    "auto_generate_synonyms_phrase_query": true,
                                    "fuzzy_transpositions": true,
                                    "boost": 1
                                  }
                                }
                              ]
  }
}
}
result 
{
  "took" : 945,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : ".fleet-agents-7",
        "_type" : "_doc",
        "_id" : "a0dsfa1013-cdcdw25f-2536-1e-sdasdvv",
        "_score" : 1.0,
        "_source" : {
          "access_api_key_id" : "zubVW3wBK1tuY2ttftZb",
          "action_seq_no" : [
            -1
          ],
          "active" : true,
          "agent" : {
            "id" : "a0bdsw21se-45d-qweac-dsamda1a23s",
            "version" : "7.15.0"
          },
          "enrolled_at" : "2021-08-07T11:27:58Z",
          "local_metadata" : {
            "elastic" : {
              "agent" : {
                "build.original" : "7.15.0 (build: 9023152025ec6251bc6b6c38009b309157f10f17 at 2021-09-16 06:31:45 +0000 UTC)",
                "id" : "a0b210-cs85f12-4536-9asfb-b4fdc1ds72f2",
                "log_level" : "info",
                "snapshot" : false,
                "upgradeable" : true,
                "version" : "7.15.0"
              }
            },
            "host" : {
              "architecture" : "x86_64",
              "hostname" : "testhost",
              "id" : "3ea6f2e82be0249c17b52db0es47ca8da",
              "ip" : [
                "127.0.0.1/8",
                "10.10.12.23/24"
              ],
              "mac" : [
                "bb:21:b5:28:a4:2e",
                "bb:21:c4:33:b1:1e"
              ],
              "name" : "testhost"
            },
            "os" : {
              "family" : "redhat",
              "full" : "Red Hat Enterprise Linux Server Maipo(7.9 (Maipo))",
              "kernel" : "3.10.0-1160.11.1.el7.x86_64",
              "name" : "Red Hat Enterprise Linux Server",
              "platform" : "rhel",
              "version" : "7.9 (Maipo)"
            }
          },
          "policy_id" : "6acc02s90-sa003e-11ffaeac-abcf-d106xzxc9794f7a3",
          "type" : "PERMANENT",
          "default_api_key" : "DF7ZW3wBN6QoF0S9O350:oa2FPHo2TUGTkMgS-t8x5Q",
          "policy_output_permissions_hash" : "dcb55dfca4c44666c4a2d0b6f2d851fbbc5d34d70dabdb95802379c204aad67d",
          "default_api_key_id" : "DF7ZW3wBN6QoF0S9O350",
          "updated_at" : "2021-10-08T22:23:37Z",
          "last_checkin_status" : "online",
          "last_checkin" : "2021-10-08T22:23:30Z",
          "policy_revision_idx" : 24,
          "policy_coordinator_idx" : 1
        }
      }
    ]
  }
}
@TheRiffRafi TheRiffRafi added the bug Fixes for quality problems that affect the customer experience label Oct 14, 2021
@botelastic botelastic bot added the needs-team Issues missing a team label label Oct 14, 2021
@melissaburpo melissaburpo added Team:Asset Management Security Asset Management Team and removed needs-team Issues missing a team label labels Oct 14, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-asset-management (Team:Asset Management)

@patrykkopycinski patrykkopycinski mentioned this issue Oct 26, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@patrykkopycinski patrykkopycinski

Labels
bug Fixes for quality problems that affect the customer experience Team:Asset Management Security Asset Management Team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Osquery] Fix live query search doesn't return relevant results for agents patrykkopycinski/kibana
4 participants
@patrykkopycinski @melissaburpo @elasticmachine @TheRiffRafi