Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Anomalies chart doesn't plot values for datafeeds comprising aggregations #65223

Closed
bczifra opened this issue May 5, 2020 · 1 comment
Closed
Assignees
Labels
bug Fixes for quality problems that affect the customer experience Feature:Anomaly Detection ML anomaly detection :ml v7.11.0

Comments

@bczifra
Copy link
Member

bczifra commented May 5, 2020

Kibana version: 7.6.1

Elasticsearch version: 7.6.1

Browser version: Chrome 81.0.4044.129 (Official Build) (64-bit)

Browser OS version: macOS High Sierra 10.13.6

Describe the bug:
When using datafeeds which contain aggregations, the anomalies chart doesn't plot values. Instead, it looks like this, with all values plotted at 0:
incorrect_chart

Steps to reproduce:
Mappings:

PUT activation
{
  "mappings" : {
      "properties" : {
        "@timestamp" : {
          "type" : "date"
        },
        "rate_plan_id": {
          "type": "keyword"
        }
      }
    }
}

PUT creation
{
  "mappings": {
    "properties": {
      "@timestamp": {
        "type": "date"
      },
      "rate_plan_id": {
        "type": "keyword"
      }
    }
  }
}

Job:

PUT _ml/anomaly_detectors/creation_activation_per_rate_plan
{
  "job_id": "creation_activation_per_rate_plan",
  "description": "",
  "groups": ["rate_plan_analysis"],
  "analysis_config": {
    "summary_count_field_name" : "doc_count",    
    "bucket_span": "15s",
    "detectors": [
      {
        "detector_description" : """mean("create_active_diff")""",
        "function" : "mean",
        "field_name" : "create_active_diff",
        "partition_field_name": "rate_plan",
        "detector_index" : 0
      }
    ],
    "influencers": [
      "rate_plan"
    ]
  },
  "data_description": {
    "time_field": "@timestamp"
  },
  "analysis_limits": {
    "model_memory_limit": "12MB"
  }
}

Datafeed:

PUT _ml/datafeeds/creation_activate_per_rate_plan
{
  "job_id": "creation_activation_per_rate_plan", 
  "indices": [
    "activation",
    "creation"
  ],
  "query_delay": "10000ms",
  "query": {
    "bool": {
      "must": [
        {
          "match_all": {}
        }
      ]
    }
  },
  "aggs": {
    "buckets": {
      "date_histogram": {
        "field": "@timestamp",
        "fixed_interval": "15s"
      },
      "aggs": {
        "@timestamp": {
          "max": {
            "field": "@timestamp"
          }
        },
        "rate_plan": {
          "terms": {
            "field": "rate_plan_id",
            "size": 10
          },
          "aggs": {
            "creations": {
              "filter": {
                "bool": {
                  "filter": {
                    "term": {
                      "_index": "creation"
                    }
                  }
                }
              }
            },
            "activations": {
              "filter": {
                "bool": {
                  "filter": {
                    "term": {
                      "_index": "activation"
                    }
                  }
                }
              }
            },
            "create_active_diff": {
              "bucket_script": {
                "buckets_path": {
                  "creations": "creations>_count",
                  "activations": "activations>_count"
                },
                "script": "params.creations - params.activations"
              }
            }
          }
        }
      }
    }
  }
}

Expected behavior:
The chart should either be disabled if we can't plot values, or values should be plotted correctly.

@peteharverson peteharverson changed the title Anomalies chart doesn't plot values for datafeeds comprising aggregations [ML] Anomalies chart doesn't plot values for datafeeds comprising aggregations May 5, 2020
@peteharverson peteharverson added :ml Feature:Anomaly Detection ML anomaly detection bug Fixes for quality problems that affect the customer experience labels May 5, 2020
@qn895
Copy link
Member

qn895 commented Dec 10, 2020

Closing via #81923 and #84594

@qn895 qn895 closed this as completed Dec 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience Feature:Anomaly Detection ML anomaly detection :ml v7.11.0
Projects
None yet
Development

No branches or pull requests

3 participants