Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] upgrade filter bug #403

Closed
kaituo opened this issue Jan 28, 2023 · 2 comments
Closed

[BUG] upgrade filter bug #403

kaituo opened this issue Jan 28, 2023 · 2 comments
Labels
bug Something isn't working

Comments

@kaituo
Copy link
Collaborator

kaituo commented Jan 28, 2023

What is the bug?
When upgrading an old detector, we have difficulty updating filter.

How can one reproduce the bug?
Steps to reproduce the behavior:
1)create a 7.10.2 cluster
2)create a detector with 2+ filters like

{
  "_id" : "oNwA9YUBsE8lmktYbzIg",
  "_version" : 5,
  "_primary_term" : 1,
  "_seq_no" : 4,
  "anomaly_detector" : {
    "name" : "single",
    "description" : "",
    "time_field" : "@timestamp",
    "indices" : [
      "server-metrics"
    ],
    "filter_query" : {
      "bool" : {
        "filter" : [
          {
            "term" : {
              "service" : {
                "value" : "app_3",
                "boost" : 1.0
              }
            }
          },
          {
            "term" : {
              "host" : {
                "value" : "server_2",
                "boost" : 1.0
              }
            }
          }
        ],
        "adjust_pure_negative" : true,
        "boost" : 1.0
      }
    },
    "detection_interval" : {
      "period" : {
        "interval" : 1,
        "unit" : "Minutes"
      }
    },
    "window_delay" : {
      "period" : {
        "interval" : 1,
        "unit" : "Minutes"
      }
    },
    "shingle_size" : 8,
    "schema_version" : 0,
    "feature_attributes" : [
      {
        "feature_id" : "Zt8E9YUBWn0Gyxrz0mwI",
        "feature_name" : "denyMax",
        "feature_enabled" : true,
        "aggregation_query" : {
          "deny_max" : {
            "max" : {
              "field" : "deny"
            }
          }
        }
      }
    ],
    "ui_metadata" : {
      "features" : {
        "denyMax" : {
          "aggregationBy" : "max",
          "aggregationOf" : "deny",
          "featureType" : "simple_aggs"
        }
      },
      "filters" : [
        {
          "fieldInfo" : [
            {
              "label" : "service",
              "type" : "keyword"
            }
          ],
          "fieldValue" : "app_3",
          "operator" : "is"
        },
        {
          "fieldInfo" : [
            {
              "label" : "host",
              "type" : "keyword"
            }
          ],
          "fieldValue" : "server_2",
          "operator" : "is"
        }
      ],
      "filterType" : "simple_filter"
    },
    "last_update_time" : 1674853271580,
    "user" : {
      "name" : "admin123",
      "backend_roles" : [ ],
      "roles" : [
        "security_manager",
        "all_access"
      ],
      "custom_attribute_names" : [ ],
      "user_requested_tenant" : null
    },
    "detector_type" : "SINGLE_ENTITY"
  }
}

Note there was only one filter type per detector.
3) upgrade to 1.3.2
4)update one filter. It won't allow you to save the updates. And exceptions from console:

Detected an unhandled Promise rejection. SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data[fatal_errors_service.tsx:120](http://localhost:5601/entry:core/src/core/public/fatal_errors/fatal_errors_service.tsx)
Uncaught (in promise) SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data
formikToFilterQueryhelpers.ts:84
formikToFilterQueryhelpers.ts:74
formikToDetectorDefinitionhelpers.ts:82
handleFormValidationDefineDetector.tsx:126
promise callback*handleFormValidationDefineDetector.tsx:123
onClickDefineDetector.tsx:214
React17
unstable_runWithPriorityscheduler.development.js:697
React19
unstable_runWithPriorityscheduler.development.js:697
React6
startrendering_service.tsx:55
startcore_system.ts:276
__osdBootstrap__osd_bootstrap.ts:56
onloadbootstrap.js:154
innerCbbootstrap.js:91
loadScriptbootstrap.js:81
loadbootstrap.js:100
loadbootstrap.js:87
onloadbootstrap.js:105
EventHandlerNonNull*bootstrap.js:48
[helpers.ts:84](http://localhost:5601/plugin:anomalyDetectionDashboards/plugins/anomaly-detection-dashboards-plugin/public/pages/ReviewAndCreate/utils/helpers.ts)

What is the expected behavior?
We should be able to apply the updates.

What is your host/environment?

  • OS: ES 7.10, OS 1.3
  • Plugins: AD
@kaituo kaituo added bug Something isn't working untriaged and removed untriaged labels Jan 28, 2023
@kaituo kaituo mentioned this issue Jan 28, 2023
4 tasks
@kaituo
Copy link
Collaborator Author

kaituo commented Jan 28, 2023

Root cause: we didn't actually add filter type when loading old detector.

Impact: after upgrading from a version before OS 1.1 to later versions, we cannnot update detector using frontend unless resaving all filters (if any).

Fix: add filter type using new way when loading old detector. When saving, all filters are added filter type.

@kaituo
Copy link
Collaborator Author

kaituo commented Jan 31, 2023

fixed in #402

@kaituo kaituo closed this as completed Jan 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant