[Lens] Indexpattern datasource persistence issue layer not cleaned up

[flash1293]

Kibana version: 7.9 BC2

Describe the bug:

It seems like when saving the state, sometimes old configuration makes its way back in.

Steps to reproduce:

1. Go to Lens
2. Create bar chart over time
3. Save visualization
4. Switch to pie
5. Create single layered pie with terms agg
6. Save visualization
7. Go to listing page and load visualization again
8. "An error occurred when loading data. Cannot read property 'id' of undefined"
9. Switch to table
10. See date histogram on time field column

Expected behavior:

• Should load pie just fine
• There should be no date histogram column in the table

Errors in browser console (if relevant):

Uncaught TypeError: Cannot read property 'id' of undefined
    at render_function.tsx:196
    at Array.some (<anonymous>)
    at PieComponent (render_function.tsx:195)
    at renderWithHooks (react-dom.development.js:16260)
    at updateFunctionComponent (react-dom.development.js:18347)
    at updateSimpleMemoComponent (react-dom.development.js:18285)
    at updateMemoComponent (react-dom.development.js:18188)
    at beginWork$1 (react-dom.development.js:20248)
    at HTMLUnknownElement.callCallback (react-dom.development.js:336)
    at Object.invokeGuardedCallbackDev (react-dom.development.js:385)

Any additional context:

This is only happening when saving the visualization in between, when omitting step 3 everything works fine. It seems like there are two layers in the datasource with only one of them mapped to the visualization, causing issues.

The console error is caused by the pie renderer not finding the metric column (accessing id of undefined in the process). This happens because it's the wrong layer as the pie function just takes the first one out of the layer map, expecting it to be the only one (which it should be): https://github.com/flash1293/kibana/blob/f8ba824ebc7e3d551b791ba7b8c04f31cddcaf10/x-pack/plugins/lens/public/pie_visualization/render_function.tsx#L47

I suspect it's picking the wrong state at some point during serializing/deserializing

This is the saved object - it contains two layers instead of one, seems like the layer is not cleared up correctly:

{
  "attributes": {
    "description": "",
    "expression": "kibana\n| kibana_context  query=\"{\\\"query\\\":\\\"\\\",\\\"language\\\":\\\"kuery\\\"}\" filters=\"[]\"\n| lens_merge_tables layerIds=\"1a4dcddf-905b-4967-9acf-2f661682b597\" \n  tables={esaggs index=\"6c7b9450-cc11-11ea-9b3e-df4dbdb4f797\" metricsAtAllLevels=true partialRows=true includeFormatHints=true  aggConfigs=\"[{\\\"id\\\":\\\"f243d848-c32c-4c79-95a5-8daac974aa58\\\",\\\"enabled\\\":true,\\\"type\\\":\\\"terms\\\",\\\"schema\\\":\\\"segment\\\",\\\"params\\\":{\\\"field\\\":\\\"geo.src\\\",\\\"orderBy\\\":\\\"66f94a51-5a99-430a-9a6c-ebaea6b364df\\\",\\\"order\\\":\\\"desc\\\",\\\"size\\\":5,\\\"otherBucket\\\":false,\\\"otherBucketLabel\\\":\\\"Other\\\",\\\"missingBucket\\\":false,\\\"missingBucketLabel\\\":\\\"Missing\\\"}},{\\\"id\\\":\\\"66f94a51-5a99-430a-9a6c-ebaea6b364df\\\",\\\"enabled\\\":true,\\\"type\\\":\\\"count\\\",\\\"schema\\\":\\\"metric\\\",\\\"params\\\":{}}]\" | lens_rename_columns idMap=\"{\\\"col-0-f243d848-c32c-4c79-95a5-8daac974aa58\\\":{\\\"label\\\":\\\"Top values of geo.src\\\",\\\"dataType\\\":\\\"string\\\",\\\"operationType\\\":\\\"terms\\\",\\\"scale\\\":\\\"ordinal\\\",\\\"sourceField\\\":\\\"geo.src\\\",\\\"isBucketed\\\":true,\\\"params\\\":{\\\"size\\\":5,\\\"orderBy\\\":{\\\"type\\\":\\\"column\\\",\\\"columnId\\\":\\\"66f94a51-5a99-430a-9a6c-ebaea6b364df\\\"},\\\"orderDirection\\\":\\\"desc\\\"},\\\"id\\\":\\\"f243d848-c32c-4c79-95a5-8daac974aa58\\\"},\\\"col-1-66f94a51-5a99-430a-9a6c-ebaea6b364df\\\":{\\\"label\\\":\\\"Count of records\\\",\\\"dataType\\\":\\\"number\\\",\\\"operationType\\\":\\\"count\\\",\\\"isBucketed\\\":false,\\\"scale\\\":\\\"ratio\\\",\\\"sourceField\\\":\\\"Records\\\",\\\"id\\\":\\\"66f94a51-5a99-430a-9a6c-ebaea6b364df\\\"}}\"}\n| lens_pie shape=\"donut\" hideLabels=false groups=\"f243d848-c32c-4c79-95a5-8daac974aa58\" metric=\"66f94a51-5a99-430a-9a6c-ebaea6b364df\" numberDisplay=\"percent\" categoryDisplay=\"default\" legendDisplay=\"default\" percentDecimals=3 nestedLegend=false",
    "state": {
      "datasourceMetaData": {
        "filterableIndexPatterns": [
          { "id": "6c7b9450-cc11-11ea-9b3e-df4dbdb4f797", "title": "*logs*" }
        ]
      },
      "datasourceStates": {
        "indexpattern": {
          "currentIndexPatternId": "6c7b9450-cc11-11ea-9b3e-df4dbdb4f797",
          "layers": {
            "1a4dcddf-905b-4967-9acf-2f661682b597": {
              "columnOrder": [
                "f243d848-c32c-4c79-95a5-8daac974aa58",
                "66f94a51-5a99-430a-9a6c-ebaea6b364df"
              ],
              "columns": {
                "66f94a51-5a99-430a-9a6c-ebaea6b364df": {
                  "dataType": "number",
                  "isBucketed": false,
                  "label": "Count of records",
                  "operationType": "count",
                  "scale": "ratio",
                  "sourceField": "Records"
                },
                "f243d848-c32c-4c79-95a5-8daac974aa58": {
                  "dataType": "string",
                  "isBucketed": true,
                  "label": "Top values of geo.src",
                  "operationType": "terms",
                  "params": {
                    "orderBy": {
                      "columnId": "66f94a51-5a99-430a-9a6c-ebaea6b364df",
                      "type": "column"
                    },
                    "orderDirection": "desc",
                    "size": 5
                  },
                  "scale": "ordinal",
                  "sourceField": "geo.src"
                }
              },
              "indexPatternId": "6c7b9450-cc11-11ea-9b3e-df4dbdb4f797"
            },
            "bd7f6e2c-703e-485d-8492-7cd71e5dd987": {
              "columnOrder": [
                "af40811f-3a2b-4328-bc36-24ca3c09236f",
                "03a8cb08-cea9-4f90-ba57-ac1b8718d1ff"
              ],
              "columns": {
                "03a8cb08-cea9-4f90-ba57-ac1b8718d1ff": {
                  "dataType": "number",
                  "isBucketed": false,
                  "label": "Count of records",
                  "operationType": "count",
                  "scale": "ratio",
                  "sourceField": "Records"
                },
                "af40811f-3a2b-4328-bc36-24ca3c09236f": {
                  "dataType": "date",
                  "isBucketed": true,
                  "label": "@timestamp",
                  "operationType": "date_histogram",
                  "params": { "interval": "auto" },
                  "scale": "interval",
                  "sourceField": "@timestamp"
                }
              },
              "indexPatternId": "6c7b9450-cc11-11ea-9b3e-df4dbdb4f797"
            }
          }
        }
      },
      "filters": [],
      "query": { "language": "kuery", "query": "" },
      "visualization": {
        "fittingFunction": "None",
        "layers": [
          {
            "categoryDisplay": "default",
            "groups": ["f243d848-c32c-4c79-95a5-8daac974aa58"],
            "layerId": "1a4dcddf-905b-4967-9acf-2f661682b597",
            "legendDisplay": "default",
            "metric": "66f94a51-5a99-430a-9a6c-ebaea6b364df",
            "nestedLegend": false,
            "numberDisplay": "percent"
          }
        ],
        "legend": { "isVisible": true, "position": "right" },
        "preferredSeriesType": "bar_stacked",
        "shape": "donut"
      }
    },
    "title": "b",
    "visualizationType": "lnsPie"
  },
  "id": "e7ffceb0-cc21-11ea-9b3e-df4dbdb4f797",
  "migrationVersion": { "lens": "7.8.0" },
  "references": [],
  "type": "lens",
  "updated_at": "2020-07-22T13:47:45.664Z",
  "version": "WzY1NSwxXQ=="
}

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[wylieconlon]

This looks like the kind of error we've seen due to the way layers are removed in the chart switcher. The thing that stands out about the steps you've described to reproduce is that pie charts don't support date histograms, so when you switch from XY -> Pie you are supposed to get a blank state. It's the responsibility of the chart switcher to remove layers in cases like this. Maybe what's happening is that there is a bug or race condition in the chart switcher which is being triggered in this case.