Transform bug fixes (#109)

* Rename plugin helper file * Update TransformIndices.tsx * Backport commits from main (#90) * Release 1.1.0.0 (#83) * Release 1.1.0.0 Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Correct copyright notices * UI fixes for new ISM UI (#84) * Removes X icon from action/transition flyout footer next to cancel Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Adds tooltips to edit/delete icon buttons on the draggable action/transition components Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Removes overlay for flyout so clicking outside doesn't close flyout and removes X close button on top right Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Adds edit/delete tooltips for state component and removes underline from hovering states Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Changes state flyout from X Close to Cancel Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Adds JSON editor for allocation action Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Adds isValid to action interface and implements in actions Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Correctly removes unused rollover keys from rollover object, handles uncontrolled inputs and NaN minDocs Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Adds error for state name already existing in policy Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Fixes duplicated actions when editing as previously we had a new ID and matching logic didn't find an action Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Fixes other action inputs that can return NaN and creates a no conditions option that is the default Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Updates snapshot Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Correctly show danger toast on update/create failures Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Fixes rollup action nesting multiple ism_template keys Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Fixes timeout/retry settings turning into uncontrolled inputs and NaN value Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Removes text transformation on inputs and updates isValid methods and passes to form labels Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Fixes allocation and rollup adding UIAction properties to policy JSON on updates Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Updates release notes w/ new PR changes Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Wraps actions/transitions in states component and updates default rollup to include ism_rollup key Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Fixes some small UI touchups/issues for new ISM UI (#85) * Moves cancel secondary button next to primary on action/transition flyout Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Removes default ISM template and increases width of empty prompt Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Adds punctuation Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Adds back X icon next to cancel for state flyout Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Makes inputs in flyout full width, updates help text, and adds some spacing Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Updates transition default to not include conditions empty object, and on change transition to delete conditions when selecting none Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Moves edit button out of policy settings content panel and on to global page for view policy Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Updates release note Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Fixes broken link Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Fixes small issues on new ISM UI (#88) * Fixes small issues on new ISM UI Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Updates snapshots Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Updates release notes Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> * Updates workflows to trigger on 1.* branches Signed-off-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> Co-authored-by: Bowen Lan <62091230+bowenlan-amzn@users.noreply.github.com> * Support data filter when viewing sample data * Update TransformOptions.tsx * Add TransformService.test.ts * Draft of date histogram panel * Add helper method and move import * Split calendar and fixed interval to 2 panels * Add rendering test * Add close popover test * Update cypress-workflow.yml * Update links.yml * Update unit-tests-workflow.yml * Update TransformOptions.tsx * Update tests * Update DateHistogramPanel.test.tsx * Make scripted metrics editor larger * Expand code editor related panels * Undo changes to workflow Signed-off-by: Annie Lee <leeyun@amazon.com> * Update TimeAggregation.tsx * Update DefineTransforms.tsx undo changes to style * Clean up code * Update snapshot Signed-off-by: Annie Lee <leeyun@amazon.com> * Refactor the order of checking if name is defined before checking for duplicate name * Update CreateTransformForm.tsx * Update DefineTransforms.tsx Signed-off-by: Annie Lee <leeyun@amazon.com> * Update DefineTransforms.tsx Signed-off-by: Annie Lee <leeyun@amazon.com> * Update DefineTransforms.tsx * Update DefineTransforms.tsx * Update DefineTransforms.tsx Co-authored-by: Drew Baugher <46505179+dbbaughe@users.noreply.github.com> Co-authored-by: Bowen Lan <62091230+bowenlan-amzn@users.noreply.github.com>
opensearch-project · Oct 19, 2021 · 6f304e3 · 6f304e3
1 parent b0b7143
commit 6f304e3
Show file tree

Hide file tree

Showing 23 changed files with 967 additions and 239 deletions.
diff --git a/public/pages/CreateRollup/components/RollupIndices/RollupIndices.tsx b/public/pages/CreateRollup/components/RollupIndices/RollupIndices.tsx
@@ -26,12 +26,13 @@
 
 import React, { Component, Fragment } from "react";
 import { EuiSpacer, EuiFormRow, EuiComboBox, EuiCallOut } from "@elastic/eui";
-import { ContentPanel } from "../../../../components/ContentPanel";
 import { EuiComboBoxOptionOption } from "@elastic/eui/src/components/combo_box/types";
+import _ from "lodash";
+import { ContentPanel } from "../../../../components/ContentPanel";
 import { IndexItem } from "../../../../../models/interfaces";
 import IndexService from "../../../../services/IndexService";
-import _ from "lodash";
 import { CoreServicesContext } from "../../../../components/core_services";
+import { wildcardOption } from "../../../../utils/helpers";
 
 interface RollupIndicesProps {
   indexService: IndexService;
@@ -73,7 +74,8 @@ export default class RollupIndices extends Component<RollupIndicesProps, RollupI
     try {
       const dataStreamsAndIndicesNamesResponse = await indexService.getDataStreamsAndIndicesNames(searchValue);
       if (dataStreamsAndIndicesNamesResponse.ok) {
-        const options = searchValue.trim() ? [{ label: `${searchValue}*` }] : [];
+        // Adding wildcard to search value
+        const options = searchValue.trim() ? [{ label: wildcardOption(searchValue) }] : [];
         const dataStreams = dataStreamsAndIndicesNamesResponse.response.dataStreams.map((label) => ({ label }));
         const indices = dataStreamsAndIndicesNamesResponse.response.indices.map((label) => ({ label }));
         this.setState({ indexOptions: options.concat(dataStreams, indices), targetIndexOptions: indices });

diff --git a/public/pages/CreateRollup/components/TimeAggregations/TimeAggregation.tsx b/public/pages/CreateRollup/components/TimeAggregations/TimeAggregation.tsx
@@ -41,10 +41,10 @@ import {
   EuiHorizontalRule,
   EuiText,
 } from "@elastic/eui";
-import { CalendarTimeunitOptions, FixedTimeunitOptions } from "../../utils/constants";
-import { RollupService } from "../../../../services";
 import moment from "moment-timezone";
+import { RollupService } from "../../../../services";
 import { FieldItem } from "../../../../../models/interfaces";
+import { CalendarTimeunitOptions, FixedTimeunitOptions } from "../../../../utils/constants";
 
 interface TimeAggregationProps {
   rollupService: RollupService;

diff --git a/public/pages/CreateRollup/utils/constants.ts b/public/pages/CreateRollup/utils/constants.ts
@@ -53,31 +53,13 @@ export const EMPTY_ROLLUP = JSON.stringify({
   },
 });
 
-export const FixedTimeunitOptions = [
-  { value: "ms", text: "Millisecond(s)" },
-  { value: "s", text: "Second(s)" },
-  { value: "m", text: "Minute(s)" },
-  { value: "h", text: "Hour(s)" },
-  { value: "d", text: "Day(s)" },
-];
-
 export const DelayTimeunitOptions = [
   { value: "SECONDS", text: "Second(s)" },
   { value: "MINUTES", text: "Minute(s)" },
   { value: "HOURS", text: "Hour(s)" },
   { value: "DAYS", text: "Day(s)" },
 ];
 
-export const CalendarTimeunitOptions = [
-  { value: "m", text: "Minute" },
-  { value: "h", text: "Hour" },
-  { value: "d", text: "Day" },
-  { value: "w", text: "Week" },
-  { value: "M", text: "Month" },
-  { value: "q", text: "Quarter" },
-  { value: "y", text: "Year" },
-];
-
 export const ScheduleIntervalTimeunitOptions = [
   { value: "MINUTES", text: "Minute(s)" },
   { value: "HOURS", text: "Hour(s)" },

diff --git a/public/pages/CreateTransform/components/DefineTransforms/DefineTransforms.tsx b/public/pages/CreateTransform/components/DefineTransforms/DefineTransforms.tsx
@@ -25,6 +25,7 @@ interface DefineTransformsProps {
   transformService: TransformService;
   notifications: CoreStart["notifications"];
   sourceIndex: string;
+  sourceIndexFilter: string;
   fields: FieldItem[];
   selectedGroupField: TransformGroupItem[];
   onGroupSelectionChange: (selectedFields: TransformGroupItem[], aggItem: TransformAggItem) => void;
@@ -41,6 +42,7 @@ export default function DefineTransforms({
   transformService,
   notifications,
   sourceIndex,
+  sourceIndexFilter,
   fields,
   selectedGroupField,
   onGroupSelectionChange,
@@ -99,7 +101,7 @@ export default function DefineTransforms({
   const fetchData = useCallback(async () => {
     setLoading(true);
     try {
-      const response = await transformService.searchSampleData(sourceIndex, { from: 0, size: DefaultSampleDataSize });
+      const response = await transformService.searchSampleData(sourceIndex, { from: 0, size: DefaultSampleDataSize }, sourceIndexFilter);
 
       if (response.ok) {
         setData(response.response.data);
@@ -151,8 +153,7 @@ export default function DefineTransforms({
       } else if (columns?.find((column) => column.id == columnId).schema == "date") {
         return data[rowIndex]._source[columnId] ? renderTime(data[rowIndex]._source[columnId]) : "-";
       } else if (columns?.find((column) => column.id == columnId).schema == "geo_point") {
-        return data[rowIndex]._source[columnId].lat + ", " + data[rowIndex]._source[columnId].lon ?
-          data[rowIndex]._source[columnId].lat + ", " + data[rowIndex]._source[columnId].lon : "-";
+        return data[rowIndex].source[columndId] ? data[rowIndex]._source[columnId].lat + ", " + data[rowIndex]._source[columnId].lon : "-";
       } else if (columns?.find((column) => column.id == columnId).schema == "boolean") {
         return data[rowIndex]._source[columnId] == null ? "-" : (data[rowIndex]._source[columnId] ? "true" : "false");
       }

diff --git a/public/pages/CreateTransform/components/IndexFilterPopover/IndexFilterPopover.tsx b/public/pages/CreateTransform/components/IndexFilterPopover/IndexFilterPopover.tsx
@@ -96,13 +96,18 @@ export default function IndexFilterPopover({
   function customEditor() {
     return (
       <EuiFormRow label="Custom query DSL">
-        <EuiCodeEditor value={queryDsl} onChange={(string) => setQueryDsl(string)} mode="json" width="100%" height="250px" />
+        <EuiCodeEditor
+          style={{ width: 0.38 * window.innerWidth, height: 0.4 * window.innerHeight }}
+          value={queryDsl}
+          onChange={(string) => setQueryDsl(string)}
+          mode="json"
+        />
       </EuiFormRow>
     );
   }
 
   return (
-    <div>
+    <div style={{ width: 0.4 * window.innerWidth }}>
       <EuiPopoverTitle>
         <EuiFlexGroup alignItems="baseline" responsive={false}>
           <EuiFlexItem>Edit data filter</EuiFlexItem>

diff --git a/public/pages/CreateTransform/components/TransformIndices/TransformIndices.tsx b/public/pages/CreateTransform/components/TransformIndices/TransformIndices.tsx
@@ -31,6 +31,7 @@ import IndexFilterPopover from "../IndexFilterPopover";
 import { FieldItem, IndexItem } from "../../../../../models/interfaces";
 import IndexService from "../../../../services/IndexService";
 import { CoreServicesContext } from "../../../../components/core_services";
+import { wildcardOption } from "../../../../utils/helpers";
 
 interface TransformIndicesProps {
   indexService: IndexService;
@@ -54,7 +55,6 @@ interface TransformIndicesState {
   targetIndexOptions: { label: string; value?: IndexItem }[];
   isPopoverOpen: boolean;
   selectFieldValue: string;
-  dataFilters: string;
 }
 
 export default class TransformIndices extends Component<TransformIndicesProps, TransformIndicesState> {
@@ -67,7 +67,6 @@ export default class TransformIndices extends Component<TransformIndicesProps, T
       targetIndexOptions: [],
       isPopoverOpen: false,
       selectFieldValue: "",
-      dataFilters: "",
     };
 
     this.onIndexSearchChange = _.debounce(this.onIndexSearchChange, 500, { leading: true });
@@ -77,13 +76,15 @@ export default class TransformIndices extends Component<TransformIndicesProps, T
     await this.onIndexSearchChange("");
   }
 
+  // TODO: created shared method with rollup indices to reduce duplicate code.
   onIndexSearchChange = async (searchValue: string): Promise<void> => {
     const { indexService } = this.props;
     this.setState({ isLoading: true, indexOptions: [] });
     try {
       const dataStreamsAndIndicesNamesResponse = await indexService.getDataStreamsAndIndicesNames(searchValue);
       if (dataStreamsAndIndicesNamesResponse.ok) {
-        const options = searchValue.trim() ? [{ label: `${searchValue}*` }] : [];
+        // Adding wildcard to search value
+        const options = searchValue.trim() ? [{ label: wildcardOption(searchValue) }] : [];
         const dataStreams = dataStreamsAndIndicesNamesResponse.response.dataStreams.map((label) => ({ label }));
         const indices = dataStreamsAndIndicesNamesResponse.response.indices.map((label) => ({ label }));
         this.setState({ indexOptions: options.concat(dataStreams, indices), targetIndexOptions: indices });

diff --git a/...ansform/components/TransformOptions/Panels/DateHistogramPanel/DateHistogramPanel.test.tsx b/...ansform/components/TransformOptions/Panels/DateHistogramPanel/DateHistogramPanel.test.tsx
@@ -0,0 +1,126 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ *
+ * Modifications Copyright OpenSearch Contributors. See
+ * GitHub history for details.
+ */
+
+import React from "react";
+import { render, fireEvent } from "@testing-library/react";
+import DateHistogramPanel from "./DateHistogramPanel";
+import { IntervalType } from "../../../../../../utils/constants";
+import { GROUP_TYPES, TRANSFORM_AGG_TYPE } from "../../../../../../../models/interfaces";
+
+describe("<DateHistogramPanel /> spec", () => {
+  it("renders the component with fixed interval", async () => {
+    const { container } = render(
+      <DateHistogramPanel
+        name="test"
+        handleGroupSelectionChange={() => {}}
+        aggList={[]}
+        closePopover={() => {}}
+        intervalType={IntervalType.FIXED}
+      />
+    );
+
+    expect(container.firstChild).toMatchSnapshot();
+  });
+  it("renders the component with calendar interval", async () => {
+    const { container } = render(
+      <DateHistogramPanel
+        name="test"
+        handleGroupSelectionChange={() => {}}
+        aggList={[]}
+        closePopover={() => {}}
+        intervalType={IntervalType.CALENDAR}
+      />
+    );
+
+    expect(container.firstChild).toMatchSnapshot();
+  });
+
+  it("calls closePopover when clicking cancel button", async () => {
+    const closePopover = jest.fn();
+    const { getByTestId } = render(
+      <DateHistogramPanel
+        name="test"
+        handleGroupSelectionChange={() => {}}
+        aggList={[]}
+        closePopover={closePopover}
+        intervalType={IntervalType.FIXED}
+      />
+    );
+
+    fireEvent.click(getByTestId("dateHistogramPanelCancelButton"));
+
+    expect(closePopover).toHaveBeenCalledTimes(1);
+  });
+
+  it("uses interval 1 when defined by calendar interval", async () => {
+    const handleGroupSelectionChange = jest.fn();
+    const name = "test";
+    const intervalType = IntervalType.CALENDAR;
+    const targetFieldName = `${name} _${GROUP_TYPES.dateHistogram}_1_y_${IntervalType.CALENDAR}`;
+    const expectedGroupItem = {
+      date_histogram: {
+        source_field: name,
+        target_field: targetFieldName,
+        calendar_interval: "1y",
+      },
+    };
+    const { queryByText, getByTestId } = render(
+      <DateHistogramPanel
+        name={name}
+        handleGroupSelectionChange={handleGroupSelectionChange}
+        aggList={[]}
+        closePopover={() => {}}
+        intervalType={intervalType}
+      />
+    );
+
+    // Shows the "Every 1" text
+    expect(queryByText("Every 1")).not.toBeNull();
+
+    fireEvent.change(getByTestId("dateHistogramTimeunitSelect"), { target: { value: "y" } });
+
+    fireEvent.click(getByTestId("dateHistogramPanelOKButton"));
+
+    expect(handleGroupSelectionChange).toHaveBeenCalledWith(expectedGroupItem, TRANSFORM_AGG_TYPE.date_histogram, targetFieldName);
+  });
+
+  it("uses correct interval when defined by fixed interval", async () => {
+    const handleGroupSelectionChange = jest.fn();
+    const name = "test";
+    const intervalType = IntervalType.FIXED;
+    const intervalValue = 234;
+    const targetFieldName = `${name} _${GROUP_TYPES.dateHistogram}_234_s_${IntervalType.FIXED}`;
+    const expectedGroupItem = {
+      date_histogram: {
+        source_field: name,
+        target_field: targetFieldName,
+        fixed_interval: "234s",
+      },
+    };
+    const { getByTestId } = render(
+      <DateHistogramPanel
+        name={name}
+        handleGroupSelectionChange={handleGroupSelectionChange}
+        aggList={[]}
+        closePopover={() => {}}
+        intervalType={intervalType}
+      />
+    );
+
+    fireEvent.change(getByTestId("dateHistogramValueInput"), { target: { value: intervalValue } });
+
+    fireEvent.change(getByTestId("dateHistogramTimeunitSelect"), { target: { value: "s" } });
+
+    fireEvent.click(getByTestId("dateHistogramPanelOKButton"));
+
+    expect(handleGroupSelectionChange).toHaveBeenCalledWith(expectedGroupItem, TRANSFORM_AGG_TYPE.date_histogram, targetFieldName);
+  });
+});