Add created, updated, and provisioned timestamps to saved template

[dbwiddis]

Description

Adds created time, last updated time, and last provisioned time to the template stored in the global context.

• Updates the last updated time when updating a workflow
• Updates the last provisioned time when provisoning a workflow

Also adds BWC tests!

Also fixes Integ tests which were essentially running twice since yamlRestTest was not properly filtered and was running them a second time.

Issues Resolved

Fixes #548

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

Attention: Patch coverage is 77.50000% with 27 lines in your changes are missing coverage. Please review.

Project coverage is 72.81%. Comparing base (e9686f7) to head (3f9a899).

❗ Current head 3f9a899 differs from pull request most recent head c600d9e. Consider uploading reports for the commit c600d9e to get more accurate results

Files	Patch %	Lines
...ework/transport/CreateWorkflowTransportAction.java	72.34%	12 Missing and 1 partial ⚠️
...rk/transport/ProvisionWorkflowTransportAction.java	53.33%	7 Missing ⚠️
...a/org/opensearch/flowframework/model/Template.java	88.67%	3 Missing and 3 partials ⚠️
...framework/indices/FlowFrameworkIndicesHandler.java	66.66%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main     #551      +/-   ##
============================================
- Coverage     72.98%   72.81%   -0.18%     
- Complexity      662      675      +13     
============================================
  Files            79       79              
  Lines          3395     3451      +56     
  Branches        264      273       +9     
============================================
+ Hits           2478     2513      +35     
- Misses          803      822      +19     
- Partials        114      116       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[joshpalis]

Lgtm, pending fix for failing integration tests. Just a small comment

[dbwiddis]

pending fix for failing integration tests.

Yeah, I chose to update the state index before updating the template in the global context which triggers this. I can easily reverse the order (it made more sense that way) but I wanted to flip the flag to provisioning as soon as possible to avoid a race condition! Will see if there's another workaround.

[dbwiddis]

POST localhost:9200/_plugins/_flow_framework/workflow/
{
  "name": "test",
  "description": "Flow template",
  "use_case": "TESTING",
  "version": {
    "template": "1.0.0",
    "compatibility": ["2.12.0", "3.0.0"]
  },
  "workflows": {
    "provision": {
      "user_params": {},
      "nodes": [
        {
          "id": "test-step",
          "type": "noop"
        }
      ]
    }
  }
}

{
    "workflow_id": "v5icD44BN3l14GXsicR6"
}

GET localhost:9200/_plugins/_flow_framework/workflow/v5icD44BN3l14GXsicR6

{
    "name": "test",
    "description": "Flow template",
    "use_case": "TESTING",
    "version": {
        "template": "1.0.0",
        "compatibility": [
            "2.12.0",
            "3.0.0"
        ]
    },
    "workflows": {
        "provision": {
            "user_params": {},
            "nodes": [
                {
                    "id": "test-step",
                    "type": "noop",
                    "previous_node_inputs": {},
                    "user_inputs": {}
                }
            ],
            "edges": []
        }
    },
    "created_time": 1709658900798,
    "last_updated_time": 1709658900798
}

PUT localhost:9200/_plugins/_flow_framework/workflow/v5icD44BN3l14GXsicR6
{
  "name": "test-update",
  "description": "Flow template",
  "use_case": "TESTING",
  "version": {
    "template": "1.0.0",
    "compatibility": ["2.12.0", "3.0.0"]
  },
  "workflows": {
    "provision": {
      "user_params": {},
      "nodes": [
        {
          "id": "test-step",
          "type": "noop"
        }
      ]
    }
  }
}

{
    "workflow_id": "v5icD44BN3l14GXsicR6"
}

GET localhost:9200/_plugins/_flow_framework/workflow/v5icD44BN3l14GXsicR6

{
    "name": "test-update",
    "description": "Flow template",
    "use_case": "TESTING",
    "version": {
        "template": "1.0.0",
        "compatibility": [
            "2.12.0",
            "3.0.0"
        ]
    },
    "workflows": {
        "provision": {
            "user_params": {},
            "nodes": [
                {
                    "id": "test-step",
                    "type": "noop",
                    "previous_node_inputs": {},
                    "user_inputs": {}
                }
            ],
            "edges": []
        }
    },
    "created_time": 1709658900798,
    "last_updated_time": 1709658977120
}

POST localhost:9200/_plugins/_flow_framework/workflow/v5icD44BN3l14GXsicR6/_provision

{
    "workflow_id": "v5icD44BN3l14GXsicR6"
}

GET localhost:9200/_plugins/_flow_framework/workflow/v5icD44BN3l14GXsicR6

{
    "name": "test-update",
    "description": "Flow template",
    "use_case": "TESTING",
    "version": {
        "template": "1.0.0",
        "compatibility": [
            "2.12.0",
            "3.0.0"
        ]
    },
    "workflows": {
        "provision": {
            "user_params": {},
            "nodes": [
                {
                    "id": "test-step",
                    "type": "noop",
                    "previous_node_inputs": {},
                    "user_inputs": {}
                }
            ],
            "edges": []
        }
    },
    "created_time": 1709658900798,
    "last_updated_time": 1709658977120,
    "last_provisioned_time": 1709659053752
}

[amitgalitz]

Start and end time should already be in the state index

[dbwiddis]

Start and end time should already be in the state index

Yes, they are.

This should match (within milliseconds) the start time of the latest provisioning attempt. Probably didn't need to add it here, but now that it's here should we leave it or take it back out?

[owaiskazi19]

Test classes yet to be reviewed. Added few comments

[owaiskazi19]

Overall looks good. Few comments

[owaiskazi19]

We can improve on the manual copying of distribution to zip in the future

[owaiskazi19]

LGTM! Thanks for addressing the comments.