Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build(snap): upgrade kuiper to 1.4.2 #3867

Merged

Conversation

MonicaisHer
Copy link
Contributor

@MonicaisHer MonicaisHer commented Jan 24, 2022

This PR is to upgrade kuiper to 1.4.2

  • add connectionSelector to re-use EdgeX message bus
  • remove unused configure and install scripts

If your build fails due to your commit message not passing the build checks, please review the guidelines here: https://github.com/edgexfoundry/edgex-go/blob/main/.github/Contributing.md

PR Checklist

Please check if your PR fulfills the following requirements:

  • I am not introducing a breaking change (if you are, flag in conventional commit message with BREAKING CHANGE: describing the break)
  • I am not introducing a new dependency (add notes below if you are)
  • I have added unit tests for the new feature or bug fix (if not, why?) Add connection test for kuiper canonical/edgex-checkbox-provider#22
  • I have fully tested (add details below) this the new feature or bug fix (if not, why?)
  • I have opened a PR for the related docs change (if not, why?)

Testing Instructions

  1. Install
snapcraft
snap install edgexfoundry_2.0.1-dev.71_amd64.snap --dangerous
snap set edgexfoundry kuiper=on
snap set edgexfoundry device-virtual=on
  1. Create stream and sink with connectionSelector
curl -X POST \
  http://localhost:59720/streams \
  -d '{
  "sql": "create stream edgexAll() WITH (FORMAT=\"JSON\", TYPE=\"edgex\", SHARED=\"true\")"
}'

curl -X POST \
  http://localhost:59720/rules \
  -d '{
  "id": "rule-demo",
   "sql": "SELECT * from edgexAll  WHERE meta(deviceName)=\"Random-Integer-Device\"",
   "actions": [
      {
         "edgex": {
            "connectionSelector": "edgex.redisMsgBus",
            "topicPrefix": "edgex/events/device", 
            "messageType": "request",
            "contentType": "application/json",
            "deviceName": "device-demo"
         }
      }
   ]
}'
  1. Validate rule is working
curl -X GET http://localhost:59720/rules/rule-demo/status

Expect:

{
  "source_edgexAll_0_records_in_total": 137,
  "source_edgexAll_0_records_out_total": 137,
  "source_edgexAll_0_exceptions_total": 0,
  "source_edgexAll_0_process_latency_us": 1,
  "source_edgexAll_0_buffer_length": 0,
  "source_edgexAll_0_last_invocation": "2022-01-24T09:17:19.956633",
  "op_2_filter_0_records_in_total": 137,
  "op_2_filter_0_records_out_total": 40,
  "op_2_filter_0_exceptions_total": 0,
  "op_2_filter_0_process_latency_us": 5,
  "op_2_filter_0_buffer_length": 0,
  "op_2_filter_0_last_invocation": "2022-01-24T09:17:19.956639",
  "op_3_project_0_records_in_total": 40,
  "op_3_project_0_records_out_total": 40,
  "op_3_project_0_exceptions_total": 0,
  "op_3_project_0_process_latency_us": 5,
  "op_3_project_0_buffer_length": 0,
  "op_3_project_0_last_invocation": "2022-01-24T09:17:14.953311",
  "sink_edgex_0_0_records_in_total": 40,
  "sink_edgex_0_0_records_out_total": 40, <---------------------------------------sink is working
  "sink_edgex_0_0_exceptions_total": 0,
  "sink_edgex_0_0_process_latency_us": 39,
  "sink_edgex_0_0_buffer_length": 0,
  "sink_edgex_0_0_last_invocation": "2022-01-24T09:17:14.953321"
}

  1. Validate rule is re-using EdgeX message bus
curl -X GET http://localhost:59880/api/v2/reading/device/name/device-demo?limit=3 | jq '.'

Expect:

{
  "apiVersion": "v2",
  "statusCode": 200,
  "totalCount": 16,
  "readings": [
    {
      "id": "54efdfa4-ac66-436d-b66a-438ed8f1b976",
      "origin": 1643012144946162000,
      "deviceName": "device-demo",
      "resourceName": "Int16",
      "profileName": "ekuiperProfile",
      "valueType": "Int64",
      "value": "11357"
    },
    {
      "id": "90e965ac-a73a-4668-b868-64b4e27030f2",
      "origin": 1643012144945702100,
      "deviceName": "device-demo",
      "resourceName": "Int8",
      "profileName": "ekuiperProfile",
      "valueType": "Int64",
      "value": "98"
    },
    {
      "id": "6dc38635-3b07-40c1-bcea-4faa80b0a284",
      "origin": 1643012144944587000,
      "deviceName": "device-demo",
      "resourceName": "Int64",
      "profileName": "ekuiperProfile",
      "valueType": "Int64",
      "value": "4729033642373210121"
    }
  ]
}

New Dependency Instructions (If applicable)

- add `connections` and `portable` directory
- specify stream to reuse the connection to EdgeX message bus
- remove the override of edgex- sink's port, since the default is 6379 now, which will not collide with app-service-configurable's port

Signed-off-by: Mengyi Wang <mengyi.wang@canonical.com>
@MonicaisHer MonicaisHer marked this pull request as ready for review January 24, 2022 13:08
@MonicaisHer MonicaisHer requested a review from farshidtz January 24, 2022 13:09
snap/snapcraft.yaml Outdated Show resolved Hide resolved
farshidtz
farshidtz previously approved these changes Jan 25, 2022
Copy link
Member

@farshidtz farshidtz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @MonicaisHer

LGTM. I suggest we wait until we complete the relevant tests before merging.

Also, please include the reasoning behind removal of the serviceServer property in the squash commit message.

- add `connectionSelector` in `eKuiperEdgeXSourceTemplate`
- remove `serviceServer`, since EdgeX v2 does not reply on 
  Core contract Service anymore (reference: https://github.com
  /lf-edge/ekuiper/blob/master/docs/en_US/edgex/
  edgex_rule_engine_tutorial.md#migrate-to-edgex-v2)

Signed-off-by: Mengyi Wang <mengyi.wang@canonical.com>
Signed-off-by: Mengyi Wang <mengyi.wang@canonical.com>
Signed-off-by: Mengyi Wang <mengyi.wang@canonical.com>
Signed-off-by: Mengyi Wang <mengyi.wang@canonical.com>
@MonicaisHer MonicaisHer force-pushed the EDGEX-172-update-kuiper-to-1-4-x branch from c83b8d7 to e1f6ea4 Compare January 25, 2022 12:59
@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@MonicaisHer
Copy link
Contributor Author

@farshidtz Thanks a lot for your review. I have added the reasoning behind the removal of the serviceServer property in the corresponding commit message.

@MonicaisHer MonicaisHer requested a review from farshidtz January 28, 2022 13:32
@farshidtz
Copy link
Member

For reference, setting SHARED="true" in rules is currently necessary to avoid a number of issues in Kuiper that look like race conditions (e.g. incomplete JSON payload, slice bounds out of range). We are investigating further and would create an issue once we have more information.

@farshidtz
Copy link
Member

Issue reported on eKuiper project: lf-edge/ekuiper#1168

Copy link
Member

@farshidtz farshidtz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's merge this now and make it available on the beta track, while waiting for lf-edge/ekuiper#1168 to be fixed.

It should be functional as long as the rules set SHARED to true.

@farshidtz farshidtz merged commit 85bf44e into edgexfoundry:main Feb 3, 2022
@farshidtz farshidtz added the jakarta dot - 2.1.1 Jakarta Dot Release 2.1.1 label Jun 3, 2022
farshidtz pushed a commit to farshidtz/edgex-go that referenced this pull request Jun 3, 2022
- remove `serviceServer`, since EdgeX v2 does not rely on Core contract Service anymore (see: https://github.com/lf-edge/ekuiper/blob/master/docs/en_US/edgex/edgex_rule_engine_tutorial.md#migrate-to-edgex-v2)
- remove the override of edgex- sink's port, since the default is 6379 now, which will not collide with app-service-configurable's port
- add `connections` and `portable` directory
- specify stream to reuse the connection to EdgeX message bus
- add `connectionSelector` in `eKuiperEdgeXSourceTemplate`
- remove unused kuiper configuration scripts
- remove unused kuiper install scripts `rm *.zip`
- remove unused comment

(cherry picked from commit 85bf44e)
farshidtz pushed a commit to farshidtz/edgex-go that referenced this pull request Jun 3, 2022
- remove `serviceServer`, since EdgeX v2 does not rely on Core contract Service anymore (see: https://github.com/lf-edge/ekuiper/blob/master/docs/en_US/edgex/edgex_rule_engine_tutorial.md#migrate-to-edgex-v2)
- remove the override of edgex- sink's port, since the default is 6379 now, which will not collide with app-service-configurable's port
- add `connections` and `portable` directory
- specify stream to reuse the connection to EdgeX message bus
- add `connectionSelector` in `eKuiperEdgeXSourceTemplate`
- remove unused kuiper configuration scripts
- remove unused kuiper install scripts `rm *.zip`
- remove unused comment

(cherry picked from commit 85bf44e)
Signed-off-by: Farshid Tavakolizadeh <farshid.tavakolizadeh@canonical.com>
farshidtz added a commit that referenced this pull request Jun 3, 2022
* build(snap): upgrade kuiper to 1.4.2 (#3867)

- remove `serviceServer`, since EdgeX v2 does not rely on Core contract Service anymore (see: https://github.com/lf-edge/ekuiper/blob/master/docs/en_US/edgex/edgex_rule_engine_tutorial.md#migrate-to-edgex-v2)
- remove the override of edgex- sink's port, since the default is 6379 now, which will not collide with app-service-configurable's port
- add `connections` and `portable` directory
- specify stream to reuse the connection to EdgeX message bus
- add `connectionSelector` in `eKuiperEdgeXSourceTemplate`
- remove unused kuiper configuration scripts
- remove unused kuiper install scripts `rm *.zip`
- remove unused comment

(cherry picked from commit 85bf44e)
Signed-off-by: Farshid Tavakolizadeh <farshid.tavakolizadeh@canonical.com>

* build(snap): upgrade kuiper to 1.4.4

Signed-off-by: Farshid Tavakolizadeh <farshid.tavakolizadeh@canonical.com>

* fix(snap): revert to remove kuiper zip package

Signed-off-by: Farshid Tavakolizadeh <farshid.tavakolizadeh@canonical.com>

Co-authored-by: Mengyi <mengyi.wang@canonical.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
jakarta dot - 2.1.1 Jakarta Dot Release 2.1.1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants