Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(core): Add a new parameter worker_event_max_payload to kong.conf #11214

Merged
merged 27 commits into from
Jul 21, 2023

Conversation

liverpool8056
Copy link
Contributor

@liverpool8056 liverpool8056 commented Jul 12, 2023

Summary

There is a limit for the size of payload allowed to be send in events lib. In some cases, Kong needs to send big payload. For example DAO:crud rely on events to invalidate cache. In these cases, the size of the payload may exceeds the limit due a large dao entity, this can prevents logics relying on the events from executing properly.

In this PR, a new parameter, worker_events_max_payload is added, which allows to specify the payload size the worker_events lib can accept. The default size is 64k, and the max allowed to set is 16M Bytes.

The corresponding PR for worker_events lib is #37

Checklist

Full changelog

  • [Implement ...]

Issue reference

FTI-4963

@chronolaw
Copy link
Contributor

chronolaw commented Jul 12, 2023

We should bump lua-resty-events to 0.2.0, see: #11203

kong/global.lua Outdated Show resolved Hide resolved
kong.conf.default Outdated Show resolved Hide resolved
kong.conf.default Outdated Show resolved Hide resolved
@pull-request-size pull-request-size bot added size/L and removed size/S labels Jul 13, 2023
@liverpool8056 liverpool8056 marked this pull request as ready for review July 13, 2023 06:01
@liverpool8056 liverpool8056 force-pushed the feat/worker_events_payload_size branch from c94ec33 to fee4454 Compare July 13, 2023 06:02
CHANGELOG.md Outdated Show resolved Hide resolved
kong/templates/kong_defaults.lua Outdated Show resolved Hide resolved
spec/02-integration/07-sdk/06-worker_events_spec.lua Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
@liverpool8056 liverpool8056 force-pushed the feat/worker_events_payload_size branch from eae4bb5 to 5d74f45 Compare July 13, 2023 07:08
CHANGELOG.md Outdated Show resolved Hide resolved
@liverpool8056 liverpool8056 force-pushed the feat/worker_events_payload_size branch from 46699af to c3ea5e0 Compare July 13, 2023 07:20
@chronolaw chronolaw changed the title Add a new parameter, worker_event_max_payload, to kong.conf feat(core): Add a new parameter worker_event_max_payload to kong.conf Jul 13, 2023
@chronolaw chronolaw added this to the 3.4.0 milestone Jul 13, 2023
@liverpool8056 liverpool8056 force-pushed the feat/worker_events_payload_size branch from 68da7d2 to e41f286 Compare July 13, 2023 08:39
@liverpool8056 liverpool8056 force-pushed the feat/worker_events_payload_size branch from 1a268df to 2312715 Compare July 13, 2023 09:49
@chronolaw chronolaw force-pushed the feat/worker_events_payload_size branch from d14925c to aa6f79d Compare July 17, 2023 07:37
@chronolaw chronolaw self-assigned this Jul 17, 2023
@kikito kikito removed this from the 3.4.0 milestone Jul 18, 2023
@kikito
Copy link
Member

kikito commented Jul 18, 2023

This was not merged in time for Feature freeze so I have removed it from the milestone. Please get special approval from a director-level person in order to merge this feature before 3.4 GA.

@chronolaw chronolaw changed the title feat(core): Add a new parameter worker_event_max_payload to kong.conf fix(core): Add a new parameter worker_event_max_payload to kong.conf Jul 21, 2023
@ADD-SP ADD-SP added this to the 3.4.0 milestone Jul 21, 2023
@guanlan guanlan merged commit ff59edb into master Jul 21, 2023
@guanlan guanlan deleted the feat/worker_events_payload_size branch July 21, 2023 02:29
team-gateway-bot pushed a commit that referenced this pull request Jul 21, 2023
#11214)

* With a hard-coded payload size, for some use cases like uploading a big
OpenAPI spec in DevPortal or updating a big config entry for plugins,
they can not work as expected. With the new parameter, the user can
decide the payload size to meet their needs.

In this PR, a new parameter, `worker_events_max_payload` is added, which
allows to specify the payload size the `worker_events` lib can accept.
The default size is 64k, and the max allowed to set is 16M Bytes.

The corresponding PR for `worker_events` lib is [#37](Kong/lua-resty-events#37)

FTI-4963

* add changelog entry

* Update kong.conf.default

Co-authored-by: Datong Sun <datong.sun@konghq.com>

* add test case and bump lua-resty-events

* correct the default value, and add an entry for bumping the version of lua-resty-events

* 1. append PR number to the changelog entry of lua-resty-events
2. correct the spec test
3. style

* Update CHANGELOG.md

---------

Co-authored-by: Datong Sun <datong.sun@konghq.com>
Co-authored-by: Chrono <chrono_cpp@me.com>
(cherry picked from commit ff59edb)
chronolaw added a commit that referenced this pull request Jul 21, 2023
#11214)

* With a hard-coded payload size, for some use cases like uploading a big
OpenAPI spec in DevPortal or updating a big config entry for plugins,
they can not work as expected. With the new parameter, the user can
decide the payload size to meet their needs.

In this PR, a new parameter, `worker_events_max_payload` is added, which
allows to specify the payload size the `worker_events` lib can accept.
The default size is 64k, and the max allowed to set is 16M Bytes.

The corresponding PR for `worker_events` lib is [#37](Kong/lua-resty-events#37)

FTI-4963

* add changelog entry

* Update kong.conf.default

Co-authored-by: Datong Sun <datong.sun@konghq.com>

* add test case and bump lua-resty-events

* correct the default value, and add an entry for bumping the version of lua-resty-events

* 1. append PR number to the changelog entry of lua-resty-events
2. correct the spec test
3. style

* Update CHANGELOG.md

---------

Co-authored-by: Datong Sun <datong.sun@konghq.com>
Co-authored-by: Chrono <chrono_cpp@me.com>
windmgc pushed a commit that referenced this pull request Jul 21, 2023
#11214)

* With a hard-coded payload size, for some use cases like uploading a big
OpenAPI spec in DevPortal or updating a big config entry for plugins,
they can not work as expected. With the new parameter, the user can
decide the payload size to meet their needs.

In this PR, a new parameter, `worker_events_max_payload` is added, which
allows to specify the payload size the `worker_events` lib can accept.
The default size is 64k, and the max allowed to set is 16M Bytes.

The corresponding PR for `worker_events` lib is [#37](Kong/lua-resty-events#37)

FTI-4963

* add changelog entry

* Update kong.conf.default

Co-authored-by: Datong Sun <datong.sun@konghq.com>

* add test case and bump lua-resty-events

* correct the default value, and add an entry for bumping the version of lua-resty-events

* 1. append PR number to the changelog entry of lua-resty-events
2. correct the spec test
3. style

* Update CHANGELOG.md

---------

Co-authored-by: Datong Sun <datong.sun@konghq.com>
Co-authored-by: Chrono <chrono_cpp@me.com>
(cherry picked from commit ff59edb)
@kikito kikito mentioned this pull request Aug 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants