feat: add openfunction plugin #7634

jackkkkklee · 2022-08-09T09:07:16Z

Description

support openfunction serverless platform
implement proposal #7404

Checklist

I have explained the need for this PR and the problem it solves
I have explained the changes or the new features added to this PR
I have added tests corresponding to this change
I have updated the documentation to reflect this change
I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

tokers · 2022-08-09T10:11:45Z

apisix/plugins/openfunction.lua

+end
+
+
+function _M.access(conf, ctx)


Can we reuse code base in https://github.com/apache/apisix/blob/master/apisix/plugins/serverless/init.lua?

Maybe you mean reuse https://github.com/apache/apisix/blob/master/apisix/plugins/serverless/generic-upstream.lua ?

And code in init.lua, you can refer to azure-function for the detail.

tokers · 2022-08-09T10:14:24Z

ci/init-plugin-test-service.sh

+}
+
+set_container_registry_secret() {
+    REGISTRY_SERVER=https://index.docker.io/v1/ REGISTRY_USER=apisixtestaccount123 REGISTRY_PASSWORD=apisixtestaccount


Why use this? We can push the docker images through kind load docker-image.

The part of the openfuction building lifecycle, pushing the image to a registry, needs it.Will it be better to build locally to avoid this part?

You should avoid exposing your secrets here. By the way, is this for building openfunction itself or the function?

Sure，for the function.
I 'll avoid it.

Is there a lighter way (using less runtime memory) to run these services? I'm worried that running these services will again fill up the memory of the server where github action is running CI.

My plan is to just run the function locally in docker with the image I created by openfunction platform previously，which doesn't need to install many services. I'll update it after test.

docs/en/latest/plugins/openfunction.md

tokers · 2022-08-09T10:15:26Z

docs/en/latest/plugins/openfunction.md

+
+## Enabling the Plugin
+
+Before configuring the Plugin, you need to have OpenFunction running. The example below shows OpenFunction installed in Helm:


Also give a link to the installation page of OpenFunction.

docs/en/latest/plugins/openfunction.md

apisix/plugins/openfunction.lua

Co-authored-by: Alex Zhang <tokers@apache.org>

tokers · 2022-08-11T09:03:22Z

@jackkkkklee Please make the CI pass. Thanks!

docs/en/latest/plugins/openfunction.md

spacewander · 2022-08-12T06:27:38Z

apisix/plugins/openfunction.lua

+local function request_processor(conf, ctx, params)
+    local headers = params.headers or {}
+    -- setting authorization headers if not already set
+    if not headers["Authorization"] and conf.authorization


Does the header here need to be lower-case? And we need a test case that the user-specific Authorization header has higher priority.

Thanks for the reminding.
Lower-case for headers is needed if you mean the local param 'headers', or you mean the header of docs above?

Test case added.

The local param headers.

Co-authored-by: 罗泽轩 <spacewanderlzx@gmail.com>

tokers · 2022-08-14T11:40:09Z

apisix/plugins/openfunction.lua

+local function request_processor(conf, ctx, params)
+    local headers = params.headers or {}
+    -- setting authorization headers if not already set
+    if not headers["Authorization"] and conf.authorization


The local param headers.

tokers · 2022-08-14T11:40:48Z

ci/init-plugin-test-service.sh

@@ -41,3 +41,13 @@ docker exec -i rmqnamesrv /home/rocketmq/rocketmq-4.6.0/bin/mqadmin updateTopic

 # prepare vault kv engine
 docker exec -i vault sh -c "VAULT_TOKEN='root' VAULT_ADDR='http://0.0.0.0:8200' vault secrets enable -path=kv -version=1 kv"
+
+# prepare openfunction env
+docker pull apisixtestaccount123/sample-go-func:v1


I'm wondering if OpenFunction releases some sample functions 🤔.

They do， but the functions just print the URI of request, which doesn't meet my test cases(e.g. test body, test headers). Therefore, I just fork and change their functions a little bit.

can we expose the Dockerfile for apisixtestaccount123/sample-go-func:v1?
My concern is that when we want to make updates based on this image later, then it will be difficult for us to do so.

I ‘ll add local function building process latter，then we just need to update the code of repositories.

Co-authored-by: Fei Han <97138894+hf400159@users.noreply.github.com>

jackkkkklee · 2022-08-26T12:10:51Z

@tokers Could you please rerun ”CI / build (ubuntu-20.04, linux_openresty, t/node t/pubsub t/router t/script t/stream-node t/utils t/w... (pull_request) Failing after 17m...“?

tzssangglass · 2022-08-28T17:59:17Z

Could you please rerun ”CI / build (ubuntu-20.04, linux_openresty, t/node t/pubsub t/router t/script t/stream-node t/utils t/w... (pull_request) Failing after 17m...“?

done

spacewander · 2022-08-31T06:06:59Z

ci/init-plugin-test-service.sh

@@ -41,3 +41,4 @@ docker exec -i rmqnamesrv /home/rocketmq/rocketmq-4.6.0/bin/mqadmin updateTopic

 # prepare vault kv engine
 docker exec -i vault sh -c "VAULT_TOKEN='root' VAULT_ADDR='http://0.0.0.0:8200' vault secrets enable -path=kv -version=1 kv"
+


We don't need to touch this file?

No,we don't . I just run the function image in the dockerfile . And I ’ll discard the change in this file .

docs/en/latest/plugins/openfunction.md

Co-authored-by: 罗泽轩 <spacewanderlzx@gmail.com>

� Conflicts: � ci/pod/docker-compose.plugin.yml � docs/en/latest/config.json � docs/zh/latest/config.json

* upstream/master: (214 commits) feat: set constants.apisix_lua_home for used by plugins (apache#7893) fix: response-rewrite plugin might cause Apache AIPSIX hanging (apache#7836) test: sleep 1 second in t/cli/test_upstream_mtls.sh (apache#7889) fix: reload once when log rotate (apache#7869) change: move etcd conf under deployment (apache#7860) fix: plugin metadata missing v3 adapter call (apache#7877) docs: add ClickHouse and Elasticsearch loggers. (apache#7848) docs(plugin): refactor limit-conn.md (apache#7857) feat: call `destroy` method when Nginx exits (apache#7866) feat: Add ability to inject headers via prefix to otel traces (apache#7822) docs(plugin): refactor proxy-cache.md (apache#7858) fix: don't enable passive healthcheck by default (apache#7850) feat: add openfunction plugin (apache#7634) fix(zipkin): send trace IDs with a reject sampling decision (apache#7833) fix: Change opentelemetry's span kind to server (apache#7830) docs(hmac-auth): additional details for generating signing_string (apache#7816) docs: correct the test-nginx description (apache#7818) docs: add docs of workflow plugin (apache#7813) docs(FAQ): add how to detect APISIX alive status (apache#7812) feat: add elasticsearch-logger (apache#7643) ...

Co-authored-by: Alex Zhang <tokers@apache.org> Co-authored-by: 罗泽轩 <spacewanderlzx@gmail.com> Co-authored-by: Fei Han <97138894+hf400159@users.noreply.github.com> Co-authored-by: LCW <Lcw769193516@163.com>

LCW added 9 commits August 9, 2022 11:35

feat: add openfunction plugin

05dc0a6

feat: add openfunction plugin

fc8a099

fix: change test uri

2b46f00

CI: prepare openfunction env

0c3887a

docs: add openfunction plugins

3d549ec

fix: increase the timeout time

e13f870

fix: correct htpasswd command

f59a15f

ci: fix openfunction ci params

69ee651

ci: fix openfunction ci params

283c5e8

jackkkkklee changed the title ~~add openfunction plugin~~ feat: add openfunction plugin Aug 9, 2022

tokers reviewed Aug 9, 2022

View reviewed changes

tzssangglass reviewed Aug 10, 2022

View reviewed changes

apisix/plugins/openfunction.lua Outdated Show resolved Hide resolved

jackkkkklee and others added 5 commits August 10, 2022 10:47

docs: fix name

467d6d0

Co-authored-by: Alex Zhang <tokers@apache.org>

docs: fix verification command

e88936c

Co-authored-by: Alex Zhang <tokers@apache.org>

Merge branch 'apache:master' into master

757f520

fix: fix Code Lint

98b9ba1

fix: fix Lint

23521f1

LCW added 4 commits August 11, 2022 21:29

ci: avoid function building and run function in docker

822b589

test: update openfunction test case

c31b6c8

docs: add openfunction index

4f0c086

feat: reuse serverless.generic-upstream

a779ca1

spacewander reviewed Aug 12, 2022

View reviewed changes

jackkkkklee and others added 6 commits August 12, 2022 14:56

docs: fix lint

5096a41

Co-authored-by: 罗泽轩 <spacewanderlzx@gmail.com>

docs: fix lint

9b5dc62

Co-authored-by: 罗泽轩 <spacewanderlzx@gmail.com>

docs: update timeout

2d02b02

test: add check user-specific auth header

f9efd9d

Merge branch 'master' of https://github.com/jackkkkklee/apisix

0dfc092

docs: update attributes description

59d5236

tokers reviewed Aug 14, 2022

View reviewed changes

jackkkkklee and others added 12 commits August 25, 2022 11:38

docs: modify expression

55ac693

Co-authored-by: Fei Han <97138894+hf400159@users.noreply.github.com>

docs: modify expression

a667c6e

Co-authored-by: Fei Han <97138894+hf400159@users.noreply.github.com>

docs: modify expression

c6a56d4

Co-authored-by: Fei Han <97138894+hf400159@users.noreply.github.com>

docs: modify expression

f8f07ab

Co-authored-by: Fei Han <97138894+hf400159@users.noreply.github.com>

docs: modify expression

35e32ef

Co-authored-by: Fei Han <97138894+hf400159@users.noreply.github.com>

docs: modify expression

5e6647a

Co-authored-by: Fei Han <97138894+hf400159@users.noreply.github.com>

docs: modify expression

04edf39

Co-authored-by: Fei Han <97138894+hf400159@users.noreply.github.com>

docs: modify expression

0643864

Co-authored-by: Fei Han <97138894+hf400159@users.noreply.github.com>

docs: modify expression

ba5e58d

Co-authored-by: Fei Han <97138894+hf400159@users.noreply.github.com>

docs: modify expression

107d685

Co-authored-by: Fei Han <97138894+hf400159@users.noreply.github.com>

docs: fix lint

0d3545c

docs: fix lint

eed4f5a

jackkkkklee requested a review from tokers August 30, 2022 04:51

tokers previously approved these changes Aug 30, 2022

View reviewed changes

jackkkkklee requested a review from spacewander August 30, 2022 10:30

spacewander reviewed Aug 31, 2022

View reviewed changes

docs: update latest admin port

b89805a

Co-authored-by: 罗泽轩 <spacewanderlzx@gmail.com>

jackkkkklee dismissed tokers’s stale review via b89805a August 31, 2022 06:28

LCW added 4 commits August 31, 2022 14:41

ci: discard change

558948a

docs: update admin api port

3ee2c41

ci: solve conflict

698f371

Merge branch 'master' of https://github.com/apache/apisix

bf5d4c9

� Conflicts: � ci/pod/docker-compose.plugin.yml � docs/en/latest/config.json � docs/zh/latest/config.json

spacewander approved these changes Sep 1, 2022

View reviewed changes

jackkkkklee requested a review from tokers September 1, 2022 08:04

tokers approved these changes Sep 2, 2022

View reviewed changes

spacewander merged commit 47187fa into apache:master Sep 2, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add openfunction plugin #7634

feat: add openfunction plugin #7634

jackkkkklee commented Aug 9, 2022

tokers Aug 9, 2022

jackkkkklee Aug 10, 2022

tokers Aug 10, 2022

tokers Aug 9, 2022

jackkkkklee Aug 10, 2022

tokers Aug 10, 2022

jackkkkklee Aug 10, 2022

tzssangglass Aug 10, 2022

jackkkkklee Aug 11, 2022

tokers Aug 9, 2022

tokers commented Aug 11, 2022

spacewander Aug 12, 2022

jackkkkklee Aug 12, 2022

tokers Aug 14, 2022

tokers Aug 14, 2022

tokers Aug 14, 2022

jackkkkklee Aug 14, 2022

tzssangglass Aug 15, 2022

jackkkkklee Aug 15, 2022

jackkkkklee commented Aug 26, 2022

tzssangglass commented Aug 28, 2022

spacewander Aug 31, 2022

jackkkkklee Aug 31, 2022


		## Enabling the Plugin

		Before configuring the Plugin, you need to have OpenFunction running. The example below shows OpenFunction installed in Helm:

		@@ -41,3 +41,4 @@ docker exec -i rmqnamesrv /home/rocketmq/rocketmq-4.6.0/bin/mqadmin updateTopic

		# prepare vault kv engine
		docker exec -i vault sh -c "VAULT_TOKEN='root' VAULT_ADDR='http://0.0.0.0:8200' vault secrets enable -path=kv -version=1 kv"

feat: add openfunction plugin #7634

feat: add openfunction plugin #7634

Conversation

jackkkkklee commented Aug 9, 2022

Description

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tokers commented Aug 11, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jackkkkklee commented Aug 26, 2022

tzssangglass commented Aug 28, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment