-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #57 from CloudOS-Group3/dev
Dev
- Loading branch information
Showing
171 changed files
with
12,779 additions
and
52 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| name: test all cases | ||
|
|
||
| on: | ||
| push: | ||
| branches: ["master", "dev", "feature/*"] | ||
|
|
||
| pull_request: | ||
| branches: ["master", "dev", "feature/*"] | ||
|
|
||
| jobs: | ||
| build: | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v2 | ||
|
|
||
| - name: Set up Go | ||
| uses: actions/setup-go@v2 | ||
| with: | ||
| go-version: 1.22.2 | ||
|
|
||
| - name: Download etcd | ||
| run: | | ||
| wget https://github.com/etcd-io/etcd/releases/download/v3.5.13/etcd-v3.5.13-linux-amd64.tar.gz | ||
| tar -zxvf etcd-v3.5.13-linux-amd64.tar.gz | ||
| cd etcd-v3.5.13-linux-amd64 | ||
| ./etcd &> /dev/null & | ||
| - name: Download nerdctl | ||
| run: | | ||
| mkdir -p nerdctl | ||
| cd nerdctl | ||
| wget https://github.com/containerd/nerdctl/releases/download/v1.0.0/nerdctl-1.0.0-linux-amd64.tar.gz | ||
| tar -xf nerdctl-1.0.0-linux-amd64.tar.gz | ||
| cp nerdctl /usr/local/bin/ | ||
| - name: Download Kafka | ||
| run: | | ||
| sudo apt-get update | ||
| sudo apt-get install openjdk-8-jdk | ||
| wget https://archive.apache.org/dist/kafka/3.6.0/kafka_2.13-3.6.0.tgz | ||
| tar -zxvf kafka_2.13-3.6.0.tgz | ||
| cd kafka_2.13-3.6.0 | ||
| KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)" | ||
| bin/kafka-storage.sh format -t $KAFKA_CLUSTER_ID -c config/kraft/server.properties | ||
| bin/kafka-server-start.sh config/kraft/server.properties &> /dev/null & | ||
| - name: Download etcdctl | ||
| run: | | ||
| wget https://github.com/etcd-io/etcd/releases/download/v3.4.14/etcd-v3.4.14-linux-amd64.tar.gz | ||
| tar -zxf etcd-v3.4.14-linux-amd64.tar.gz | ||
| mv etcd-v3.4.14-linux-amd64/etcdctl /usr/local/bin | ||
| sudo chmod +x /usr/local/bin/ | ||
| etcdctl version | ||
| - name: Flannel Config | ||
| run: | | ||
| export ETCDCTL_API=2 | ||
| etcdctl --endpoints http://127.0.0.1:2379 set /coreos.com/network/config '{"Network": "10.0.0.0/16", "SubnetLen": 24, "SubnetMin": "10.0.1.0","SubnetMax": "10.0.20.0", "Backend": {"Type": "vxlan"}}' | ||
| nerdctl run -d --privileged -v /run/flannel:/run/flannel --net host --name flannel registry.cn-hangzhou.aliyuncs.com/acs/flannel:v0.15.1.22-20a397e6-aliyun \ | ||
| -etcd-prefix /coreos.com/network -iface eth0 -etcd-endpoints http://192.168.0.180:2379 -ip-masq | ||
| nerdctl network ls | ||
| - name: Download Nginx | ||
| run: | | ||
| sudo apt install -y curl gnupg2 ca-certificates lsb-release | ||
| sudo apt install -y nginx | ||
| sudo systemctl start nginx | ||
| - name: Run tests | ||
| run: sudo go test ./... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5,4 +5,6 @@ | |
| .idea | ||
|
|
||
| # windows executable files | ||
| **/*.exe | ||
| **/*.exe | ||
|
|
||
| build | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -37,3 +37,5 @@ cd cmd | |
| go run kubectl.go | ||
| ``` | ||
|
|
||
| kubectl apply -f \<filename\> | ||
| 已经可以使用，请正确指定yaml文件的路径 | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| package main | ||
|
|
||
| import ( | ||
| "minik8s/pkg/apiserver/server" | ||
| "minik8s/pkg/config" | ||
| ) | ||
|
|
||
| func main() { | ||
|
|
||
| host, port := config.GetHostAndPort() | ||
| server := server.NewAPIserver(host, port) | ||
|
|
||
| server.Run() | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| package main | ||
|
|
||
| import "minik8s/pkg/controller/controllermanager" | ||
|
|
||
| func main() { | ||
| Controllers := controllermanager.NewControllerManager() | ||
| Controllers.Run(make(chan bool)) | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| package main | ||
|
|
||
| import ( | ||
| "testing" | ||
| ) | ||
|
|
||
| func TestMain(m *testing.M) { | ||
|
|
||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| package main | ||
|
|
||
| import ( | ||
| "minik8s/pkg/kubelet/subscriber" | ||
| "os" | ||
| ) | ||
|
|
||
| func main() { | ||
| if len(os.Args) < 3 { | ||
| server := subscriber.NewKubeletSubscriber("") | ||
| server.Run() | ||
| } else { | ||
| if os.Args[1] == "--name" { | ||
| server := subscriber.NewKubeletSubscriber(os.Args[2]) | ||
| server.Run() | ||
| } | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| package main | ||
|
|
||
| import ( | ||
| "minik8s/pkg/kubeproxy" | ||
| "os" | ||
| ) | ||
|
|
||
| func main() { | ||
| if len(os.Args) < 3 { | ||
| proxy := kubeproxy.NewKubeProxy("") | ||
| proxy.Run() | ||
| } else { | ||
| if os.Args[1] == "--name" { | ||
| proxy := kubeproxy.NewKubeProxy(os.Args[2]) | ||
| proxy.Run() | ||
| } | ||
| } | ||
|
|
||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| package main | ||
|
|
||
| import "minik8s/pkg/scheduler" | ||
|
|
||
| func main() { | ||
| server := scheduler.NewScheduler() | ||
| server.Run() | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,82 @@ | ||
| # Severless实现 | ||
|
|
||
| ### 1. Function抽象 | ||
|
|
||
| **使用方式**： | ||
|
|
||
| - 用户可以通过单个python文件定义function，python文件名无要求，但是主函数名必须为`main`； | ||
| - 用户需要指定参数和返回值的数量、类型，也可指定名字（用于workflow之间传递指定参数）； | ||
| - 用户可以指定函数invoke方式：http或event trigger； | ||
| - 用户需在对应文件夹下提供requirement.txt | ||
|
|
||
| yaml文件示例如下： | ||
|
|
||
| ```yaml | ||
| apiVersion: v1 | ||
| kind: Function | ||
| metadata: | ||
| name: BuyTrainTicket | ||
| language: python | ||
| filePath: /root/minik8s/testdata/workflow1/BuyTrainTicket/ | ||
| triggerType: | ||
| http: true | ||
| params: | ||
| - name: x | ||
| type: int | ||
| result: | ||
| - name: x | ||
| type: int | ||
| - name: greeting | ||
| type: string | ||
| ``` | ||
| 实现方式： | ||
| #### 1.1 创建function | ||
| 通过 `apply -f <filepath>` 指令，通过yaml文件创建function： | ||
|
|
||
| 1. 存入etcd； | ||
|
|
||
| 2. build函数镜像 | ||
|
|
||
| 1. 将所有需要的文件复制到工作路径，安装运行环境 | ||
| 2. 在容器中写入一个server.py，在容器运行时监听8080端口，并且执行用户函数，将结果发送至集群的apiserver | ||
|
|
||
| 3. 将镜像push到minik8s所维护的镜像仓库 | ||
|
|
||
| > 该镜像仓库是运行在master IP 下的 5050 端口上的一个 Docker registry | ||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
| 1. ⽀持Function抽象。⽤⼾可以通过单个⽂件（zip包或代码⽂件）定义函数内容，通过指令上传给 | ||
| minik8s，并且通过http trigger和event trigger调⽤函数。 | ||
| ◦ 函数需要⾄少⽀持Python语⾔ | ||
| ◦ 函数的格式，return的格式，update、invoke指令的格式可以⾃定义 | ||
| ◦ 函数调⽤的⽅式：⽀持⽤⼾通过http请求、和绑定事件触发两种⽅式调⽤函数。函数可以通过 | ||
| 指令指定要绑定的事件源，事件的类型可以是时间计划、⽂件修改或其他⾃定义内容 | ||
| 2. ⽀持Serverless Workflow抽象：⽤⼾可以定义Serverless DAG，包括以下⼏个组成成分： | ||
| ◦ 函数调⽤链：在调⽤函数时传参给第⼀个函数，之后依次调⽤各个函数，前⼀个函数的输出作 | ||
| 为后⼀个函数的输⼊，最终输出结果。 | ||
| ◦ 分⽀：根据上⼀个函数的输出，控制⾯决定接下来运⾏哪⼀个分⽀的函数。 | ||
| ◦ Serverless Workflow可以通过配置⽂件来定义，参考AWS StepFunction或Knative的做法。除 | ||
| 此之外，同学们也可以⾃⾏定义编程模型来构建Serverless Workflow，只要workflow能达到 | ||
| 上述要求即可。 | ||
| 3. Serverless的⾃动扩容（Scale-to-0） | ||
| ◦ Serverless的实例应当在函数请求⾸次到来时被创建（冷启动），并且在⻓时间没有函数请求再 | ||
| 次到来时被删除（scale-to-0）。同时，Serverless能够监控请求数变化，当请求数量增多时能 | ||
| 够⾃动扩容⾄>1实例。 | ||
| ◦ Serverless应当能够正确处理⼤量的并发请求(数⼗并发)，演⽰时使⽤wrk2或者Jmeter进⾏压 | ||
| ⼒测试。 | ||
| 4. 找⼀个较为复杂的开源的Serverless应⽤或者⾃⾏实现⼀个较为复杂的Serverless应⽤，该应⽤必 | ||
| 须有现实的应⽤场景(⽐如下图所⽰的 Image Processing)，不能只是简单的加减乘除或者hello | ||
| world。将该应⽤部署在minik8s上，基于这个应⽤来展⽰Serverless相关功能，并在验收报告中结 | ||
| 合该应⽤的需求和特点详细分析该应⽤使⽤Serverless架构的必要性和优势。 Serverless样例可参考Serverlessbench中的Alexa：ServerlessBench/Testcase4-Applicationbreakdown/alexa at master · SJTU-IPADS/ServerlessBench |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.