feature: 文件分发 & 存储支持制品库 #93

ZhuoZhuoCrayon · 2021-09-09T08:11:03Z

No description provided.

codecov-commenter · 2021-09-09T08:15:39Z

Codecov Report

Merging #93 (8fa56f5) into V2.1.348rc (5def023) will increase coverage by 0.49%.
The diff coverage is 81.31%.

@@              Coverage Diff               @@
##           V2.1.348rc      #93      +/-   ##
==============================================
+ Coverage       58.89%   59.38%   +0.49%     
==============================================
  Files             396      410      +14     
  Lines           25970    26510     +540     
==============================================
+ Hits            15295    15743     +448     
- Misses          10675    10767      +92

Impacted Files	Coverage Δ
apps/backend/agent/manager.py	`17.13% <0.00%> (ø)`
apps/backend/components/collections/bulk_job.py	`65.67% <0.00%> (ø)`
...s/backend/components/collections/bulk_job_redis.py	`75.82% <0.00%> (+0.71%)`	⬆️
apps/backend/subscription/steps/base.py	`42.16% <0.00%> (ø)`
apps/backend/subscription/steps/plugin.py	`35.30% <0.00%> (ø)`
apps/node_man/tools/plugin_v2.py	`82.89% <ø> (+0.33%)`	⬆️
apps/node_man/views/plugin_v2.py	`48.61% <0.00%> (ø)`
apps/node_man/handlers/plugin_v2.py	`58.00% <29.41%> (-7.52%)`	⬇️
apps/core/files/file_source.py	`40.54% <40.54%> (ø)`
apps/node_man/periodic_tasks/update_proxy_file.py	`25.39% <50.00%> (ø)`
... and 52 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5def023...8fa56f5. Read the comment docs.

apps/backend/api/job.py

apps/backend/components/collections/agent.py

zhangzhw8 · 2021-09-10T06:34:54Z

apps/backend/management/commands/copy_file_to_nginx.py

@@ -29,7 +29,7 @@ def handle(self, *args, **options):
        # 接入点配置的nginx路径
        nginx_paths = [ap.nginx_path for ap in AccessPoint.objects.all() if ap.nginx_path]
        # 默认nginx路径
-        nginx_paths.append(settings.NGINX_DOWNLOAD_PATH)
+        nginx_paths.append(settings.DOWNLOAD_PATH)


43行这里没改造，还是用的本地文件copy，应该调整为storage.save

zhangzhw8 · 2021-09-10T06:48:33Z

apps/backend/plugin/serializers.py

@@ -287,6 +310,15 @@ def validate(self, data):
    creator = serializers.CharField()
    bk_app_code = serializers.CharField()

+    def validate(self, attrs):
+        if attrs["category"] not in DownloadRecord.CATEGORY_TASK_DICT:


这个为什么不直接在serializer字段上加choices

zhangzhw8 · 2021-09-10T07:00:44Z

apps/backend/plugin/views.py

+            )
+            pkg_parse_result.update(
+                {
+                    "pkg_abs_path": package_info["pkg_relative_path"],


abs_path，relative_path 迷惑性

zhangzhw8 · 2021-09-10T07:58:57Z

apps/backend/plugin/views.py

+        try:
+            if files.md5sum(file_obj=storage.open(name=file_path)) != md5:
+                raise ValidationError(_("上传文件MD5校验失败，请确认重试"))
+        except Exception as e:


这里的异常捕获会把上面的 MD5校验异常捕获，抛出文件不存在的异常

zhangzhw8 · 2021-09-10T08:29:22Z

apps/backend/tests/plugin/views/test_plugin_production.py

+        upload_result = self.client.post(
+            path="/backend/package/upload/",
+            data={
+                **self.API_AUTH_PARAMS,


**self.API_AUTH_PARAMS每个请求都需要的话，可以考虑在 PluginBaseTestCase 里重写self.client，请求时在参数里加上 API_AUTH_PARAMS

或者在更上层的BaseTestCase基类中去实现

apps.utils.unittest.base.CustomAPIClient 加了 common_request_data ，上层 TestCase 在 setUp 等钩子通过self.client.common_request_data 设置通用请求

zhangzhw8 · 2021-09-10T09:41:56Z

apps/core/files/storage.py

+    ) -> List[Dict[str, Any]]:
+
+        # 获取或创建文件源
+        file_source_obj = BkJobFileSourceManager.get_or_create_file_source(


每次handle的时候都去检查或创建，会带来额外的性能消耗？

每次handle的时候都去检查或创建，会带来额外的性能消耗？
已加缓存

ZhuoZhuoCrayon self-assigned this Sep 9, 2021

zhangzhw8 reviewed Sep 10, 2021

View reviewed changes

ZhuoZhuoCrayon force-pushed the V2.1.346rc branch from 0217976 to cc9d915 Compare September 15, 2021 07:35

ZhuoZhuoCrayon changed the base branch from V2.1.346rc to V2.1.347rc September 15, 2021 09:16

ZhuoZhuoCrayon force-pushed the _V2.1.X/issue#2 branch 2 times, most recently from 60caa11 to 44e2e8b Compare September 16, 2021 09:24

zhangzhw8 changed the base branch from V2.1.347rc to V2.1.348rc September 16, 2021 09:42

ZhuoZhuoCrayon added 4 commits September 16, 2021 22:08

feature: 插件包管理支持对象存储模式，公共方法优化整合 (#2)

af1d0bf

feature: 作业平台文件分发支持制品库(#2)

31fe4b7

test: 测试-插件包管理支持对象存储模式 (#2)

356813f

optimization: 代码优化(#2)

8fa56f5

ZhuoZhuoCrayon force-pushed the _V2.1.X/issue#2 branch from 44e2e8b to 8fa56f5 Compare September 16, 2021 14:09

ZhuoZhuoCrayon added kind/feature 功能 module/backend 后台 module/build 构建 module/saas SaaS version/V2.1.X_dev V2.1.X 开发分支 labels Sep 18, 2021

zhangzhw8 merged commit 4b91618 into V2.1.348rc Sep 22, 2021

ZhuoZhuoCrayon deleted the _V2.1.X/issue#2 branch March 8, 2022 03:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feature: 文件分发 & 存储支持制品库 #93

feature: 文件分发 & 存储支持制品库 #93

ZhuoZhuoCrayon commented Sep 9, 2021

codecov-commenter commented Sep 9, 2021 •

edited

Loading

zhangzhw8 Sep 10, 2021

zhangzhw8 Sep 10, 2021

zhangzhw8 Sep 10, 2021

zhangzhw8 Sep 10, 2021

zhangzhw8 Sep 10, 2021

zhangzhw8 Sep 10, 2021

ZhuoZhuoCrayon Sep 16, 2021

zhangzhw8 Sep 10, 2021

ZhuoZhuoCrayon Sep 16, 2021

feature: 文件分发 & 存储 支持制品库 #93

feature: 文件分发 & 存储 支持制品库 #93

Conversation

ZhuoZhuoCrayon commented Sep 9, 2021

codecov-commenter commented Sep 9, 2021 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

feature: 文件分发 & 存储支持制品库 #93

feature: 文件分发 & 存储支持制品库 #93

codecov-commenter commented Sep 9, 2021 •

edited

Loading