feature: 插件包管理支持对象存储模式 (#2) #17
Conversation
Codecov Report
@@ Coverage Diff @@
## V2.1.X_develop #17 +/- ##
===================================================
+ Coverage 59.00% 85.71% +26.71%
===================================================
Files 390 1 -389
Lines 25738 7 -25731
===================================================
- Hits 15186 6 -15180
+ Misses 10552 1 -10551 Continue to review full report at Codecov.
|
ZhuoZhuoCrayon
force-pushed
the
_V2.1.X_develop/feature_issue_2
branch
from
9c006ed to
05d39b7
Compare
apps/node_man/models.py
Outdated
| dir_path, | ||
| # 判断是否第三方插件的路径 | ||
| arcname="external_plugins/%s" % package_name if is_external else "plugins/", | ||
| arcname=f"external_plugins/{project}" if is_external else f"plugins/{project}", |
There was a problem hiding this comment.
提取常量：external_plugins plugins
| STORAGE_TYPE = os.getenv("STORAGE_TYPE", StorageType.FILE_SYSTEM.value) | ||
|
|
||
| # 是否覆盖同名文件 | ||
| FILE_OVERWRITE = get_type_env("FILE_OVERWRITE", _type=bool, default=False) |
There was a problem hiding this comment.
默认是False吗？似乎节点管理的场景应该默认是True比较合适
There was a problem hiding this comment.
特定场景下需要覆盖，比如插件包导入的成品，DOWNLOAD_PATH下，需要保证同名覆盖，逻辑已显式传入file_overwrite=True
但是在用户自主上传的情况下及导出的情况，同名文件建议是另存，覆盖会导致同名但文件内容不同的情况，无法完成问题溯源
apps/backend/plugin/tasks.py
Outdated
| is_release=is_release, | ||
| creator=task_params["bk_username"], | ||
| select_pkg_abs_paths=select_pkg_abs_paths, | ||
| select_pkg_relative_paths=select_pkg_abs_paths, |
There was a problem hiding this comment.
接口字段命名不规范，改为在序列化器兼容select_pkg_abs_paths 和 select_pkg_relative_paths ，后续前端更正
apps/utils/files.py
Outdated
| from apps.node_man import constants | ||
|
|
||
|
|
||
| class file_open: |
| @@ -25,6 +30,7 @@ | |||
| urlpatterns = [ | |||
| url(r"api/", include(routers.urls)), | |||
| url(r"^package/upload/$", upload_package), | |||
| url(r"^package/upload_cos/$", upload_package_by_cos), | |||
There was a problem hiding this comment.
这里应该 package/upload/ 里根据环境变量选择上传的方法， LocalStroage or Cos ，屏蔽掉这个逻辑而不暴露出来让调用方选择哪个API？
是上次讨论的nginx转发的问题对吧，这个再讨论下
| def __exit__(self, exc_type, exc_val, exc_tb): | ||
|
|
||
| # 指针复位，避免 closed=False 场景下，影响上层逻辑对该文件对象的复用 | ||
| # 参考：https://stackoverflow.com/questions/3906137/why-cant-i-call-read-twice-on-an-open-file |
There was a problem hiding this comment.
同向读写可以通过提供冗余逻辑解决
但写后读的情况，比如文件下载后，通过tarfile解析时，就需要指针复位，比起维护两个上下文，重置指针好像优雅些，也不占内存
apps/utils/files.py
Outdated
| hash_md5.update(chunk) | ||
|
|
||
| except IOError: | ||
| return "-1" |
There was a problem hiding this comment.
MD5计算失败是不是抛出异常更好，而不是返回 "-1" 隐藏了问题
| return hash_md5.hexdigest() | ||
|
|
||
|
|
||
| def download_file( |
There was a problem hiding this comment.
后台可以接受两种参数创建上传记录，其中一种就是download_url，主要是考虑后续监控等第三方可以在插件归档到自己仓库的同时，传download_url到节点管理，同步第三方平台的插件。
| self.file_overwrite = file_overwrite | ||
|
|
||
| # 如果文件允许覆盖，去掉 O_CREAT 配置，存在文件打开时不报错 | ||
| if self.file_overwrite: |
There was a problem hiding this comment.
如果上面的条件非真，这里就 AttributeError 了
There was a problem hiding this comment.
继承了FileOverWriteMixin，具备属性fileoverwrite
apps/node_man/models.py
Outdated
| @@ -923,57 +924,73 @@ def create_record( | |||
| :return: True | raise Exception | |||
| """ | |||
| # 1. 判断是否存在project.yaml文件 | |||
There was a problem hiding this comment.
这里的逻辑跟parse_package重复率很高，是否可以合并复用？
feature: 插件包管理支持对象存储模式，公共方法优化整合 (TencentBlueKing#2)
ZhuoZhuoCrayon
force-pushed
the
_V2.1.X_develop/feature_issue_2
branch
from
05d39b7 to
e9717f9
Compare
|
分支规范修改，转移至 #31 |
issue #2
相关工作：
改造范围：
基本思路：