Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support read data from mysql when upload data #574

Open
wants to merge 628 commits into
base: main
Choose a base branch
from

Conversation

FancyXun
Copy link

@FancyXun FancyXun commented Jul 19, 2024

Follow this issue: FederatedAI/FATE#5669

Support uploading data from mysql:

Change the 'file' in upload_guest_mysql.json like bellow:

{
"file":"mysql://root:root123456@192.168.40.22:13312/fl/BREAST_HETERO_GUEST",
"head": true,
"partitions": 16,
"extend_sid": true,
"meta": {
"delimiter": ",",
"label_name": "y",
"match_id_name": "id"
},
"namespace": "experiment",
"name": "breast_hetero_guest_mysql"
}

Upload data
flow data upload -c upload_guest_mysql.json

The three formats of the file variable in this JSON are as follows:

mysql://user:password@host_ip:host_port/db/table
file:///path/to/local_file.csv
/path/to/local_file.csv

zhihuiwan and others added 30 commits December 7, 2023 21:08
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: chengtcc <864261919@qq.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
zhihuiwan and others added 12 commits February 29, 2024 16:57
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: zhihuiwan <15779896112@163.com>
Signed-off-by: robbie228 <v_lmli@webank.com>
Signed-off-by: robbie228 <v_lmli@webank.com>
Signed-off-by: robbie228 <v_lmli@webank.com>
Signed-off-by: robbie228 <v_lmli@webank.com>
Signed-off-by: robbie228 <v_lmli@webank.com>
@FancyXun
Copy link
Author

@sagewe
@dylan-fan

@sagewe
Copy link
Contributor

sagewe commented Jul 19, 2024

首先很感谢你的贡献,社区对DCO有强制要求,请依照 Details 进行修复

有空看看 @zhihuiwan

@sagewe sagewe requested a review from zhihuiwan July 19, 2024 02:48
FancyXun added 2 commits July 19, 2024 11:33
Signed-off-by: fancyxun <837633751@qq.com>
Signed-off-by: fancyxun <837633751@qq.com>
@FancyXun
Copy link
Author

首先很感谢你的贡献,社区对DCO有强制要求,请依照 Details 进行修复

有空看看 @zhihuiwan

Done!

@FancyXun
Copy link
Author

补充一下:这样做的好处就是,能兼容现有的upload.json,只需要根据里面file的前缀判断是用读取mysql或者本地文件,json不需要新增内容。 @sagewe

mgqa34 and others added 8 commits July 29, 2024 14:07
Signed-off-by: mgqa34 <mgq3374541@163.com>
Signed-off-by: mgqa34 <mgq3374541@163.com>
Signed-off-by: mgqa34 <mgq3374541@163.com>
Merge v2.2.0's Updates Into Main Branch
Signed-off-by: FancyXun <837633751@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants