feat(drivers): add MediaFire driver support

[ILoveScratch2]

Description / 描述

添加MediaFire驱动
cherry-pick from alist#9319 alist#9321

原作者D@' 3z K!7 经过@ILoveScratch2 修改

Motivation and Context / 背景

添加Mediafire驱动，并修改以适配OpenList

How Has This Been Tested? / 测试

测试，可成功挂在Mediafire

Checklist / 检查清单

• I have read the CONTRIBUTING document.
  我已阅读 CONTRIBUTING 文档。
• I have formatted my code with go fmt or prettier.
  我已使用 go fmt 或 prettier 格式化提交的代码。
• I have added appropriate labels to this PR (or mentioned needed labels in the description if lacking permissions).
  我已为此 PR 添加了适当的标签（如无权限或需要的标签不存在，请在描述中说明，管理员将后续处理）。
• I have requested review from relevant code authors using the "Request review" feature when applicable.
  我已在适当情况下使用"Request review"功能请求相关代码作者进行审查。
• I have updated the repository accordingly (If it’s needed).
  我已相应更新了相关仓库（若适用）。
  • OpenList-Frontend #XXXX
  • OpenList-Docs #194

[KirCute]

感觉很多问题都是因为原作者不熟悉代码，改一下d.cron和秒传的问题就行，别的不太重要

[ILoveScratch2]

感觉很多问题都是因为原作者不熟悉代码，改一下d.cron和秒传的问题就行，别的不太重要

（其实也基本都是原作者写的，就改了改适配）

[j2rong4cn]

上传还可以进一步优化，重试、多线程，，具体参考其他使用stream.NewStreamSectionReader和errgroup.NewOrderedGroupWithContext的代码

[Da3zKi7]

@ILoveScratch2 Hello, what is pending?

@j2rong4cn upload was already optimized by small chunks and resumable logic, all handled by original code and MediaFire's API logic.

[Da3zKi7]

@j2rong4cn I'll check stream.NewStreamSectionReader and errgroup.NewOrderedGroupWithContext I never committed here.

[KirCute]

@j2rong4cn I'll check stream.NewStreamSectionReader and errgroup.NewOrderedGroupWithContext I never committed here.

Using file.GetHash().GetHash(utils.SHA256) may allow you to obtain the SHA256 of the file. If the file already has the required hash value, we recommend avoiding fully caching the file stream locally. Instead, as the frontend provides data segment by segment, OpenList should transfer it piece by piece to the driver. Please keep this in mind during your check. Thank you for your contributions to AList and OpenList.

[Da3zKi7]

@j2rong4cn I'll check stream.NewStreamSectionReader and errgroup.NewOrderedGroupWithContext I never committed here.

Using file.GetHash().GetHash(utils.SHA256) may allow you to obtain the SHA256 of the file. If the file already has the required hash value, we recommend avoiding fully caching the file stream locally. Instead, as the frontend provides data segment by segment, OpenList should transfer it piece by piece to the driver. Please keep this in mind during your check. Thank you for your contributions to AList and OpenList.

Got it thanks, I'm not a master in Golang)) but I try to help.

So...

If stream.NewStreamSectionReader and errgroup.NewOrderedGroupWithContext are commited the driver is ready to be merged?

[KirCute]

If stream.NewStreamSectionReader and errgroup.NewOrderedGroupWithContext are commited the driver is ready to be merged?

Right.

[Da3zKi7]

@KirCute I will try, do not expect too much 😂😂😂

[KirCute]

@KirCute I will try, do not expect too much 😂😂😂

OpenList/drivers/123_open/upload.go

Lines 50 to 168 in d465da4

	ss, err := stream.NewStreamSectionReader(file, int(chunkSize), &up)
	if err != nil {
	return err
	}
	uploadNums := (size + chunkSize - 1) / chunkSize
	thread := min(int(uploadNums), d.UploadThread)
	threadG, uploadCtx := errgroup.NewOrderedGroupWithContext(ctx, thread,
	retry.Attempts(3),
	retry.Delay(time.Second),
	retry.DelayType(retry.BackOffDelay))
	for partIndex := range uploadNums {
	if utils.IsCanceled(uploadCtx) {
	break
	}
	partIndex := partIndex
	partNumber := partIndex + 1 // 分片号从1开始
	offset := partIndex * chunkSize
	size := min(chunkSize, size-offset)
	var reader *stream.SectionReader
	var rateLimitedRd io.Reader
	sliceMD5 := ""
	// 表单
	b := bytes.NewBuffer(make([]byte, 0, 2048))
	threadG.GoWithLifecycle(errgroup.Lifecycle{
	Before: func(ctx context.Context) error {
	if reader == nil {
	var err error
	// 每个分片一个reader
	reader, err = ss.GetSectionReader(offset, size)
	if err != nil {
	return err
	}
	// 计算当前分片的MD5
	sliceMD5, err = utils.HashReader(utils.MD5, reader)
	if err != nil {
	return err
	}
	}
	return nil
	},
	Do: func(ctx context.Context) error {
	// 重置分片reader位置，因为HashReader、上一次失败已经读取到分片EOF
	reader.Seek(0, io.SeekStart)
	b.Reset()
	w := multipart.NewWriter(b)
	// 添加表单字段
	err = w.WriteField("preuploadID", createResp.Data.PreuploadID)
	if err != nil {
	return err
	}
	err = w.WriteField("sliceNo", strconv.FormatInt(partNumber, 10))
	if err != nil {
	return err
	}
	err = w.WriteField("sliceMD5", sliceMD5)
	if err != nil {
	return err
	}
	// 写入文件内容
	_, err = w.CreateFormFile("slice", fmt.Sprintf("%s.part%d", file.GetName(), partNumber))
	if err != nil {
	return err
	}
	headSize := b.Len()
	err = w.Close()
	if err != nil {
	return err
	}
	head := bytes.NewReader(b.Bytes()[:headSize])
	tail := bytes.NewReader(b.Bytes()[headSize:])
	rateLimitedRd = driver.NewLimitedUploadStream(ctx, io.MultiReader(head, reader, tail))
	// 创建请求并设置header
	req, err := http.NewRequestWithContext(ctx, http.MethodPost, uploadDomain+"/upload/v2/file/slice", rateLimitedRd)
	if err != nil {
	return err
	}
	// 设置请求头
	req.Header.Add("Authorization", "Bearer "+d.AccessToken)
	req.Header.Add("Content-Type", w.FormDataContentType())
	req.Header.Add("Platform", "open_platform")
	res, err := base.HttpClient.Do(req)
	if err != nil {
	return err
	}
	defer res.Body.Close()
	if res.StatusCode != 200 {
	return fmt.Errorf("slice %d upload failed, status code: %d", partNumber, res.StatusCode)
	}
	var resp BaseResp
	respBody, err := io.ReadAll(res.Body)
	if err != nil {
	return err
	}
	err = json.Unmarshal(respBody, &resp)
	if err != nil {
	return err
	}
	if resp.Code != 0 {
	return fmt.Errorf("slice %d upload failed: %s", partNumber, resp.Message)
	}
	progress := 10.0 + 85.0*float64(threadG.Success())/float64(uploadNums)
	up(progress)
	return nil
	},
	After: func(err error) {
	ss.FreeSectionReader(reader)
	},
	})
	}
	if err := threadG.Wait(); err != nil {
	return err
	}

You can refer to the upload logic of the 123open driver. Don't be stressful about the code, if there are any issues we will fix it then.

[Da3zKi7]

@KirCute Thanks mate, I was working on it, I hope I caught you

[KirCute]

我发现所有api调用最后都会汇总到d.apiRequest，在apiRequest里已经调用d.limiter.Wait了，我就把最外层的WaitLimit删了

[KirCute]

多线程上传这块我现在也不知道是啥情况，别的没问题了

[j2rong4cn]

多线程上传这块我现在也不知道是啥情况，别的没问题了

什么症状

[KirCute]

什么症状

没什么症状，只是我不知道你们聊到什么程度了，如果现在这版你们测过没问题那就合吧