Skip to content

feat(drivers/mediafire): support concurrent upload #1387

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ILoveScratch2 merged 11 commits into from
Sep 29, 2025
Merged

feat(drivers/mediafire): support concurrent upload #1387

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
d68b1da
feat(drivers): add MediaFire driver with concurrent upload support
Da3zKi7 Sep 26, 2025
06abed9
feat(drivers): add MediaFire driver with concurrent upload support
Da3zKi7 Sep 26, 2025
7beeb51
feat(stream): add DiscardSection method to StreamSectionReader for sk…
j2rong4cn Sep 27, 2025
9b33d93
feat(mediafire): refactor resumableUpload logic for improved upload h…
j2rong4cn Sep 27, 2025
c3031e1
fix(mediafire): stop cron job and clear action token in Drop method
j2rong4cn Sep 27, 2025
94bc043
.
j2rong4cn Sep 27, 2025
0b24556
Merge branch 'mediafire' into feat/mediafire
j2rong4cn Sep 28, 2025
21c6ea9
fix(mediafire): optimize buffer sizing logic in uploadUnits method
j2rong4cn Sep 28, 2025
e756a67
fix(docs): remove duplicate MediaFire
j2rong4cn Sep 28, 2025
7081986
fix(mediafire): revert 'optimization', large files should not be full…
Da3zKi7 Sep 28, 2025
4b7f41f
Merge branch 'mediafire' into Da3zKi7-feat/mediafire
ILoveScratch2 Sep 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README_nl.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ Dank u voor uw ondersteuning en begrip
- [x] [UPYUN Storage Service](https://www.upyun.com/products/file-storage)
- [x] [WebDAV](https://en.wikipedia.org/wiki/WebDAV)
- [x] Teambition([China](https://www.teambition.com), [Internationaal](https://us.teambition.com))
- [x] [MediaFire](https://www.mediafire.com)
- [x] [Mediatrack](https://www.mediatrack.cn)
- [x] [139yun](https://yun.139.com) (Persoonlijk, Familie, Groep)
- [x] [YandexDisk](https://disk.yandex.com)
Expand Down
24 changes: 23 additions & 1 deletion drivers/mediafire/driver.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,15 @@ D@' 3z K!7 - The King Of Cracking

Modifications by ILoveScratch2<ilovescratch@foxmail.com>
Date: 2025-09-21

Date: 2025-09-26
Final opts by @Suyunjing @j2rong4cn @KirCute @Da3zKi7
*/

import (
"context"
"fmt"
"math/rand"
"net/http"
"strconv"
"time"
Expand All @@ -22,6 +26,7 @@ import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/internal/stream"
"github.com/OpenListTeam/OpenList/v4/pkg/cron"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
"golang.org/x/time/rate"
)
Expand All @@ -30,6 +35,8 @@ type Mediafire struct {
model.Storage
Addition

cron *cron.Cron

actionToken string
limiter *rate.Limiter

Expand Down Expand Up @@ -66,7 +73,18 @@ func (d *Mediafire) Init(ctx context.Context) error {
}
// Validate and refresh session token if needed
if _, err := d.getSessionToken(ctx); err != nil {
return d.renewToken(ctx)

d.renewToken(ctx)
Copy link

Copilot AI Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error returned by d.renewToken(ctx) is ignored. If token renewal fails, the Init method should return the error instead of continuing execution.

Suggested change
d.renewToken(ctx)
if err := d.renewToken(ctx); err != nil {
return fmt.Errorf("Init :: [MediaFire] failed to renew token: %w", err)
}

Copilot uses AI. Check for mistakes.

// Avoids 10 mins token expiry (6- 9)
num := rand.Intn(4) + 6

d.cron = cron.NewCron(time.Minute * time.Duration(num))
d.cron.Do(func() {
// Crazy, but working way to refresh session token
d.renewToken(ctx)
Copy link

Copilot AI Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The context ctx from Init method is captured in the cron closure, but this context may be canceled or expired when the cron job runs later. The cron job should use a background context or create a new context for token renewal.

Suggested change
d.renewToken(ctx)
d.renewToken(context.Background())

Copilot uses AI. Check for mistakes.
})

}

return nil
Expand All @@ -76,6 +94,10 @@ func (d *Mediafire) Init(ctx context.Context) error {
func (d *Mediafire) Drop(ctx context.Context) error {
// Clear cached resources
d.actionToken = ""
if d.cron != nil {
d.cron.Stop()
d.cron = nil
}
return nil
}

Expand Down
4 changes: 4 additions & 0 deletions drivers/mediafire/meta.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ D@' 3z K!7 - The King Of Cracking

Modifications by ILoveScratch2<ilovescratch@foxmail.com>
Date: 2025-09-21

Date: 2025-09-26
Final opts by @Suyunjing @j2rong4cn @KirCute @Da3zKi7
*/

import (
Expand All @@ -26,6 +29,7 @@ type Addition struct {
OrderBy string `json:"order_by" type:"select" options:"name,time,size" default:"name"`
OrderDirection string `json:"order_direction" type:"select" options:"asc,desc" default:"asc"`
ChunkSize int64 `json:"chunk_size" type:"number" default:"100"`
UploadThreads int `json:"upload_threads" type:"number" default:"3" help:"concurrent upload threads"`
LimitRate float64 `json:"limit_rate" type:"float" default:"2" help:"limit all api request rate ([limit]r/1s)"`
}

Expand Down
Loading