Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support uploading multiple files in a single request #354

Closed
Syfaro opened this issue Jul 21, 2020 · 3 comments · Fixed by #356
Closed

Support uploading multiple files in a single request #354

Syfaro opened this issue Jul 21, 2020 · 3 comments · Fixed by #356
Labels
enhancement New feature or request help wanted Extra attention is needed
Milestone

Comments

@Syfaro
Copy link
Member

Syfaro commented Jul 21, 2020

Certain methods (sendMediaGroup, editMessageMedia, anything that supports thumbnails) allow uploading multiple files. Currently, UploadFile is structured in such a way it can only accept a single file to upload:

telegram-bot-api/bot.go

Lines 151 to 159 in da98517

// UploadFile makes a request to the API with a file.
//
// Requires the parameter to hold the file not be in the params.
// File should be a string to a file path, a FileBytes struct,
// a FileReader struct, or a url.URL.
//
// Note that if your FileReader has a size set to -1, it will read
// the file into memory to calculate a size.
func (bot *BotAPI) UploadFile(endpoint string, params Params, fieldname string, file interface{}) (APIResponse, error) {

Therefore, there needs to be a change to support uploading multiple files. One blocking issue is that the underlying library used to send files without reading everything into memory, multipartstreamer, only accepts one file per request. A replacement would have to be created.

I would love to get feedback on the best way to implement this.

@Syfaro Syfaro added enhancement New feature or request help wanted Extra attention is needed labels Jul 21, 2020
@Syfaro Syfaro added this to the 5.0 milestone Jul 21, 2020
@HirbodBehnam
Copy link

HirbodBehnam commented Jul 25, 2020

Hello!
Before reading my comment keep in mind that I'm really noob.
I suggest using io.pipe with mime/multipart and io.copy to upload your files with minimal ram usage. (The buffer size of io.Copy is 32KB)
Here is a small code sample I write for multi-file uploads with comments on most of the lines to explain what is going on (as you can see you can even use bytes.NewReader to send data from memory):
https://gist.github.com/HirbodBehnam/d0344f2377cbcd96bc058f31360a7658
I hope it helps

Edit: If you think this is ok, tell me and I try to update the api with this approach and open a pull request.

@Syfaro Syfaro linked a pull request Jul 26, 2020 that will close this issue
@Syfaro
Copy link
Member Author

Syfaro commented Jul 26, 2020

Hi @HirbodBehnam, thank you for the code sample! That seems like a great strategy for doing multiple uploads.

However, because it involved a rather large internal change to get it into a good condition, I implemented it myself. I left a comment crediting you for your contribution though, I hope that's okay.

telegram-bot-api/bot.go

Lines 155 to 221 in ce4fc98

// This code modified from the very helpful @HirbodBehnam
// https://github.com/go-telegram-bot-api/telegram-bot-api/issues/354#issuecomment-663856473
go func() {
defer w.Close()
defer m.Close()
for field, value := range params {
if err := m.WriteField(field, value); err != nil {
panic(err)
}
}
for _, file := range files {
switch f := file.File.(type) {
case string:
fileHandle, err := os.Open(f)
if err != nil {
panic(err)
}
defer fileHandle.Close()
part, err := m.CreateFormFile(file.Name, fileHandle.Name())
if err != nil {
panic(err)
}
io.Copy(part, fileHandle)
case FileBytes:
part, err := m.CreateFormFile(file.Name, f.Name)
if err != nil {
panic(err)
}
buf := bytes.NewBuffer(f.Bytes)
io.Copy(part, buf)
case FileReader:
part, err := m.CreateFormFile(file.Name, f.Name)
if err != nil {
panic(err)
}
if f.Size != -1 {
io.Copy(part, f.Reader)
} else {
data, err := ioutil.ReadAll(f.Reader)
if err != nil {
panic(err)
}
buf := bytes.NewBuffer(data)
io.Copy(part, buf)
}
case FileURL:
val := string(f)
if err := m.WriteField(file.Name, val); err != nil {
panic(err)
}
case FileID:
val := string(f)
if err := m.WriteField(file.Name, val); err != nil {
panic(err)
}
default:
panic(errors.New(ErrBadFileType))
}
}
}()

@HirbodBehnam
Copy link

Hello again
That's 100% fine with me!
I'm glad I helped

@Syfaro Syfaro closed this as completed Mar 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants