Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improved URL Parsing #111

Merged
merged 20 commits into from
Apr 13, 2020
Merged

Improved URL Parsing #111

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
3b1a547
Escape regex characters
brendan-matroid Mar 23, 2020
466b777
Convert indentation to tabs
brendan-matroid Mar 23, 2020
8cee023
Fix build
brendan-matroid Mar 23, 2020
f94d54d
Parse lines using shellquote
brendan-matroid Mar 24, 2020
2d70bcc
spaces -> tabs
brendan-matroid Mar 24, 2020
991887e
Merge pull request #1 from matroid/parse-regex-chars
brendan-matroid Mar 24, 2020
313326c
Vendor dependencies
brendan-matroid Mar 24, 2020
19750da
Use QuoteMeta
brendan-matroid Mar 24, 2020
b0887b6
Testcase for regex chars, special chars, spaces
brendan-matroid Mar 24, 2020
2125df2
Merge pull request #2 from matroid/regex-chars-tidyup
brendan-matroid Mar 24, 2020
78de762
Don't overwrite o.Prefix
brendan-matroid Apr 12, 2020
9fe3e41
Add empty line 0, use backticks
brendan-matroid Apr 12, 2020
510f154
Don't Sprintf string if it has no arguments to allow for strings cont…
brendan-matroid Apr 12, 2020
2a1f07c
Merge pull request #3 from matroid/fix-tests
brendan-matroid Apr 12, 2020
863881c
Merge github.com:peak/s5cmd
brendan-matroid Apr 12, 2020
22dd92e
Make testcase consistent with other cases
brendan-matroid Apr 12, 2020
9291a9a
Fix filename
brendan-matroid Apr 12, 2020
0821d92
Fix expected output
brendan-matroid Apr 12, 2020
61d38e8
De-duplicate strings, use string formatting
brendan-matroid Apr 13, 2020
96df1e0
Move string declaration to top of function
brendan-matroid Apr 13, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion command/cmd_run.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import (
"strings"

"github.com/urfave/cli/v2"
"github.com/kballard/go-shellquote"

"github.com/peak/s5cmd/parallel"
)
Expand Down Expand Up @@ -68,7 +69,11 @@ var RunCommand = &cli.Command{
continue
}

fields := strings.Fields(line)
fields, err := shellquote.Split(line)
if err != nil {
return err
}

if len(fields) == 0 {
continue
}
Expand Down
1 change: 1 addition & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ require (
github.com/iancoleman/strcase v0.0.0-20191112232945-16388991a334
github.com/johannesboyne/gofakes3 v0.0.0-20191228161223-9aee1c78a252
github.com/karrick/godirwalk v1.15.3
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51
github.com/kr/pretty v0.2.0 // indirect
github.com/posener/complete v1.2.3
github.com/termie/go-shutil v0.0.0-20140729215957-bcacb06fecae
Expand Down
2 changes: 2 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af h1:pmfjZENx5i
github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
github.com/karrick/godirwalk v1.15.3 h1:0a2pXOgtB16CqIqXTiT7+K9L73f74n/aNQUnH6Ortew=
github.com/karrick/godirwalk v1.15.3/go.mod h1:j4mkqPuvaLI8mp1DroR3P6ad7cyYd4c1qeJ3RV7ULlk=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
github.com/kr/pretty v0.2.0 h1:s5hAObm+yFO5uHYt5dYjxi2rXrsnmRpJx4OYvIWUaQs=
github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
Expand Down
6 changes: 6 additions & 0 deletions objurl/objurl.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ const (

// matchAllRe is the regex to match everything
matchAllRe string = ".*"

// regexCharacters need to be escaped to be interpreted literally
regexCharacters string = "\\^$.|?*+()[{"
)

type objurlType int
Expand Down Expand Up @@ -209,6 +212,9 @@ func (o *ObjectURL) setPrefixAndFilter() error {
o.Prefix = o.Path[:loc]
o.filter = o.Path[loc:]
}
for _, char := range regexCharacters {
o.Prefix = strings.Replace(o.Prefix, string(char), "\\" + string(char), -1)
igungor marked this conversation as resolved.
Show resolved Hide resolved
}

filterRegex := matchAllRe
if o.filter != "" {
Expand Down