Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: check if the remote exists just before reading it #1713

Merged
merged 4 commits into from
Sep 7, 2024
Merged

refactor: check if the remote exists just before reading it #1713

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4d48b5d
refactor: check if the remote exists in the read to avoid doing it in…
vmaerten Jul 7, 2024
3074a9f
fix: timeout error was not working
vmaerten Jul 7, 2024
db2654e
fix: use cached copy if available
vmaerten Jul 7, 2024
ccadcad
Merge branch 'main' into refactor/remote-check-remote-later
vmaerten Aug 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 14 additions & 10 deletions taskfile/node_http.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@ import (
// An HTTPNode is a node that reads a Taskfile from a remote location via HTTP.
type HTTPNode struct {
*BaseNode
URL *url.URL
URL *url.URL
logger *logger.Logger
timeout time.Duration
}

func NewHTTPNode(
Expand All @@ -36,18 +38,12 @@ func NewHTTPNode(
if url.Scheme == "http" && !insecure {
return nil, &errors.TaskfileNotSecureError{URI: entrypoint}
}
ctx, cf := context.WithTimeout(context.Background(), timeout)
defer cf()
url, err = RemoteExists(ctx, l, url)
if err != nil {
return nil, err
}
if errors.Is(ctx.Err(), context.DeadlineExceeded) {
return nil, &errors.TaskfileNetworkTimeoutError{URI: url.String(), Timeout: timeout}
}

return &HTTPNode{
BaseNode: base,
URL: url,
timeout: timeout,
logger: l,
}, nil
}

Expand All @@ -60,13 +56,21 @@ func (node *HTTPNode) Remote() bool {
}

func (node *HTTPNode) Read(ctx context.Context) ([]byte, error) {
url, err := RemoteExists(ctx, node.logger, node.URL, node.timeout)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You removed the context.WithTimeout

You could put it back here before this

Because the timeout should be used with the head and the get

Suggested change
url, err := RemoteExists(ctx, node.logger, node.URL, node.timeout)
ctx, cancel := context.WithTimeout(ctx, node.timeout)
defer cancel()
url, err := RemoteExists(ctx, node.logger, node.URL)

You are not using the timeout right now in remote except for the error

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The timeout is set in the reader :

ctx, cf := context.WithTimeout(context.Background(), r.timeout)
defer cf()

// Read the file
b, err = node.Read(ctx)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then you should catch the CTX error in the caller ? This way, you don't introduce the timeout parameter, that is only used for reporting errors,no?

Maybe I missed something again 😅😁

if err != nil {
return nil, err
}
node.URL = url
req, err := http.NewRequest("GET", node.URL.String(), nil)
if err != nil {
return nil, errors.TaskfileFetchFailedError{URI: node.URL.String()}
}

resp, err := http.DefaultClient.Do(req.WithContext(ctx))
if err != nil {
if errors.Is(err, context.DeadlineExceeded) {
return nil, &errors.TaskfileNetworkTimeoutError{URI: node.URL.String(), Timeout: node.timeout}
}
return nil, errors.TaskfileFetchFailedError{URI: node.URL.String()}
}
defer resp.Body.Close()
Expand Down
3 changes: 2 additions & 1 deletion taskfile/reader.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -207,8 +207,9 @@ func (r *Reader) readNode(node Node) (*ast.Taskfile, error) {

// Read the file
b, err = node.Read(ctx)
var taskfileNetworkTimeoutError *errors.TaskfileNetworkTimeoutError
// If we timed out then we likely have a network issue
if node.Remote() && errors.Is(ctx.Err(), context.DeadlineExceeded) {
if node.Remote() && errors.As(err, &taskfileNetworkTimeoutError) {
// If a download was requested, then we can't use a cached copy
if r.download {
return nil, &errors.TaskfileNetworkTimeoutError{URI: node.Location(), Timeout: r.timeout}
Expand Down
6 changes: 5 additions & 1 deletion taskfile/taskfile.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"path/filepath"
"slices"
"strings"
"time"

"github.com/go-task/task/v3/errors"
"github.com/go-task/task/v3/internal/filepathext"
Expand Down Expand Up @@ -40,7 +41,7 @@ var (
// at the given URL with any of the default Taskfile files names. If any of
// these match a file, the first matching path will be returned. If no files are
// found, an error will be returned.
func RemoteExists(ctx context.Context, l *logger.Logger, u *url.URL) (*url.URL, error) {
func RemoteExists(ctx context.Context, l *logger.Logger, u *url.URL, timeout time.Duration) (*url.URL, error) {
// Create a new HEAD request for the given URL to check if the resource exists
req, err := http.NewRequest("HEAD", u.String(), nil)
if err != nil {
Expand All @@ -50,6 +51,9 @@ func RemoteExists(ctx context.Context, l *logger.Logger, u *url.URL) (*url.URL,
// Request the given URL
resp, err := http.DefaultClient.Do(req.WithContext(ctx))
if err != nil {
if errors.Is(ctx.Err(), context.DeadlineExceeded) {
return nil, &errors.TaskfileNetworkTimeoutError{URI: u.String(), Timeout: timeout}
}
return nil, errors.TaskfileFetchFailedError{URI: u.String()}
}
defer resp.Body.Close()
Expand Down
Loading