Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BE-594 | Delivery package #525

Merged
merged 1 commit into from
Oct 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions delivery/grpc/grpc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
// Package grpc provides a custom gRPC client implementation for Cosmos SDK-based applications.
package grpc

import (
"fmt"

proto "github.com/cosmos/gogoproto/proto"

"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
"google.golang.org/grpc/encoding"
)

type OsmomathCodec struct {
parentCodec encoding.Codec
}
p0mvn marked this conversation as resolved.
Show resolved Hide resolved

func (c OsmomathCodec) Marshal(v interface{}) ([]byte, error) {
protoMsg, ok := v.(proto.Message)
if !ok {
return nil, fmt.Errorf("failed to assert proto.Message")
}
return proto.Marshal(protoMsg)
}
p0mvn marked this conversation as resolved.
Show resolved Hide resolved

func (c OsmomathCodec) Unmarshal(data []byte, v interface{}) error {
protoMsg, ok := v.(proto.Message)
if !ok {
return fmt.Errorf("failed to assert proto.Message")
}
return proto.Unmarshal(data, protoMsg)
}

func (c OsmomathCodec) Name() string {
return "gogoproto"
}

// Client wraps a gRPC ClientConn, providing a custom connection.
// Connection is set up with custom options, including the use of a custom codec
// for gogoproto and OpenTelemetry instrumentation.
// Client addresses marshaling math.LegacyDec issue: https://github.com/cosmos/cosmos-sdk/issues/18430
type Client struct {
*grpc.ClientConn
}

// NewClient creates a new gRPC client connection to the specified endpoint.
func NewClient(grpcEndpoint string) (*Client, error) {
customCodec := &OsmomathCodec{parentCodec: encoding.GetCodec("proto")}

grpcOpts := []grpc.DialOption{
grpc.WithTransportCredentials(insecure.NewCredentials()),
grpc.WithStatsHandler(otelgrpc.NewClientHandler()),
grpc.WithDefaultCallOptions(grpc.ForceCodec(customCodec)),
p0mvn marked this conversation as resolved.
Show resolved Hide resolved
Comment on lines +48 to +54
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused - why do we need this? Cosmos SDK should have all of the GRPC-related packages and libraries with the appropriate codec registered

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we need to do this here when in the rest of the places we were able to avoid this custom codec logic:

grpcClient, err := grpc.NewClient(grpcURI,
grpc.WithTransportCredentials(insecure.NewCredentials()),
grpc.WithStatsHandler(otelgrpc.NewClientHandler()),
)
if err != nil {
return nil, err
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a great question! Custom codec is required to fix a panic that would occur unmarshaling to osmomath.LegacyDec data types otherwise. I have also documented it in the source code here.

For example:

panic: invalid Go type math.LegacyDec for field osmosis.txfees.v1beta1.QueryEipBaseFeeResponse.base_fee

goroutine 4056 [running]:
google.golang.org/protobuf/internal/impl.newSingularConverter({0x460fc10, 0x32bfd60}, {0x46114b0, 0xc0108f5400})
        /home/deividas/go/pkg/mod/google.golang.org/protobuf@v1.34.2/internal/impl/convert.go:142 +0x9ad
google.golang.org/protobuf/internal/impl.NewConverter({0x460fc10, 0x32bfd60}, {0x46114b0, 0xc0108f5400})
        /home/deividas/go/pkg/mod/google.golang.org/protobuf@v1.34.2/internal/impl/convert.go:60 +0x8f
google.golang.org/protobuf/internal/impl.fieldInfoForScalar({0x46114b0, 0xc0108f5400}, {{0x2ec85e8, 0x7}, {0x0, 0x0}, {0x460fc10, 0x32bfd60}, {0x2ec85f0, 0x7f}, ...}, ...)
        /home/deividas/go/pkg/mod/google.golang.org/protobuf@v1.34.2/internal/impl/message_reflect_field.go:270 +0x193
google.golang.org/protobuf/internal/impl.(*MessageInfo).makeKnownFieldsFunc(0xc0108fa9a0, {0xffffffffffffffff, {0x0, 0x0}, 0xffffffffffffffff, {0x0, 0x0}, 0xffffffffffffffff, {0x0, 0x0}, ...})
        /home/deividas/go/pkg/mod/google.golang.org/protobuf@v1.34.2/internal/impl/message_reflect.go:80 +0x78a
google.golang.org/protobuf/internal/impl.(*MessageInfo).makeReflectFuncs(0xc0108fa9a0, {0x460fc10, 0x2e28b20}, {0xffffffffffffffff, {0x0, 0x0}, 0xffffffffffffffff, {0x0, 0x0}, 0xffffffffffffffff, ...})
        /home/deividas/go/pkg/mod/google.golang.org/protobuf@v1.34.2/internal/impl/message_reflect.go:42 +0x58
google.golang.org/protobuf/internal/impl.(*MessageInfo).initOnce(0xc0108fa9a0)
        /home/deividas/go/pkg/mod/google.golang.org/protobuf@v1.34.2/internal/impl/message.go:90 +0x1b0
google.golang.org/protobuf/internal/impl.(*MessageInfo).init(...)
        /home/deividas/go/pkg/mod/google.golang.org/protobuf@v1.34.2/internal/impl/message.go:72
google.golang.org/protobuf/internal/impl.(*messageReflectWrapper).ProtoMethods(0x458b1e0?)
        /home/deividas/go/pkg/mod/google.golang.org/protobuf@v1.34.2/internal/impl/message_reflect_gen.go:162 +0x25
google.golang.org/protobuf/proto.protoMethods(...)
        /home/deividas/go/pkg/mod/google.golang.org/protobuf@v1.34.2/proto/proto_methods.go:19
google.golang.org/protobuf/proto.UnmarshalOptions.unmarshal({{}, 0x1, 0x1, 0x0, {0x4595590, 0xc00028a300}, 0x2710}, {0xc0108c2ab0, 0x12, 0x12}, ...)
        /home/deividas/go/pkg/mod/google.golang.org/protobuf@v1.34.2/proto/decode.go:95 +0xe8
google.golang.org/protobuf/proto.Unmarshal({0xc0108c2ab0, 0x12, 0x12}, {0x458b1e0?, 0xc0108f2ad0?})
        /home/deividas/go/pkg/mod/google.golang.org/protobuf@v1.34.2/proto/decode.go:57 +0x5d
google.golang.org/grpc/encoding/proto.codec.Unmarshal({}, {0xc0108c2ab0?, 0x12?, 0x12?}, {0x31246a0?, 0xc0101bd888?})
        /home/deividas/go/pkg/mod/google.golang.org/grpc@v1.65.0/encoding/proto/proto.go:56 +0x65
google.golang.org/grpc.recv(0xc0108f2a80?, {0x7f8d7829f7c0, 0x6551e00}, 0xc0101bd8b0?, {0x0?, 0x0?}, {0x31246a0, 0xc0101bd888}, 0x3124a60?, 0xc00ca5f218, ...)
        /home/deividas/go/pkg/mod/google.golang.org/grpc@v1.65.0/rpc_util.go:839 +0xc2
google.golang.org/grpc.(*csAttempt).recvMsg(0xc0108b12b0, {0x31246a0, 0xc0101bd888}, 0x7f8d7829f7c0?)
        /home/deividas/go/pkg/mod/google.golang.org/grpc@v1.65.0/stream.go:1086 +0x289
google.golang.org/grpc.(*clientStream).RecvMsg.func1(0x0?)
        /home/deividas/go/pkg/mod/google.golang.org/grpc@v1.65.0/stream.go:929 +0x1f
google.golang.org/grpc.(*clientStream).withRetry(0xc010d96000, 0xc00ca5f338, 0xc00ca5f380)
        /home/deividas/go/pkg/mod/google.golang.org/grpc@v1.65.0/stream.go:778 +0x13a
google.golang.org/grpc.(*clientStream).RecvMsg(0xc010d96000, {0x31246a0?, 0xc0101bd888?})
        /home/deividas/go/pkg/mod/google.golang.org/grpc@v1.65.0/stream.go:928 +0x110
google.golang.org/grpc.invoke({0x45c5038?, 0xc001a199e0?}, {0x336b69f?, 0x2?}, {0x3124a60, 0x6551e00}, {0x31246a0, 0xc0101bd888}, 0x7f8d7077ce88?, {0x0, ...})
        /home/deividas/go/pkg/mod/google.golang.org/grpc@v1.65.0/call.go:73 +0xcb
google.golang.org/grpc.(*ClientConn).Invoke(0xc00205ac08, {0x45c5038?, 0xc001a199e0?}, {0x336b69f?, 0xc00ca5f500?}, {0x3124a60?, 0x6551e00?}, {0x31246a0?, 0xc0101bd888?}, {0x0, ...})
        /home/deividas/go/pkg/mod/google.golang.org/grpc@v1.65.0/call.go:37 +0x23f
github.com/osmosis-labs/osmosis/v26/x/txfees/types.(*queryClient).GetEipBaseFee(0xc0108f2570, {0x45c5038, 0xc001a199e0}, 0x6551e00, {0x0, 0x0, 0x0})
        /home/deividas/go/pkg/mod/github.com/osmosis-labs/osmosis/v26@v26.0.0-20240827120025-dd5be6c09d02/x/txfees/types/query.pb.go:585 +0xc8
github.com/osmosis-labs/sqs/domain/cosmos/tx.CalculateFeeCoin({0x45c5038, 0xc001a199e0}, 0x32ea097?, 0x2bf58)
        /home/deividas/go/src/github.com/deividaspetraitis/sqs/domain/cosmos/tx/tx.go:195 +0x9a
github.com/osmosis-labs/sqs/domain/cosmos/tx.BuildTx({0x45c5038, 0xc001a199e0}, 0xc001b122c8, {0x459bf28?, 0xc00181e5e8?}, {{0x45dffc0, 0xc001b0dbc0}, {0x4605038, 0xc00009d440}, {0x45dd338, ...}, ...}, ...)
        /home/deividas/go/src/github.com/deividaspetraitis/sqs/domain/cosmos/tx/tx.go:66 +0x21c
github.com/osmosis-labs/sqs/ingest/usecase/plugins/orderbook/claimbot.sendBatchClaimTx({0x45c5038, 0xc001a199e0}, {0x459bf28, 0xc00181e5e8}, 0xc001b122c8, {0x4581fc0, 0xc000cd72c0}, {0xc0025dfbc0, 0x3f}, {0xc000bfb080, ...})
        /home/deividas/go/src/github.com/deividaspetraitis/sqs/ingest/usecase/plugins/orderbook/claimbot/tx.go:91 +0x2cf
github.com/osmosis-labs/sqs/ingest/usecase/plugins/orderbook/claimbot.(*claimbot).processBatchClaimOrders(0xc001624d80, {0x45c5038, 0xc001a199e0}, {{0xc002c93680, 0x56}, {0xc002c936d7, 0x44}, 0x78a, {0xc0025dfbc0, 0x3f}}, ...)
        /home/deividas/go/src/github.com/deividaspetraitis/sqs/ingest/usecase/plugins/orderbook/claimbot/plugin.go:137 +0x245
github.com/osmosis-labs/sqs/ingest/usecase/plugins/orderbook/claimbot.(*claimbot).ProcessEndBlock(0xc001624d80, {0x45c5038?, 0xc000cbf7d0?}, 0x1559520, {0xc000d174a0?, 0xc000cbfb60?, 0xc000cbfb00?})
        /home/deividas/go/src/github.com/deividaspetraitis/sqs/ingest/usecase/plugins/orderbook/claimbot/plugin.go:117 +0x4d2
github.com/osmosis-labs/sqs/ingest/usecase.(*ingestUseCase).executeEndBlockProcessPlugins(0xc0012c1420, {0x45c5038, 0xc000cbf7d0}, 0x1559520, {0xc000d174a0?, 0xc000cbfb60?, 0xc000cbfb00?})
        /home/deividas/go/src/github.com/deividaspetraitis/sqs/ingest/usecase/ingest_usecase.go:481 +0xed
created by github.com/osmosis-labs/sqs/ingest/usecase.(*ingestUseCase).ProcessBlockData in goroutine 162
        /home/deividas/go/src/github.com/deividaspetraitis/sqs/ingest/usecase/ingest_usecase.go:199 +0xe05
exit status 2

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But LegacyDec is just a string, and we have other messages / queries that function without custom codec.

I'm still unsure why this is necessary.

Is it possible to import this client:
https://github.com/osmosis-labs/osmosis/blob/09ff5a2d72306f5b5a4b4dd29996cb5ab37136d7/x/txfees/keeper/grpc_query.go#L25-L28

and then use this method:
https://github.com/osmosis-labs/osmosis/blob/09ff5a2d72306f5b5a4b4dd29996cb5ab37136d7/x/txfees/keeper/grpc_query.go#L89-L92

Copy link
Collaborator Author

@deividaspetraitis deividaspetraitis Oct 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may be missing some key details, but it seems that constructing txfees.NewQuerier would much more involving since it expects Keeper as parameter:

type Keeper struct {
	storeKey storetypes.StoreKey

	accountKeeper      types.AccountKeeper
	bankKeeper         types.BankKeeper
	poolManager        types.PoolManager
	protorevKeeper     types.ProtorevKeeper
	distributionKeeper types.DistributionKeeper
	consensusKeeper    types.ConsensusKeeper
	dataDir            string

	paramSpace paramtypes.Subspace
}

As a reference I was using both node/sqs repos to find examples of how to construct NewQuerier in simple way, but with no results, also I have doubts whether this apporoach would resolve the issue, because it uses same QueryEipBaseFeeResponse as we do here in SQS. This data structure has field BaseFee of a type osmomath.LegacyDec and I am getting this error with following command too:

❯ osmosisd q txfees base-fee --grpc-addr 127.0.0.1:9090 --grpc-insecure
panic: invalid Go type math.LegacyDec for field osmosis.txfees.v1beta1.QueryEipBaseFeeResponse.base_fee

goroutine 1 [running]:
google.golang.org/protobuf/internal/impl.newSingularConverter({0x639d528, 0x5678e00}, {0x639ecb8, 0xc001c6fb00})
        google.golang.org/protobuf@v1.34.2/internal/impl/convert.go:142 +0x9ad
google.golang.org/protobuf/internal/impl.NewConverter({0x639d528, 0x5678e00}, {0x639ecb8, 0xc001c6fb00})
        google.golang.org/protobuf@v1.34.2/internal/impl/convert.go:60 +0x8f
google.golang.org/protobuf/internal/impl.fieldInfoForScalar({0x639ecb8, 0xc001c6fb00}, {{0x5207b1e, 0x7}, {0x0, 0x0}, {0x639d528, 0x5678e00}, {0x5207b26, 0x7f}, ...}, ...)
        google.golang.org/protobuf@v1.34.2/internal/impl/message_reflect_field.go:270 +0x193
google.golang.org/protobuf/internal/impl.(*MessageInfo).makeKnownFieldsFunc(0xc0028f2000, {0xffffffffffffffff, {0x0, 0x0}, 0xffffffffffffffff, {0x0, 0x0}, 0xffffffffffffffff, {0x0, 0x0}, ...})
        google.golang.org/protobuf@v1.34.2/internal/impl/message_reflect.go:80 +0x78a
google.golang.org/protobuf/internal/impl.(*MessageInfo).makeReflectFuncs(0xc0028f2000, {0x639d528, 0x51447c0}, {0xffffffffffffffff, {0x0, 0x0}, 0xffffffffffffffff, {0x0, 0x0}, 0xffffffffffffffff, ...})
        google.golang.org/protobuf@v1.34.2/internal/impl/message_reflect.go:42 +0x58
google.golang.org/protobuf/internal/impl.(*MessageInfo).initOnce(0xc0028f2000)
        google.golang.org/protobuf@v1.34.2/internal/impl/message.go:90 +0x1b0
google.golang.org/protobuf/internal/impl.(*MessageInfo).init(...)
        google.golang.org/protobuf@v1.34.2/internal/impl/message.go:72
google.golang.org/protobuf/internal/impl.(*messageReflectWrapper).ProtoMethods(0x630a4a0?)
        google.golang.org/protobuf@v1.34.2/internal/impl/message_reflect_gen.go:162 +0x25
google.golang.org/protobuf/proto.protoMethods(...)
        google.golang.org/protobuf@v1.34.2/proto/proto_methods.go:19
google.golang.org/protobuf/proto.UnmarshalOptions.unmarshal({{}, 0x1, 0x1, 0x0, {0x6315d20, 0xc0002b0120}, 0x2710}, {0xc00288a798, 0x12, 0x12}, ...)
        google.golang.org/protobuf@v1.34.2/proto/decode.go:95 +0xe8
google.golang.org/protobuf/proto.Unmarshal({0xc00288a798, 0x12, 0x12}, {0x630a4a0?, 0xc0028bb4c0?})
        google.golang.org/protobuf@v1.34.2/proto/decode.go:57 +0x5d
google.golang.org/grpc/encoding/proto.codec.Unmarshal({}, {0xc00288a798?, 0x12?, 0x12?}, {0x54b12c0?, 0xc002e82dc0?})
        google.golang.org/grpc@v1.65.0/encoding/proto/proto.go:56 +0x65
google.golang.org/grpc.recv(0xc00300e3c0?, {0x7fec4c175698, 0x88b2560}, 0xc00118a918?, {0x0?, 0x0?}, {0x54b12c0, 0xc002e82dc0}, 0x54b1680?, 0x0, ...)
        google.golang.org/grpc@v1.65.0/rpc_util.go:839 +0xc2
google.golang.org/grpc.(*csAttempt).recvMsg(0xc002f9fa00, {0x54b12c0, 0xc002e82dc0}, 0x7fec4c175698?)
        google.golang.org/grpc@v1.65.0/stream.go:1086 +0x289
google.golang.org/grpc.(*clientStream).RecvMsg.func1(0x0?)
        google.golang.org/grpc@v1.65.0/stream.go:929 +0x1f
google.golang.org/grpc.(*clientStream).withRetry(0xc00309d200, 0xc000e56108, 0xc000e56150)
        google.golang.org/grpc@v1.65.0/stream.go:778 +0x13a
google.golang.org/grpc.(*clientStream).RecvMsg(0xc00309d200, {0x54b12c0?, 0xc002e82dc0?})
        google.golang.org/grpc@v1.65.0/stream.go:928 +0x110
google.golang.org/grpc.invoke({0x6348f48?, 0xc002fc0360?}, {0x5747b24?, 0x4?}, {0x54b1680, 0x88b2560}, {0x54b12c0, 0xc002e82dc0}, 0x18?, {0x88b2560, ...})
        google.golang.org/grpc@v1.65.0/call.go:73 +0xcb
google.golang.org/grpc.(*ClientConn).Invoke(0xc0029ccc08, {0x6348f48?, 0xc002fc0360?}, {0x5747b24?, 0x15?}, {0x54b1680?, 0x88b2560?}, {0x54b12c0?, 0xc002e82dc0?}, {0x88b2560, ...})
        google.golang.org/grpc@v1.65.0/call.go:37 +0x23f
github.com/cosmos/cosmos-sdk/client.Context.Invoke({{0x0, 0x0, 0x0}, {0x6374338, 0xc002fc0030}, 0xc0029ccc08, {0x0, 0x0}, {0x6392e10, 0xc001318fb0}, ...}, ...)
        github.com/cosmos/cosmos-sdk@v0.50.9/client/grpc_query.go:61 +0x785
github.com/osmosis-labs/osmosis/v26/x/txfees/types.(*queryClient).GetEipBaseFee(0xc0030b6030, {0x6348f48, 0xc002fc0360}, 0x88b2560, {0x88b2560, 0x0, 0x0})
        github.com/osmosis-labs/osmosis/v26/x/txfees/types/query.pb.go:585 +0xc8
reflect.Value.call({0x5271780?, 0xc0030b6030?, 0xc0030b6030?}, {0x56a5414, 0x4}, {0xc000e57660, 0x2, 0x639d528?})
        reflect/value.go:596 +0xca6
reflect.Value.Call({0x5271780?, 0xc0030b6030?, 0x0?}, {0xc000e57660?, 0x0?, 0x0?})
        reflect/value.go:380 +0xb9
github.com/osmosis-labs/osmosis/osmoutils/osmocli.callQueryClientFn({0x6348f48, 0xc002fc0360}, {0x56b6fd0, 0xd}, {0x6320500, 0x88b2560}, {0x5271780?, 0xc0030b6030?})
        github.com/osmosis-labs/osmosis/osmoutils@v0.0.14/osmocli/query_cmd_wrap.go:158 +0x274
github.com/osmosis-labs/osmosis/osmoutils/osmocli.BuildQueryCli[...].func3({0xc002f7cc30, 0x0, 0x3})
        github.com/osmosis-labs/osmosis/osmoutils@v0.0.14/osmocli/query_cmd_wrap.go:187 +0x1ae
github.com/spf13/cobra.(*Command).execute(0xc002c76608, {0xc002f7cbd0, 0x3, 0x3})
        github.com/spf13/cobra@v1.8.1/command.go:985 +0xaca
github.com/spf13/cobra.(*Command).ExecuteC(0xc0027e6308)
        github.com/spf13/cobra@v1.8.1/command.go:1117 +0x3ff
github.com/spf13/cobra.(*Command).Execute(...)
        github.com/spf13/cobra@v1.8.1/command.go:1041
github.com/spf13/cobra.(*Command).ExecuteContext(...)
        github.com/spf13/cobra@v1.8.1/command.go:1034
github.com/cosmos/cosmos-sdk/server/cmd.Execute(0xc0027e6308, {0x56abd2e, 0x8}, {0xc001107b18, 0x18})
        github.com/cosmos/cosmos-sdk@v0.50.9/server/cmd/execute.go:34 +0x187
main.main()
        github.com/osmosis-labs/osmosis/v26/cmd/osmosisd/main.go:16 +0x37

This issue is documented on cosmos-sdk too: cosmos/cosmos-sdk#18430 (comment)

}

grpcConn, err := grpc.NewClient(
grpcEndpoint,
grpcOpts...,
)
p0mvn marked this conversation as resolved.
Show resolved Hide resolved
if err != nil {
return nil, fmt.Errorf("failed to dial Cosmos gRPC service: %w", err)
}

return &Client{
ClientConn: grpcConn,
}, nil
p0mvn marked this conversation as resolved.
Show resolved Hide resolved
}
38 changes: 38 additions & 0 deletions delivery/grpc/grpc_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
package grpc_test

import (
"testing"

"github.com/osmosis-labs/sqs/delivery/grpc"

"github.com/stretchr/testify/assert"
)

// TestNewClient tests the NewClient function
func TestNewClient(t *testing.T) {
tests := []struct {
name string
endpoint string
wantErr bool
}{
{
name: "Valid endpoint",
endpoint: "localhost:9090",
wantErr: false,
},
}

for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
client, err := grpc.NewClient(tt.endpoint)
if tt.wantErr {
assert.Error(t, err)
assert.Nil(t, client)
} else {
assert.NoError(t, err)
assert.NotNil(t, client)
assert.NotNil(t, client.ClientConn)
}
})
}
}
38 changes: 38 additions & 0 deletions delivery/http/get.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
package http

import (
"context"
"io"
"log"
"net"
"net/http"
"time"
)

// Get issues GET request to given URL using default httpClient.
func Get(ctx context.Context, url string) ([]byte, error) {
ctx, cancel := context.WithTimeout(ctx, 10*time.Second)
defer cancel()

req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return nil, err
}
p0mvn marked this conversation as resolved.
Show resolved Hide resolved

resp, err := DefaultClient.Do(req)
if err != nil {
netErr, ok := err.(net.Error)
if ok && netErr.Timeout() {
log.Printf("Request to %s timed out, continuing...", url)
return nil, nil
}
return nil, err
}
p0mvn marked this conversation as resolved.
Show resolved Hide resolved
defer resp.Body.Close()

body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, err
}
return body, nil
}
p0mvn marked this conversation as resolved.
Show resolved Hide resolved
71 changes: 71 additions & 0 deletions delivery/http/get_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
package http_test

import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"

sqshttp "github.com/osmosis-labs/sqs/delivery/http"

"github.com/stretchr/testify/assert"
)

func TestGet(t *testing.T) {
tests := []struct {
name string
url string
expectedBody string
timeout time.Duration
serverResponse func(w http.ResponseWriter, r *http.Request)
}{
{
name: "Success",
url: "/success",
expectedBody: "Hello, World!",
serverResponse: func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hello, World!"))
},
},
{
name: "Timeout",
url: "/timeout",
expectedBody: "",
timeout: 10 * time.Millisecond,
serverResponse: func(w http.ResponseWriter, r *http.Request) {
time.Sleep(20 * time.Millisecond)
w.Write([]byte("Too late"))
},
},
{
name: "Server Error",
url: "/error",
expectedBody: "Internal Server Error\n",
serverResponse: func(w http.ResponseWriter, r *http.Request) {
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
},
},
}

defaultTimeout := sqshttp.DefaultClient.Timeout
resetClient := func() {
sqshttp.DefaultClient.Timeout = defaultTimeout
}

for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(tt.serverResponse))
defer server.Close()

sqshttp.DefaultClient.Timeout = tt.timeout
defer resetClient()

ctx := context.Background()
body, err := sqshttp.Get(ctx, server.URL+tt.url)
assert.NoError(t, err)
assert.Equal(t, string(body), tt.expectedBody)

})
}
}
11 changes: 11 additions & 0 deletions delivery/http/http.go
Original file line number Diff line number Diff line change
@@ -1,10 +1,21 @@
package http

import (
"net/http"
"time"

"github.com/labstack/echo/v4"
"github.com/osmosis-labs/sqs/validator"

"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
)

// DefaultClient represents default HTTP client for issuing outgoing HTTP requests.
var DefaultClient = &http.Client{
Timeout: 5 * time.Second, // Adjusted timeout to 5 seconds
Transport: otelhttp.NewTransport(http.DefaultTransport),
}

// RequestUnmarshaler is any type capable to unmarshal data from HTTP request to itself.
type RequestUnmarshaler interface {
UnmarshalHTTPRequest(c echo.Context) error
Expand Down
Loading