Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ctl: replace gc_safepoint call with PD HTTP SDK #8504

Merged
merged 36 commits into from
Aug 21, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
3d65bc8
gc_safepoint HTTP SDK
JackL9u Aug 7, 2024
c307b89
fix an indentation mismatch
JackL9u Aug 7, 2024
637c50c
fix some coding style issues
JackL9u Aug 8, 2024
002862c
format the code
JackL9u Aug 8, 2024
419b903
fix a var-name issue
JackL9u Aug 8, 2024
6ac2bc2
add test functions
JackL9u Aug 8, 2024
8a8a40f
add test functions for gc safepoint call
JackL9u Aug 8, 2024
cc78799
fix coding style for testing suite
JackL9u Aug 8, 2024
c88639b
fix coding style for testing suite again
JackL9u Aug 8, 2024
870f9cc
changed the test case for get SafePoint
JackL9u Aug 8, 2024
be5c4a1
edit test function
JackL9u Aug 8, 2024
d6ed9d8
edited some testcases
JackL9u Aug 8, 2024
2d278ca
put get/delete safepoint test into a single function
JackL9u Aug 8, 2024
6c7de04
removed unnecessary comments
JackL9u Aug 8, 2024
11d5353
made other modifications
JackL9u Aug 9, 2024
acec910
changed test funciton name
JackL9u Aug 9, 2024
1a1912a
do some experiments
JackL9u Aug 9, 2024
83de36f
delete my test function
JackL9u Aug 9, 2024
cd0a7d5
add an empty test funciton
JackL9u Aug 9, 2024
3858cb3
remove the test delete logic from test funciton
JackL9u Aug 9, 2024
11be503
remove the test delete logic from test funciton, and move the test lo…
JackL9u Aug 9, 2024
dc62cc2
add requirePDClient to cobra command
JackL9u Aug 9, 2024
7adce30
add the delete test back
JackL9u Aug 9, 2024
014ec40
try again
JackL9u Aug 9, 2024
4666fb5
add sorting logic to gc_safepoint test
JackL9u Aug 15, 2024
670fe6a
add a second-round get check after safepoints are deleted
JackL9u Aug 15, 2024
fb8a5b5
remove the requirePDClient in delete subcommand
JackL9u Aug 15, 2024
7ef43f2
fix a typo
JackL9u Aug 15, 2024
98b09b7
added tests
JackL9u Aug 15, 2024
785e975
add the delete gc_worker check
JackL9u Aug 15, 2024
f522d45
add a comment
JackL9u Aug 15, 2024
6f4d633
modified the delete gc_worker test
JackL9u Aug 16, 2024
e3517c8
run the tests again
JackL9u Aug 16, 2024
d25953b
add a comment
JackL9u Aug 16, 2024
dded6c8
add delete non-exist safepoints tests
JackL9u Aug 21, 2024
7965d10
Merge branch 'master' into dev1
ti-chi-bot[bot] Aug 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions client/http/api.go
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ const (
Status = "/pd/api/v1/status"
Version = "/pd/api/v1/version"
operators = "/pd/api/v1/operators"
safepoint = "pd/api/v1/gc/safepoint"
JackL9u marked this conversation as resolved.
Show resolved Hide resolved
// Micro Service
microServicePrefix = "/pd/api/v2/ms"
// Keyspace
Expand Down
30 changes: 30 additions & 0 deletions client/http/interface.go
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,8 @@
/* Other interfaces */
GetMinResolvedTSByStoresIDs(context.Context, []uint64) (uint64, map[uint64]uint64, error)
GetPDVersion(context.Context) (string, error)
GetGCSafePoint(context.Context) (ListServiceGCSafepoint, error)
DeleteGCSafePoint(context.Context, string) (string, error)
/* Micro Service interfaces */
GetMicroServiceMembers(context.Context, string) ([]MicroServiceMember, error)
GetMicroServicePrimary(context.Context, string) (string, error)
Expand Down Expand Up @@ -1024,3 +1026,31 @@
}
return &keyspaceMetaPB, nil
}

// GetGCSafePoint get the gc_safepoint
JmPotato marked this conversation as resolved.
Show resolved Hide resolved
func (c *client) GetGCSafePoint(ctx context.Context) (ListServiceGCSafepoint, error) {
var gcsafepoint ListServiceGCSafepoint
err := c.request(ctx, newRequestInfo().
WithName(GetGCSafePointName).
WithURI(safepoint).
WithMethod(http.MethodGet).
WithResp(&gcsafepoint))
if err != nil {
return gcsafepoint, err

Check warning on line 1039 in client/http/interface.go

View check run for this annotation

Codecov / codecov/patch

client/http/interface.go#L1031-L1039

Added lines #L1031 - L1039 were not covered by tests
}
return gcsafepoint, nil

Check warning on line 1041 in client/http/interface.go

View check run for this annotation

Codecov / codecov/patch

client/http/interface.go#L1041

Added line #L1041 was not covered by tests
JackL9u marked this conversation as resolved.
Show resolved Hide resolved
}

// DeleteGCSafePoint deletes a gc_safepoint
JmPotato marked this conversation as resolved.
Show resolved Hide resolved
func (c *client) DeleteGCSafePoint(ctx context.Context, service_id string) (string, error) {
JmPotato marked this conversation as resolved.
Show resolved Hide resolved
var msg string
err := c.request(ctx, newRequestInfo().
WithName(DeleteGCSafePointName).
WithURI(safepoint+"/"+service_id).
WithMethod(http.MethodDelete).
WithResp(&msg))
if err != nil {
return msg, err

Check warning on line 1053 in client/http/interface.go

View check run for this annotation

Codecov / codecov/patch

client/http/interface.go#L1045-L1053

Added lines #L1045 - L1053 were not covered by tests
}
return msg, nil

Check warning on line 1055 in client/http/interface.go

View check run for this annotation

Codecov / codecov/patch

client/http/interface.go#L1055

Added line #L1055 was not covered by tests
}
JmPotato marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 2 additions & 0 deletions client/http/request_info.go
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,8 @@ const (
deleteOperators = "DeleteOperators"
UpdateKeyspaceGCManagementTypeName = "UpdateKeyspaceGCManagementType"
GetKeyspaceMetaByNameName = "GetKeyspaceMetaByName"
GetGCSafePointName = "GetGCSafePoint"
DeleteGCSafePointName = "DeleteGCSafePoint"
)

type requestInfo struct {
Expand Down
14 changes: 14 additions & 0 deletions client/http/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,20 @@ import (
pd "github.com/tikv/pd/client"
)

// NOTE: This type is in sync with pd/pkg/storage/endpoint/gc_safe_point.go
JackL9u marked this conversation as resolved.
Show resolved Hide resolved
type ServiceSafePoint struct {
ServiceID string `json:"service_id"`
ExpiredAt int64 `json:"expired_at"`
SafePoint uint64 `json:"safe_point"`
}

// NOTE: This type is in sync with pd/server/spi/service_gc_safepoint.go
type ListServiceGCSafepoint struct {
ServiceGCSafepoints []*ServiceSafePoint `json:"service_gc_safe_points"`
MinServiceGcSafepoint uint64 `json:"min_service_gc_safe_point,omitempty"`
GCSafePoint uint64 `json:"gc_safe_point"`
}

// ClusterState saves some cluster state information.
// NOTE: This type sync with https://github.com/tikv/pd/blob/5eae459c01a797cbd0c416054c6f0cad16b8740a/server/cluster/cluster.go#L173
type ClusterState struct {
Expand Down
1 change: 1 addition & 0 deletions pkg/storage/endpoint/gc_safe_point.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ import (

// ServiceSafePoint is the safepoint for a specific service
// NOTE: This type is exported by HTTP API. Please pay more attention when modifying it.
// this type is in sync with client/http/types.go
JmPotato marked this conversation as resolved.
Show resolved Hide resolved
type ServiceSafePoint struct {
ServiceID string `json:"service_id"`
ExpiredAt int64 `json:"expired_at"`
Expand Down
1 change: 1 addition & 0 deletions server/api/service_gc_safepoint.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ func newServiceGCSafepointHandler(svr *server.Server, rd *render.Render) *servic

// ListServiceGCSafepoint is the response for list service GC safepoint.
// NOTE: This type is exported by HTTP API. Please pay more attention when modifying it.
// this type is now in sync with pd/client/http/types.go ListServiceGCSafepoint
JmPotato marked this conversation as resolved.
Show resolved Hide resolved
type ListServiceGCSafepoint struct {
ServiceGCSafepoints []*endpoint.ServiceSafePoint `json:"service_gc_safe_points"`
MinServiceGcSafepoint uint64 `json:"min_service_gc_safe_point,omitempty"`
Expand Down
21 changes: 4 additions & 17 deletions tools/pd-ctl/pdctl/command/gc_safepoint_command.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,9 @@

import (
"encoding/json"
"net/http"
"sort"

"github.com/spf13/cobra"
"github.com/tikv/pd/server/api"
)

var (
serviceGCSafepointPrefix = "pd/api/v1/gc/safepoint"
)

// NewServiceGCSafepointCommand return a service gc safepoint subcommand of rootCmd
Expand All @@ -50,16 +44,11 @@
}

func showSSPs(cmd *cobra.Command, _ []string) {
r, err := doRequest(cmd, serviceGCSafepointPrefix, http.MethodGet, http.Header{})
safepoint, err := PDCli.GetGCSafePoint(cmd.Context())

Check warning on line 47 in tools/pd-ctl/pdctl/command/gc_safepoint_command.go

View check run for this annotation

Codecov / codecov/patch

tools/pd-ctl/pdctl/command/gc_safepoint_command.go#L47

Added line #L47 was not covered by tests
if err != nil {
cmd.Printf("Failed to get service GC safepoint: %s\n", err)
return
}
var safepoint api.ListServiceGCSafepoint
if err := json.Unmarshal([]byte(r), &safepoint); err != nil {
cmd.Printf("Failed to unmarshal service GC safepoint: %s\n", err)
return
}
sort.Slice(safepoint.ServiceGCSafepoints, func(i, j int) bool {
return safepoint.ServiceGCSafepoints[i].SafePoint < safepoint.ServiceGCSafepoints[j].SafePoint
})
Expand All @@ -68,20 +57,18 @@
cmd.Printf("Failed to marshal service GC safepoint: %s\n", err)
return
}
cmd.Println(string(data))
jsonPrint(cmd, string(data))

Check warning on line 60 in tools/pd-ctl/pdctl/command/gc_safepoint_command.go

View check run for this annotation

Codecov / codecov/patch

tools/pd-ctl/pdctl/command/gc_safepoint_command.go#L60

Added line #L60 was not covered by tests
JmPotato marked this conversation as resolved.
Show resolved Hide resolved
}

func deleteSSP(cmd *cobra.Command, args []string) {
if len(args) != 1 {
cmd.Usage()
return
}
serviceID := args[0]
deleteURL := serviceGCSafepointPrefix + "/" + serviceID
r, err := doRequest(cmd, deleteURL, http.MethodDelete, http.Header{})
r, err := PDCli.DeleteGCSafePoint(cmd.Context(), args[0])

Check warning on line 68 in tools/pd-ctl/pdctl/command/gc_safepoint_command.go

View check run for this annotation

Codecov / codecov/patch

tools/pd-ctl/pdctl/command/gc_safepoint_command.go#L68

Added line #L68 was not covered by tests
if err != nil {
cmd.Printf("Failed to delete service GC safepoint: %s\n", err)
return
}
cmd.Println(r)
jsonPrint(cmd, r)

Check warning on line 73 in tools/pd-ctl/pdctl/command/gc_safepoint_command.go

View check run for this annotation

Codecov / codecov/patch

tools/pd-ctl/pdctl/command/gc_safepoint_command.go#L73

Added line #L73 was not covered by tests
}
Loading