Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[vtctld] Migrate topo management RPCs #7395

Merged
merged 30 commits into from
Feb 4, 2021
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
c26306c
Add a bunch of topo management rpcs, run the generators, add test stubs
ajm188 Jan 24, 2021
7175fbc
Implement ChangeTabletType, add mock to testutil.tabletManagerClient,…
ajm188 Jan 24, 2021
653d3f3
Extract tablet awk-formatting to cli function
ajm188 Jan 24, 2021
a7bb06e
Implement CLI entrypoint for `ChangeTabletType`
ajm188 Jan 24, 2021
e36b6b6
Implement `CreateKeyspace` with tests
ajm188 Jan 25, 2021
88a4431
Add parser helpers for keyspace enum types and extract tablet type pa…
ajm188 Jan 25, 2021
9cfc46b
Implement CLI entrypoint for `CreateKeyspace`
ajm188 Jan 25, 2021
c5ed719
Implement GetShard and tests
ajm188 Jan 25, 2021
cd30750
Implement CLI for `GetShard`
ajm188 Jan 25, 2021
8d7426f
Add `AddShards` helper to testutil
ajm188 Jan 25, 2021
4dd2a96
Implement RemoveShardCell and add extensive testing
ajm188 Jan 25, 2021
d58e478
Extract `RemoveShardCell` logic to helper function for reuse in `Remo…
ajm188 Jan 26, 2021
9983e22
Implement `RemoveKeyspaceCell` with tests
ajm188 Jan 26, 2021
0e69a34
Add CLI entrypoint to `RemoveKeyspaceCell`
ajm188 Jan 26, 2021
c379ca7
Implement `CreateShard` with tests
ajm188 Jan 26, 2021
7e4a8be
Add CLI entrypoint for `CreateShard`
ajm188 Jan 26, 2021
5d8090c
Move isMasterTablet to a public function in grpcvtctldserver, call fr…
ajm188 Jan 26, 2021
d6cc15f
Rename `IsMasterTablet` to `IsPrimaryTablet`
ajm188 Jan 26, 2021
25e0a48
Move `IsPrimaryTablet` to topotools, which is probably a better share…
ajm188 Jan 26, 2021
4e118c8
Implement `DeleteTablets` and add tests
ajm188 Jan 26, 2021
e713f57
Add CLI entrypoint for `DeleteTablets`
ajm188 Jan 26, 2021
c20a128
Implement `DeleteShards` with tests
ajm188 Jan 27, 2021
762a813
Add CLI entrypoint for `DeleteShards`
ajm188 Jan 27, 2021
7030f62
Implement `DeleteKeyspace` and tests
ajm188 Jan 27, 2021
74c4883
Add CLI entrypoint for `DeleteKeyspace`
ajm188 Jan 27, 2021
01560e4
Reorder imports
ajm188 Jan 28, 2021
1efbfca
Use `topotools` for a slightly more complete ChangeType fake implemen…
ajm188 Jan 30, 2021
b834775
Add context around flag name to all parsing errors
ajm188 Feb 2, 2021
24e4527
Log error on read-after-write for ChangeTabletType, and handle nil
ajm188 Feb 2, 2021
99dd25d
PR feedback: fix comments
ajm188 Feb 4, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions go/cmd/vtctldclient/cli/awk.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,13 @@ import (
"fmt"
"sort"
"strings"
"time"

"vitess.io/vitess/go/vt/logutil"
"vitess.io/vitess/go/vt/topo"
"vitess.io/vitess/go/vt/topo/topoproto"

topodatapb "vitess.io/vitess/go/vt/proto/topodata"
)

// MarshalMapAWK returns a string representation of a string->string map in an
Expand All @@ -38,3 +45,29 @@ func MarshalMapAWK(m map[string]string) string {

return "[" + strings.Join(pairs, " ") + "]"
}

// MarshalTabletAWK marshals a tablet into an AWK-friendly line.
func MarshalTabletAWK(t *topodatapb.Tablet) string {
ti := topo.TabletInfo{
Tablet: t,
}

keyspace := t.Keyspace
if keyspace == "" {
keyspace = "<null>"
}

shard := t.Shard
if shard == "" {
shard = "<null>"
}

mtst := "<null>"
// special case for old primary that hasn't been updated in the topo
// yet.
if t.MasterTermStartTime != nil && t.MasterTermStartTime.Seconds > 0 {
mtst = logutil.ProtoToTime(t.MasterTermStartTime).Format(time.RFC3339)
}

return fmt.Sprintf("%v %v %v %v %v %v %v %v", topoproto.TabletAliasString(t.Alias), keyspace, shard, topoproto.TabletTypeLString(t.Type), ti.Addr(), ti.MysqlAddr(), MarshalMapAWK(t.Tags), mtst)
}
93 changes: 93 additions & 0 deletions go/cmd/vtctldclient/cli/pflag.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
/*
Copyright 2021 The Vitess Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package cli

import (
"github.com/spf13/pflag"

"vitess.io/vitess/go/flagutil"
"vitess.io/vitess/go/vt/key"
"vitess.io/vitess/go/vt/topo/topoproto"

topodatapb "vitess.io/vitess/go/vt/proto/topodata"
)

// StringMapValue augements flagutil.StringMapValue so it can be used as a
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: should be augments

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// pflag.Value.
type StringMapValue struct {
flagutil.StringMapValue
}

// Type is part of the pflag.Value interface.
func (v *StringMapValue) Type() string {
return "cli.StringMapValue"
}

// KeyspaceIDTypeFlag adds the pflag.Value interface to a
// topodatapb.KeyspaceIdType.
type KeyspaceIDTypeFlag topodatapb.KeyspaceIdType
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where is this used? didn't see a reference in this PR

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


var _ pflag.Value = (*KeyspaceIDTypeFlag)(nil)

// Set is part of the pflag.Value interface.
func (v *KeyspaceIDTypeFlag) Set(arg string) error {
t, err := key.ParseKeyspaceIDType(arg)
if err != nil {
return err
}

*v = KeyspaceIDTypeFlag(t)

return nil
}

// String is part of the pflag.Value interface.
func (v *KeyspaceIDTypeFlag) String() string {
return key.KeyspaceIDTypeString(topodatapb.KeyspaceIdType(*v))
}

// Type is part of the pflag.Value interface.
func (v *KeyspaceIDTypeFlag) Type() string {
return "cli.KeyspaceIdTypeFlag"
}

// KeyspaceTypeFlag adds the pflag.Value interface to a topodatapb.KeyspaceType.
type KeyspaceTypeFlag topodatapb.KeyspaceType

var _ pflag.Value = (*KeyspaceTypeFlag)(nil)

// Set is part of the pflag.Value interface.
func (v *KeyspaceTypeFlag) Set(arg string) error {
kt, err := topoproto.ParseKeyspaceType(arg)
if err != nil {
return err
}

*v = KeyspaceTypeFlag(kt)

return nil
}

// String is part of the pflag.Value interface.
func (v *KeyspaceTypeFlag) String() string {
return topoproto.KeyspaceTypeString(topodatapb.KeyspaceType(*v))
}

// Type is part of the pflag.Value interface.
func (v *KeyspaceTypeFlag) Type() string {
return "cli.KeyspaceTypeFlag"
}
43 changes: 43 additions & 0 deletions go/cmd/vtctldclient/cli/shards.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
/*
Copyright 20201 The Vitess Authors.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: 20201=>2021

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package cli

import (
"vitess.io/vitess/go/vt/topo/topoproto"

vtctldatapb "vitess.io/vitess/go/vt/proto/vtctldata"
)

// ParseKeyspaceShards takes a list of positional arguments and converts them to
// vtctldatapb.Shard objects.
func ParseKeyspaceShards(args []string) ([]*vtctldatapb.Shard, error) {
shards := make([]*vtctldatapb.Shard, 0, len(args))

for _, arg := range args {
keyspace, shard, err := topoproto.ParseKeyspaceShard(arg)
if err != nil {
return nil, err
}

shards = append(shards, &vtctldatapb.Shard{
Keyspace: keyspace,
Name: shard,
})
}

return shards, nil
}
40 changes: 40 additions & 0 deletions go/cmd/vtctldclient/cli/tablets.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
/*
Copyright 2021 The Vitess Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package cli

import (
"vitess.io/vitess/go/vt/topo/topoproto"

topodatapb "vitess.io/vitess/go/vt/proto/topodata"
)

// TabletAliasesFromPosArgs takes a list of positional (non-flag) arguments and
// converts them to tablet aliases.
func TabletAliasesFromPosArgs(args []string) ([]*topodatapb.TabletAlias, error) {
aliases := make([]*topodatapb.TabletAlias, 0, len(args))

for _, arg := range args {
alias, err := topoproto.ParseTabletAlias(arg)
if err != nil {
return nil, err
}

aliases = append(aliases, alias)
}

return aliases, nil
}
Loading