session: add no register API for using TiDB as a library #17513

YuJuncen · 2020-05-29T06:39:45Z

What problem does this PR solve?

Issue Number: close br#299

Problem Summary:

BR uses TiDB as a library, but currently when we call session.GetDomain, domap.Get will automatically register itself to PD. Then a strange TiDB will present at dashboard.

see more at br#299.

What is changed and how it works?

What's Changed:

I added a config option no-register to TiDB.

Then, when this is enabled, topologySyncer won't be started, hence this TiDB won't register itself to dashboard.

How it Works:

We add a if-guard to where initializing topologySyncer at infoSyncer.Init and Session.Init.

Check List

Tests

Manual test (add detailed scripts or steps below)
- I replace this version of TiDB to BR, and set no-register = true, then BR won't register it self as a TiDB anymore.

More things

Maybe we have more things to do, for now please add WIP tag for this:

~~Add test cases.~~ (IMO, this is too hard to be tested automatically.)
~~Don't make NoRegistered TiDB become DDL owner.~~ (We can set run-DDL = false)

Release note

No release note

…ster

CLAassistant · 2020-05-29T06:39:49Z

All committers have signed the CLA.

sre-bot · 2020-05-29T06:39:56Z

No release note, Please follow https://github.com/pingcap/community/blob/master/contributors/release-note-checker.md

codecov · 2020-05-29T06:46:19Z

Codecov Report

Merging #17513 into master will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #17513   +/-   ##
===========================================
  Coverage   79.3272%   79.3272%           
===========================================
  Files           541        541           
  Lines        145636     145636           
===========================================
  Hits         115529     115529           
  Misses        20812      20812           
  Partials       9295       9295

YuJuncen · 2020-05-29T06:58:43Z

@breeswish PTAL, or... Please add some labels...

bb7133 · 2020-06-01T04:26:48Z

When BR is integrated into TiDB, we don't need this PR, right? I think the integration is very close.

breezewish · 2020-06-01T04:37:48Z

domain/domain.go

@@ -655,6 +655,14 @@ func NewDomain(store kv.Storage, ddlLease time.Duration, statsLease time.Duratio

 // Init initializes a domain.
 func (do *Domain) Init(ddlLease time.Duration, sysFactory func(*Domain) (pools.Resource, error)) error {
+	if err := do.InitWithNoRegister(ddlLease, sysFactory); err != nil {


This also disables the DDL syncer, will it cause problems?

Seems InfoSyncer won't do things about DDL... Is there something I ignored...?

I don't know how BR uses the DDL structure. If BR uses InitWithNoRegister to new a DDL, then the DDL also will campaign DDL owner, but it will make load schema slower then Init with InfoSyncer. Is it OK?

…ster

session/tidb.go

…o-register

zimulala · 2020-06-23T11:02:25Z

domain/infosync/info.go

@@ -149,6 +149,9 @@ func (is *InfoSyncer) init(ctx context.Context) error {
 	if err != nil {
 		return err
 	}
+	if config.GetGlobalConfig().NoRegister {


I think we'd better to use config.GetGlobalConfig().NoRegister as a argument to GlobalInfoSyncerInit. It is best to reduce the strong correlation between internal objects and config.GetGlobalConfig().

config/config.go

YuJuncen · 2020-06-30T04:48:27Z

/run-all-tests

YuJuncen · 2020-06-30T05:08:13Z

That is strange... It fails but I cannot find any useful information about why it fails... Let me retry it.

/run-all-tests

zimulala

LGTM

ti-srebot · 2020-07-07T10:29:34Z

@zimulala,Thanks for your review.

YuJuncen · 2020-07-08T07:58:07Z

@breeswish PTAL~

YuJuncen · 2020-07-14T02:52:30Z

/run-all-tests

[...137 lines of PASS]
[2020-07-14T02:46:35.115Z] FAIL	github.com/pingcap/tidb/ddl	17.459s

...Why? 🤔

YuJuncen · 2020-07-14T03:03:34Z

/run-common-test

[2020-07-14T02:55:27.258Z] time="2020-07-14T10:55:27+08:00" level=fatal msg="run test [insert_select] err: sql:insert into t1 values (1,1);: run \"insert into t1 values (1,1);\" at line 126 err Error 1146: Table 'test.t1' doesn't exist"

....

YuJuncen · 2020-07-14T03:10:21Z

/run-common-test

[2020-07-14T03:04:45.669Z] [2020/07/14 11:04:25.160 +08:00] [ERROR] [main.go:215] ["The current temporary storage dir has been occupied by another instance, check tmp-storage-path config and make sure they are different."] [TempStoragePath="/tmp/1000_tidb/MC4wLjAuMDo0MDAxLzAuMC4wLjA6OTA4MQ==/tmp-storage"] [error="fslock: lock is held"] [stack="github.com/pingcap/log.Error\n\t/home/jenkins/agent/workspace/tidb_ghpr_build/go/pkg/mod/github.com/pingcap/log@v0.0.0-20200511115504-543df19646ad/global.go:42\nmain.initializeTempDir\n\t/home/jenkins/agent/workspace/tidb_ghpr_build/go/src/github.com/pingcap/tidb/tidb-server/main.go:215\nmain.main\n\t/home/jenkins/agent/workspace/tidb_ghpr_build/go/src/github.com/pingcap/tidb/tidb-server/main.go:172\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:203"]

We died at different place each time 🙃...

YuJuncen · 2020-07-14T03:36:17Z

/run-unit-test

crazycs520

LGTM

YuJuncen · 2020-07-14T03:54:04Z

/merge

ti-srebot · 2020-07-14T03:54:07Z

@YuJuncen Oops! auto merge is restricted to Committers of the SIG.See the corresponding SIG page for more information. Related SIGs: ddl(slack).

crazycs520 · 2020-07-14T03:58:14Z

/merge

ti-srebot · 2020-07-14T03:58:36Z

/run-all-tests

* domain,session: add NoRegister entry * session: run fmt * session: remove old NoRegister API * config: add no register config * config: add example to example config file. * session: remove unused type * *: rename NoRegister to SkipRegisterToDashboard Co-authored-by: crazycs <crazycs520@gmail.com>

…8587)

Hillium added 3 commits May 26, 2020 15:55

domain,session: add NoRegister entry

1dd3c2c

Merge branch 'master' of https://github.com/pingcap/tidb into no-regi…

1c3280a

…ster

Merge branch 'master' of https://github.com/pingcap/tidb into no-regi…

e3010e8

…ster

session: run fmt

0179b99

github-actions bot added the sig/sql-infra SIG: SQL Infra label May 29, 2020

breezewish reviewed Jun 1, 2020

View reviewed changes

YuJuncen and others added 6 commits June 8, 2020 10:09

Merge branch 'master' into no-register

58977ab

Merge branch 'master' of https://github.com/pingcap/tidb into no-regi…

ee8aa92

…ster

session: remove old NoRegister API

90c3b95

config: add no register config

912ef2a

config: add example to example config file.

8352e75

Merge branch 'master' into no-register

ffc243c

YuJuncen requested review from breezewish and zimulala June 23, 2020 06:08

github-actions bot added the component/config label Jun 23, 2020

zimulala reviewed Jun 23, 2020

View reviewed changes

session/tidb.go Outdated Show resolved Hide resolved

yujuncen added 2 commits June 23, 2020 16:02

session: remove unused type

9c49398

Merge branch 'no-register' of https://github.com/YuJuncen/tidb into n…

48003e7

…o-register

zimulala reviewed Jun 23, 2020

View reviewed changes

config/config.go Outdated Show resolved Hide resolved

yujuncen and others added 2 commits June 24, 2020 10:23

*: rename NoRegister to SkipRegisterToDashboard

cffa03b

Merge branch 'master' into no-register

c8e2d61

Merge branch 'master' into no-register

a9bd073

YuJuncen requested a review from zimulala July 3, 2020 03:11

Merge branch 'master' into no-register

08ea547

zimulala reviewed Jul 7, 2020

View reviewed changes

ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Jul 7, 2020

YuJuncen added 2 commits July 13, 2020 11:49

Merge branch 'master' into no-register

31541d3

Merge branch 'master' into no-register

0f8077b

Merge branch 'master' into no-register

82f3f9d

crazycs520 approved these changes Jul 14, 2020

View reviewed changes

ti-srebot removed the status/LGT1 Indicates that a PR has LGTM 1. label Jul 14, 2020

ti-srebot approved these changes Jul 14, 2020

View reviewed changes

ti-srebot added the status/LGT2 Indicates that a PR has LGTM 2. label Jul 14, 2020

Merge branch 'master' into no-register

bf7f3e4

ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Jul 14, 2020

ti-srebot merged commit 07c6788 into pingcap:master Jul 14, 2020

breezewish mentioned this pull request Jul 14, 2020

BR registers itself as TiDB pingcap/br#299

Closed

YuJuncen mentioned this pull request Jul 15, 2020

session: add no register API for using TiDB as a library #18587

Merged

zimulala pushed a commit that referenced this pull request Jul 15, 2020

session: add no register API for using TiDB as a library (#17513) (#1…

c05ceac

…8587)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

session: add no register API for using TiDB as a library #17513

session: add no register API for using TiDB as a library #17513

YuJuncen commented May 29, 2020 •

edited

Loading

CLAassistant commented May 29, 2020 •

edited

Loading

sre-bot commented May 29, 2020

codecov bot commented May 29, 2020 •

edited

Loading

YuJuncen commented May 29, 2020 •

edited

Loading

bb7133 commented Jun 1, 2020

breezewish Jun 1, 2020

YuJuncen Jun 2, 2020

zimulala Jun 2, 2020

zimulala Jun 23, 2020

YuJuncen commented Jun 30, 2020

YuJuncen commented Jun 30, 2020

zimulala left a comment

ti-srebot commented Jul 7, 2020

YuJuncen commented Jul 8, 2020

YuJuncen commented Jul 14, 2020

YuJuncen commented Jul 14, 2020

YuJuncen commented Jul 14, 2020

YuJuncen commented Jul 14, 2020

crazycs520 left a comment

YuJuncen commented Jul 14, 2020

ti-srebot commented Jul 14, 2020

crazycs520 commented Jul 14, 2020

ti-srebot commented Jul 14, 2020

session: add no register API for using TiDB as a library #17513

session: add no register API for using TiDB as a library #17513

Conversation

YuJuncen commented May 29, 2020 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

More things

Release note

CLAassistant commented May 29, 2020 • edited Loading

sre-bot commented May 29, 2020

codecov bot commented May 29, 2020 • edited Loading

Codecov Report

YuJuncen commented May 29, 2020 • edited Loading

bb7133 commented Jun 1, 2020

breezewish Jun 1, 2020

Choose a reason for hiding this comment

YuJuncen Jun 2, 2020

Choose a reason for hiding this comment

zimulala Jun 2, 2020

Choose a reason for hiding this comment

zimulala Jun 23, 2020

Choose a reason for hiding this comment

YuJuncen commented Jun 30, 2020

YuJuncen commented Jun 30, 2020

zimulala left a comment

Choose a reason for hiding this comment

ti-srebot commented Jul 7, 2020

YuJuncen commented Jul 8, 2020

YuJuncen commented Jul 14, 2020

YuJuncen commented Jul 14, 2020

YuJuncen commented Jul 14, 2020

YuJuncen commented Jul 14, 2020

crazycs520 left a comment

Choose a reason for hiding this comment

YuJuncen commented Jul 14, 2020

ti-srebot commented Jul 14, 2020

crazycs520 commented Jul 14, 2020

ti-srebot commented Jul 14, 2020

YuJuncen commented May 29, 2020 •

edited

Loading

CLAassistant commented May 29, 2020 •

edited

Loading

codecov bot commented May 29, 2020 •

edited

Loading

YuJuncen commented May 29, 2020 •

edited

Loading