Employ etcd for distributed locking

[hackerwins]

What this PR does / why we need it:

Employ etcd for distributed locking

In order to allow multiple agents to process client requests, etcd is
employed. In standalone mode with only one agent, memory-based locker is
used, not etcd.

Which issue(s) this PR fixes:

Address #11

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Additional documentation:

Checklist:

• Added relevant tests or not required
• Didn't break anything

[hackerwins]

Stress test for distributed locking based on etcd

Description of test conditions

• Multiple routines access the same key at the same time.
• etcd is also run on the machine running the test.
• Machine info
  • MacBook Air (Retina, 13-inch, 2020)
  • Processor 1.1 GHz Quad-Core Intel Core i5
  • Memory 16 GB 3733 MHz LPDDR4X

start := time.Now()

size := 500
sum := 0
var wg gosync.WaitGroup
for i := 0; i < size; i++ {
	wg.Add(1)
	go func() {
		defer wg.Done()

		ctx := context.Background()
		locker := cli.NewLocker(ctx, sync.Key(t.Name()))
		locker.Lock()
		sum += 1
		locker.Unlock()
	}()
}
wg.Wait()
assert.Equal(t, size, sum)

log.Logger.Infof("lock count: %d, elapsed: %s", size, time.Since(start))

Results

// Performance of locker.Locker (Memory, moby.locker)
2021-02-12T23:11:44.384+0900	INFO	lock count: 10, elapsed: 177.267µs
2021-02-12T23:11:57.554+0900	INFO	lock count: 100, elapsed: 331.794µs
2021-02-12T23:06:24.930+0900	INFO	lock count: 1000, elapsed: 2.271639ms

// Performance of concurrency.Mutex (Distributed locking, etcd)
2021-02-12T23:11:27.319+0900	INFO	lock count: 10, elapsed: 199.070746ms
2021-02-12T23:10:52.406+0900	INFO	lock count: 100, elapsed: 1.133782094s
2021-02-12T23:14:37.825+0900	INFO	lock count: 200, elapsed: 3.849113763s
2021-02-12T23:15:32.183+0900	INFO	lock count: 500, elapsed: 14.822853146s
2021-02-12T23:01:29.410+0900	INFO	lock count: 1000, elapsed: 48.902339709s

// Performance of concurrency.Mutex (etcd, When accessing different keys) 
2021-02-12T23:21:36.212+0900	INFO	lock count: 500, elapsed: 997.036022ms

[codecov]

Codecov Report

Merging #148 (b7e966e) into main (cf19e73) will decrease coverage by 0.01%.
The diff coverage is 55.55%.

@@            Coverage Diff             @@
##             main     #148      +/-   ##
==========================================
- Coverage   57.05%   57.03%   -0.02%     
==========================================
  Files          33       34       +1     
  Lines        3106     3191      +85     
==========================================
+ Hits         1772     1820      +48     
- Misses       1154     1187      +33     
- Partials      180      184       +4     

Impacted Files	Coverage Δ
pkg/document/json/rga_tree_list.go	53.28% <ø> (ø)
pkg/document/json/rht.go	53.44% <ø> (ø)
pkg/document/json/rht_pq_map.go	59.15% <ø> (ø)
pkg/llrb/llrb.go	81.60% <ø> (ø)
yorkie/backend/db/db.go	66.66% <ø> (ø)
yorkie/config.go	81.25% <ø> (ø)
yorkie/yorkie.go	0.00% <0.00%> (ø)
yorkie/backend/sync/etcd/client.go	34.78% <34.78%> (ø)
yorkie/rpc/server.go	53.49% <66.66%> (+4.50%)	⬆️
api/converter/from_bytes.go	64.42% <100.00%> (ø)
... and 2 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cf19e73...540d2b7. Read the comment docs.

[ppeeou]

LGTM 🔥

[dc7303]

Sorry, I am not familiar with ETCD, so there is a delay in reviewing it. 😭
I will take a closer look and give you an opinion.

[mojosoeun]

@hackerwins Thank you for your PR! Could you elaborate on this PR in the weekly meeting? Because im also not used to etcd, it would be great to hear about this task!

[dc7303]

Thanks for the wonderful work. Thanks to you, I am learning a lot. 🙇‍♂️

I am trying to fully understand this feature and have made a few comments along the way. Please check.

[hackerwins]

Since we are using lease based lock, we need to introduce fencing token.
Lease lock and fencing token are described in the article by Martin Kleppmann.

[dc7303]

I have a few inquiries and suggestions!
Please answer after checking. :)

[dc7303]

LGTM 👍 I learned a lot while reviewing. Thank you! :)