Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement new membership in curp #947

Draft
wants to merge 326 commits into
base: refactor-v0.7.0-new
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
326 commits
Select commit Hold shift + click to select a range
67ccaf8
chore(deps): bump tempfile from 3.10.1 to 3.12.0
dependabot[bot] Sep 9, 2024
4c290bc
refactor: curp client propose
bsbds Aug 31, 2024
3da9ae3
refactor: send both er and asr to avoid undefined behaviour
bsbds Sep 2, 2024
6b29601
test: add sleep between er and asr in tests
bsbds Sep 2, 2024
aa3b568
chore: make the logging message more relevant
bsbds Sep 23, 2024
fd409a5
feat: add auto reconnect implementation for curp client
bsbds Aug 28, 2024
d3ba151
refactor: reimplement curp client state
bsbds Sep 3, 2024
0ed2d5b
refactor: add new to `Config`
bsbds Sep 3, 2024
dae5e11
feat: implement map functions for `ClusterState`
bsbds Sep 3, 2024
699004d
feat: implement `get_quorum` method for `ClusterState`
bsbds Sep 3, 2024
0e67685
chore: rename state.rs to cluster_state.rs
bsbds Sep 3, 2024
9eb11d9
fix: curp client config
bsbds Sep 3, 2024
6485185
feat: reimplement fetch cluster
bsbds Sep 3, 2024
6e70326
chore: move cluster_state to upper level
bsbds Sep 3, 2024
4e8de14
chore: move fetch_impl to upper level
bsbds Sep 3, 2024
ae27b19
chore: move config.rs to upper level
bsbds Sep 3, 2024
1f2b540
refactor: client fetch
bsbds Sep 3, 2024
30c9978
chore: remove mutable methods from ClusterState
bsbds Sep 4, 2024
b185824
refactor: map_leader will always returns a future
bsbds Sep 4, 2024
a41d0c9
refactor: curp client lease keep alive
bsbds Sep 4, 2024
311fed4
refactor: move some mutable state from `Unary` to `Retry`
bsbds Sep 5, 2024
512d878
chore: remove unused `LeaderStateUpdate` trait and impls
bsbds Sep 5, 2024
eb2b13f
refactor: move `ClientApi` implementation to `RepeatableClientApi` fo…
bsbds Sep 5, 2024
5dfbc2e
refactor: use cluster state in context for RepeatableClientApi implem…
bsbds Sep 5, 2024
0c54251
refactor: use cluster_state in Context for propose impl
bsbds Sep 5, 2024
8521862
refactor: remove `State` from `Unary`, use `Context` instead
bsbds Sep 5, 2024
8951af9
refactor: use `Fetch` to implement `fetch_cluster`
bsbds Sep 5, 2024
1acba90
chore: allow unused error handling methods
bsbds Sep 5, 2024
72859ca
chore: remove unused `UnaryConfig`
bsbds Sep 5, 2024
f8bc066
chore: remove unused async and result in `propose_impl`
bsbds Sep 5, 2024
ae366fe
refactor: add override connect to `Fetch`
bsbds Sep 5, 2024
fbe7275
refactor: client build config
bsbds Sep 5, 2024
926ba69
refactor: use an enum for cluster state
bsbds Sep 5, 2024
c60330e
refactor: implement initial cluster state fetch mechanism
bsbds Sep 5, 2024
bd30d1f
chore: rename some types in `cluster_state.rs`
bsbds Sep 5, 2024
03cecec
fix: reconnect keep alive hang
bsbds Sep 5, 2024
9f322df
feat: add `StreamingProtocol` for implement handling bypassed streami…
bsbds Sep 5, 2024
6345c96
fix: timeout in log truncation tests
bsbds Sep 5, 2024
555978e
refactor: updates cluster state on error in keep alive
bsbds Sep 5, 2024
7e340f2
fix: panic when tonic closes channel asynchronously
bsbds Sep 5, 2024
39d48ad
fix: set client inside xline to use raw curp addr
bsbds Sep 5, 2024
7a34aea
refactor: exit keep alive task on cluster shutdown
bsbds Sep 6, 2024
2513da7
fix: `wait_id_update` may lost event
bsbds Sep 6, 2024
6b9a889
chore: remove unused files in client
bsbds Sep 6, 2024
4b11fc4
refactor: pass `ClusterState` to Retry::new
bsbds Sep 6, 2024
0355f3f
test: rewrite curp client tests
bsbds Sep 6, 2024
869f48b
refactor: use external closure to build command for `Fetch`
bsbds Sep 6, 2024
2710a5d
test: update fetch tests
bsbds Sep 6, 2024
932b745
tests: update unary client tests
bsbds Sep 6, 2024
fa1e930
test: fix fetch tests
bsbds Sep 6, 2024
c3eec8d
refactor: client fetch will not retry indefinitely
bsbds Sep 8, 2024
f07d836
test: fix tests
bsbds Sep 8, 2024
4bd61cb
fix: keep alive task not canceled on runtime drop
bsbds Sep 8, 2024
35c5ea0
test: rewrite keep alive tests
bsbds Sep 8, 2024
f41fe98
test: fix madsim tests
bsbds Sep 9, 2024
81a8b4b
chroe: fix typo
bsbds Sep 9, 2024
975927f
chore: move client apis to a seperate submodule
bsbds Sep 10, 2024
4842542
feat: re-add index barrier
bsbds Jul 24, 2024
84a9253
feat: implement client and server member learner rpcs
bsbds Jul 19, 2024
d301c63
feat: implement learner requests on curp server
bsbds Jul 24, 2024
b9a21df
feat(xline-client): implement xline member client for leaner requests
bsbds Jul 29, 2024
b5da08b
feat: implement joint quorum set
bsbds Jul 31, 2024
98e6925
feat(curp/client): implement client propose of new membership change …
bsbds Sep 10, 2024
c33ce2c
feat: implement fetch membership on the server
bsbds Aug 2, 2024
68d3072
feat: implement fetch membership on client
bsbds Sep 10, 2024
f21c8f2
feat(curp/client): implement membership update on curp client
bsbds Sep 10, 2024
18eafbd
refactor(raw_curp): implement appending membership entries on follower
bsbds Aug 5, 2024
baaafc8
refactor(raw_curp): implement append membership entries on leader
bsbds Aug 8, 2024
3e2c740
refactor: add a `in_flight` method for `MembershipState`
bsbds Aug 8, 2024
4496a5b
refactor: implement member add and member remove for `Membership`
bsbds Aug 8, 2024
b272864
feat: implement member add and remove in raw curp
bsbds Aug 9, 2024
613aa85
refactor: rewrite learner add/remove
bsbds Aug 9, 2024
a34b186
refactor: Add a new `NodeMembershipState` in RawCurp
bsbds Aug 9, 2024
c2839af
chore: add member add/remove to auth wrapper
bsbds Aug 9, 2024
76e9a7d
refactor: curp membership startup
bsbds Aug 12, 2024
3d64afa
feat: add membership change config
bsbds Aug 12, 2024
36e80b0
feat: implement server member rpc connect
bsbds Aug 13, 2024
28cd734
feat: implement server rpc connect update
bsbds Aug 13, 2024
803ee73
feat(raw_curp): implement sync node task for new membership
bsbds Aug 14, 2024
9a98822
refactor: build sync_events using membership_info
bsbds Aug 14, 2024
bacd0a3
feat: implement `voter_connects`
bsbds Aug 14, 2024
a6c8848
refactor: remove the `Config` type in vote
bsbds Aug 14, 2024
d4554d7
refactor: implement vote result check in new membership change
bsbds Aug 14, 2024
2d64905
refactor: sp recovery in new membership change
bsbds Aug 14, 2024
da62cb3
refactor: log commit condition in new membership change
bsbds Aug 14, 2024
c2b45ee
refactor: init leader state using membership info
bsbds Aug 15, 2024
9abe8a5
refactor: remove old membership change implementation
bsbds Aug 15, 2024
76b201c
feat: add/remove member in ConnectApi
bsbds Sep 10, 2024
06155e9
feat: fixup curp client fetch implementation
bsbds Sep 11, 2024
c476f46
chore: move retry/mod.rs to retry.rs
bsbds Sep 11, 2024
311cc81
refactor: update cluster state
bsbds Sep 11, 2024
bee42d1
refactor: remove `discover_from` in curp client
bsbds Sep 11, 2024
17e7663
refactor: remove fetch cluster
bsbds Sep 11, 2024
a7201a9
feat: Add cluster id generation for Membership
bsbds Sep 12, 2024
79fa759
refactor: remove ClusterInfo and related code
bsbds Sep 12, 2024
326930f
feat: implement membership persistent
bsbds Sep 12, 2024
cca809a
feat: persist membership in raw_curp
bsbds Sep 12, 2024
fbc7918
refactor: sync follower task
bsbds Sep 12, 2024
f8670c0
refactor: removes publish rpc
bsbds Sep 13, 2024
b065c1f
chore: remove unused leader_tx broadcast channel
bsbds Sep 13, 2024
3677d86
refactor: remove `connects` from `RawCurp`
bsbds Sep 13, 2024
f77e55e
refactor: rewrite FollowerStatus
bsbds Sep 13, 2024
055ba6f
refactor: merge all states of nodes
bsbds Sep 14, 2024
7250786
refactor: fix xline servers
bsbds Sep 14, 2024
f528c97
2
bsbds Sep 14, 2024
9290470
refactor: remove unused code in members.rs
bsbds Sep 14, 2024
8b62aa8
chore: fix curp group
bsbds Sep 18, 2024
6dfc21d
refactor: add `NodeMetadata` to membership config
bsbds Sep 18, 2024
9f8af84
refactor: implement `parse_membership`
bsbds Sep 18, 2024
ce9b9d4
refactor: api of NodeMetadata::new
bsbds Sep 19, 2024
c20290b
fix: prevent deadlock while holding `RawCurp.st`
bsbds Sep 19, 2024
6e9d12d
fix: use dummy id for initial cluster fetch
bsbds Sep 19, 2024
8cedd5d
fix: initial membership state
bsbds Sep 19, 2024
7603cee
fix: replace `HashMap` with `BTreeMap` in `CurpGroup`
bsbds Sep 19, 2024
4f99db7
fix: membership calculation
bsbds Sep 19, 2024
787609b
fix: node role update
bsbds Sep 19, 2024
c8753eb
fix: ensures leader in membership request handling
bsbds Sep 20, 2024
8766fdd
chore: remove unused field in `ClientBuilder`
bsbds Sep 20, 2024
b23cb2d
refactor: allow setting full cluster metadata in `ClientBuilder`
bsbds Sep 20, 2024
3259609
test: rewrite membership change related tests
bsbds Sep 19, 2024
729d3b5
fix: leader will reset transferee after the transferee has been removed
bsbds Sep 20, 2024
1d2f731
fix: always commit membership index in `append_membership`
bsbds Sep 20, 2024
cf4bc70
fix: behavior of `Membership::contains`
bsbds Sep 20, 2024
e26f850
test: rewrite membership change tests in `raw_curp`
bsbds Sep 20, 2024
148ab14
refactor: implement `ClusterServer`
bsbds Sep 20, 2024
0cc4b90
refactor: move node id speficication to the client side
bsbds Sep 23, 2024
32fa899
refactor: implement member update in `ClusterServer`
bsbds Sep 23, 2024
ac28e95
refactor: merge membership change rpcs into one
bsbds Sep 24, 2024
7e745c4
refactor: remove notification on `NodeState` drop
bsbds Sep 24, 2024
25d8fc8
refactor: move sync follower task spawning to CurpNode
bsbds Sep 24, 2024
683cc01
refactor: refine curp membership handling code
bsbds Sep 25, 2024
b3ff465
test: update test code
bsbds Sep 26, 2024
92bea76
WIP: refactor membership change implementation
bsbds Sep 26, 2024
dff2a7a
refactor: reimplement `MembershipState`
bsbds Sep 26, 2024
cef1c5a
refactor: membership impl
bsbds Sep 27, 2024
80c43f3
refactor: log update code
bsbds Sep 27, 2024
3f658b0
chore: split leader and follower membership impls in `RawCurp`
bsbds Sep 27, 2024
2155427
refactor: prevent multiple concurrent membership changes
bsbds Sep 27, 2024
dcf68ec
refactor: remove outdated membership state entries on commit
bsbds Sep 27, 2024
4b447b6
fix: MembershipState
bsbds Sep 29, 2024
f84ac70
fix: Membership
bsbds Sep 29, 2024
0f30956
fix: membership changes calculation
bsbds Sep 29, 2024
d3491b5
refactor: use external committed log index state in `MembershipState`
bsbds Sep 29, 2024
8ade71c
fix: use Option<LogIndex> for `truncate_at` instead of using
bsbds Sep 29, 2024
6dc3954
refactor: MembershipState::commit
bsbds Sep 29, 2024
07ae7ec
refactor: merge membership state update methods
bsbds Sep 30, 2024
95d68be
fix: connect nodes should exclude self
bsbds Sep 30, 2024
3acbf22
refactor: Fetch implementation
bsbds Sep 30, 2024
87bd37b
refactor: fetch cluster
bsbds Sep 30, 2024
6efdd5c
refactor: check cluster version during propose
bsbds Oct 8, 2024
df73b78
fix: client record conflict
bsbds Oct 8, 2024
8592d84
fix: client fetch membership should fetch largest term
bsbds Oct 8, 2024
5401535
refactor: implement lazy connect in rpc::connect
bsbds Oct 8, 2024
d4b53fc
test: return fetch membership response in MockStreamConnectApi
bsbds Oct 8, 2024
4306127
test: commit log index in membership tests
bsbds Oct 8, 2024
c9a4065
test: do not update follower state in leader_handle_move_leader
bsbds Oct 8, 2024
17ad1e2
refactor:update client membership state when client fetch membership
bsbds Oct 8, 2024
acb5a07
test: do not wait for node shutdown in membership change
bsbds Oct 8, 2024
49a830e
test: fix remove leader will invalidate client id
bsbds Oct 8, 2024
1aed29c
fix: spawn sync follower task should exclude self
bsbds Oct 8, 2024
6855ed6
test: fix toml config
bsbds Oct 8, 2024
5ec4055
test: fix duplicate node id
bsbds Oct 8, 2024
31a3342
fix: exclude self connection when broadcasting vote
bsbds Oct 8, 2024
8895005
fix: add a workaround when removing the leader in cluster server
bsbds Oct 8, 2024
5050be4
fix: use client urls in lease server redirection
bsbds Oct 8, 2024
f900032
test: fix test_status
bsbds Oct 8, 2024
74c5905
test: add tests for Joint
bsbds Oct 10, 2024
24209e1
test: add tests for QuorumSet
bsbds Oct 10, 2024
4f18ba1
test: add tests for membership
bsbds Oct 10, 2024
b6cb60a
test: add tests for fetch
bsbds Oct 10, 2024
629c512
chore: rename ClusterStateReady to ClusterStateFull
bsbds Oct 11, 2024
4ca4057
test: add tests for ClusterState
bsbds Oct 11, 2024
535344c
test: add tests for KeepAlive
bsbds Oct 11, 2024
4d33f77
test: add tests for Retry
bsbds Oct 11, 2024
6626072
test: add tests for NodeStates
bsbds Oct 11, 2024
0b95dc2
test: add tests for membership persistent
bsbds Oct 11, 2024
c5ea1ba
refactor: implement membership recovery
bsbds Oct 11, 2024
ac5fdc7
test: add tests for raw_curp::member_impl
bsbds Oct 11, 2024
3c3357b
test: add tests for CurpNode
bsbds Oct 11, 2024
e5e87b5
test: add membership change curp integration tests
bsbds Oct 14, 2024
0ecdda7
test: add membership change xline integration tests
bsbds Oct 14, 2024
10f7a86
feat: implement wait learner on curp node
bsbds Oct 12, 2024
7f2c6b1
feat: implement wait learner on curp client
bsbds Oct 14, 2024
130f14a
refactor: reimplement curp follower replication
bsbds Oct 16, 2024
e1287be
refactor: remove old replication code
bsbds Oct 17, 2024
137a19f
refactor: enable new replication implementation
bsbds Oct 17, 2024
24fbbca
chore: move replication state update logic to `RawCurp`
bsbds Oct 17, 2024
5e535df
test: rewrite tests for log replication
bsbds Oct 17, 2024
c8014a6
chore: do not convert tonic::Status to CurpError in replication send …
bsbds Oct 17, 2024
f9a53ce
fix: do not use InnerConnectApi::id because it's deprecated
bsbds Oct 17, 2024
cc9adac
fix: leader should abort replication on step down
bsbds Oct 17, 2024
994bf3e
fix: update role using latest membership
bsbds Oct 17, 2024
467f9e2
refactor: client retry
bsbds Oct 17, 2024
6ba65be
fix
bsbds Oct 18, 2024
2debf5c
fix: do not abort unary request when client id expires
bsbds Oct 18, 2024
5aafbb3
fix: pass leader_id and term to build_membership_response
bsbds Oct 18, 2024
bb92926
fix: only append to MembershipState when the index is greater
bsbds Oct 18, 2024
ec059be
refactor: remove self_next_index
bsbds Oct 18, 2024
87b8719
fix: update next index to last_log_index instead
bsbds Oct 18, 2024
6ac1d46
fix: prevent fetch membership from learner
bsbds Oct 18, 2024
79ac278
fix: add_node_should_add_new_node_to_curp should truncate index
bsbds Oct 18, 2024
05e7eda
fix: try_become_leader_now should also call respawn_replication
bsbds Oct 18, 2024
391efca
fix: state_machine_worker should use cached term instead
bsbds Oct 18, 2024
530fa9f
fix: Action::UpdateMatchIndex should also update next_index
bsbds Oct 18, 2024
2fd19cd
refactor: remove SyncFollower from task manager
bsbds Oct 18, 2024
79892af
chore: remove logging Action::GetLogFrom
bsbds Oct 18, 2024
0fb6aab
refactor: heartbeat will also send commit index to calibrate nodes
bsbds Oct 21, 2024
98384c8
feat: implement wait shutdown in CurpNode
bsbds Oct 21, 2024
08cdb21
refactor: seperate handle append entries and handle heartbeat on foll…
bsbds Oct 21, 2024
70ff862
test: refactor test shutdown_rpc_should_shutdown_the_cluster
bsbds Oct 21, 2024
bb028cc
fix: update_states_with_membership will always connect to all nodes f…
bsbds Oct 21, 2024
9191651
test: do not reconstruct MemebershipInfo in curp_group
bsbds Oct 21, 2024
5df50f1
fix: mark election task as cancel safe
bsbds Oct 21, 2024
b3e434a
chore: fix clippy build_membership_response
bsbds Oct 21, 2024
130f885
fix: madsim tests
bsbds Oct 22, 2024
72880a7
fix: merge vote and pre-vote failure result
bsbds Oct 23, 2024
53d8403
test: add membership madsim tests
bsbds Oct 23, 2024
af583a9
test: fix madsim tests
bsbds Oct 24, 2024
138fe34
fix: membership not truncate on append entries
bsbds Oct 24, 2024
bf35fe8
refactor: merge update_membership_state and persistent_membership_state
bsbds Oct 24, 2024
bf8cfbf
test: add membership idempotent test
bsbds Oct 24, 2024
8ccd5c5
refactor: add cluster version to `ChangeMembershipRequest`
bsbds Oct 28, 2024
e1510b8
refactor: do not send membership request if changes are already applied
bsbds Oct 28, 2024
6255664
refactor: split curp client API
bsbds Nov 3, 2024
74ef8b5
refactor: remove curp client keep alive
bsbds Nov 3, 2024
98161f9
refactor: remove curp client cmd id tracker
bsbds Nov 3, 2024
4c71387
refactor: remove curp server duplication check
bsbds Nov 3, 2024
d0a20d8
refactor: remove curp server er and asr buffer
bsbds Nov 4, 2024
ff3c8b8
refactor: remove curp server gc task
bsbds Nov 4, 2024
e7b71ac
refactor: remove curp server client lease
bsbds Nov 4, 2024
faf81e5
refactor: refine cmd worker implementation
bsbds Nov 4, 2024
098c15e
refactor: remove unused rpc implementations
bsbds Nov 4, 2024
ad251a2
test: fix tests
bsbds Nov 4, 2024
d9d36b6
test: fix membership test
bsbds Nov 4, 2024
f53080a
feat: add SyncSpecPool rpc definition
bsbds Nov 5, 2024
76272a9
refactor: spec pool get all entry implementation
bsbds Nov 5, 2024
454fa2b
feat: implement spec pool sync
bsbds Nov 5, 2024
784c506
test: add speculative pool gc tests
bsbds Nov 6, 2024
9d1b9fd
chore: update curp proto
bsbds Nov 7, 2024
9865e06
refactor: use sha256 for membership version generation
bsbds Nov 7, 2024
e7a035d
chore: refine retry code
bsbds Nov 10, 2024
349b81e
refactor: implement speculative pool gc
bsbds Nov 11, 2024
b3866e2
refactor: for each follower implementation
bsbds Nov 12, 2024
f3df695
test: add sp version tests
bsbds Nov 12, 2024
617ba12
chore: remove unused sync spec pool RPC
bsbds Nov 12, 2024
8c797d6
chore: merge gc and update_version
bsbds Nov 12, 2024
1d2340c
refactor: persistent sp version
bsbds Nov 12, 2024
5943fc1
test: add sp gc tests for curp server
bsbds Nov 12, 2024
473ca27
fix: allow gc with the same version(by ignoring)
bsbds Nov 12, 2024
9ea6b0b
test: add sp gc tests for curp server
bsbds Nov 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/ISSUE_TEMPLATE/bug_report.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ body:
- 0.4.1
- 0.5.0
- 0.6.0
- 0.6.1 (Default)
- 0.6.1
- 0.7.0 (Default)
validations:
required: true
- type: textarea
Expand Down
8 changes: 2 additions & 6 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,8 @@ jobs:
with:
submodules: recursive

- name: Configure sccache
uses: actions/github-script@v7
with:
script: |
core.exportVariable('ACTIONS_CACHE_URL', process.env.ACTIONS_CACHE_URL || '');
core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || '');
- name: Setup sccache-cache
uses: mozilla-actions/sccache-action@v0.0.5

call_build_xline:
name: Build and Upload Artifacts
Expand Down
6 changes: 5 additions & 1 deletion .github/workflows/build_env.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
name: Build CI Env Image

on:
workflow_dispatch: {}
push:
paths:
- "ci/build-env.sh"
- "ci/Dockerfile"
workflow_dispatch:

jobs:
build_env:
Expand Down
9 changes: 3 additions & 6 deletions .github/workflows/build_xline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,12 +39,9 @@ jobs:
with:
submodules: recursive

- name: Configure sccache
uses: actions/github-script@v7
with:
script: |
core.exportVariable('ACTIONS_CACHE_URL', process.env.ACTIONS_CACHE_URL || '');
core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || '');
- name: Setup sccache-cache
uses: mozilla-actions/sccache-action@v0.0.5

- name: Prepare release binaries
id: prepare_binaries
run: |
Expand Down
52 changes: 32 additions & 20 deletions .github/workflows/pull_request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,15 @@ jobs:
run:
shell: bash
env:
SCCACHE_GHA_ENABLED: "on"
container: ghcr.io/xline-kv/build-env:latest
SCCACHE_GHA_ENABLED: "true"
CARGO_INCREMENTAL: 0 # CI will compile all crates from beginning. So disable incremental compile may reduce compile target size.
container:
image: ghcr.io/xline-kv/build-env:latest
volumes:
- /usr/local/lib/android/:/tmp/android/
- /usr/share/dotnet:/tmp/dotnet
- /opt/ghc:/tmp/ghc
- /usr/lib/firefox:/tmp/firefox
strategy:
fail-fast: true
matrix:
Expand All @@ -24,26 +31,35 @@ jobs:
name: "Normal",
args: "",
rustflags: "",
test: "llvm-cov nextest --all-features --workspace --codecov --output-path codecov.info",
test: "llvm-cov nextest --all-features --workspace --codecov --output-path codecov.info && cargo test --doc",
}
- {
name: "Madsim",
args: "--package=simulation",
rustflags: "--cfg madsim",
test: "nextest run --package=simulation",
test: "nextest run --package=simulation && cargo test -p simulation --doc",
}
name: Tests ${{ matrix.config.name }}
steps:
- name: View free disk space
run: df -h /

- name: Setup sccache-cache
uses: mozilla-actions/sccache-action@v0.0.5

- uses: actions/checkout@v4
with:
submodules: recursive

- name: Configure sccache
uses: actions/github-script@v7
with:
script: |
core.exportVariable('ACTIONS_CACHE_URL', process.env.ACTIONS_CACHE_URL || '');
core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || '');
- name: Free Disk Space
run: |
rm -rf /tmp/android/* || true
rm -rf /tmp/dotnet/* || true
rm -rf /tmp/ghc/* || true
rm -rf /tmp/firefox/* || true

- name: View free disk space
run: df -h /

- name: Trailing spaces check
run: ci/scripts/check-trailing-spaces.sh
Expand All @@ -63,23 +79,23 @@ jobs:
- name: Workspace hack check
run: cargo hakari generate --diff && cargo hakari manage-deps --dry-run && cargo hakari verify

- run: sccache --zero-stats > /dev/null
- run: ${SCCACHE_PATH} --zero-stats > /dev/null

- name: Clippy ${{ matrix.config.name }}
env:
RUSTFLAGS: ${{ matrix.config.rustflags }}
run: cargo clippy ${{ matrix.config.args }} --all-targets --all-features -- -D warnings

- name: Sccache stats ${{ matrix.config.name }}
run: sccache --show-stats && sccache --zero-stats > /dev/null
run: ${SCCACHE_PATH} --show-stats && ${SCCACHE_PATH} --zero-stats > /dev/null

- name: Test ${{ matrix.config.name }}
env:
RUSTFLAGS: ${{ matrix.config.rustflags }}
run: cargo ${{ matrix.config.test }}

- name: Sccache stats ${{ matrix.config.name }}
run: sccache --show-stats
run: ${SCCACHE_PATH} --show-stats

- name: Upload coverage to Codecov
if: matrix.config.name == 'Normal'
Expand Down Expand Up @@ -112,7 +128,7 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Check Spelling
uses: crate-ci/typos@v1.23.3
uses: crate-ci/typos@v1.24.5

build:
name: Build
Expand All @@ -122,12 +138,8 @@ jobs:
with:
submodules: recursive

- name: Configure sccache
uses: actions/github-script@v7
with:
script: |
core.exportVariable('ACTIONS_CACHE_URL', process.env.ACTIONS_CACHE_URL || '');
core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || '');
- name: Setup sccache-cache
uses: mozilla-actions/sccache-action@v0.0.5

- name: Build xline image
run: |
Expand Down
14 changes: 5 additions & 9 deletions .github/workflows/validation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,25 +16,21 @@ jobs:
with:
submodules: recursive

- name: Configure sccache
uses: actions/github-script@v7
with:
script: |
core.exportVariable('ACTIONS_CACHE_URL', process.env.ACTIONS_CACHE_URL || '');
core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || '');
- name: Setup sccache-cache
uses: mozilla-actions/sccache-action@v0.0.5

call_build_xline:
name: Build and Upload Artifacts
uses: ./.github/workflows/build_xline.yml
with:
docker_xline_image: 'ghcr.io/xline-kv/build-env:latest'
docker_xline_image: "ghcr.io/xline-kv/build-env:latest"
additional_setup_commands: |
sudo apt-get install -y --force-yes expect
ldd ./xline
ldd ./benchmark
cp ../fixtures/{private,public}.pem .
docker build . -t ghcr.io/xline-kv/xline:latest
docker pull gcr.io/etcd-development/etcd:v3.5.5
binaries: 'xline,benchmark'
script_name: 'validation_test.sh'
binaries: "xline,benchmark"
script_name: "validation_test.sh"
uploadLogs: true
Loading