Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Socket overhaul #128

Open
wants to merge 76 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 67 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
213caa3
Changed socket type to Dealer, and added outline for creating message…
mzhong1 Jan 22, 2020
d4dfb92
implement create function for message request map
mzhong1 Jan 22, 2020
56371f8
Add operation to message map
mzhong1 Jan 23, 2020
7bde826
get an operation from the map
mzhong1 Jan 23, 2020
c0f9dc5
Remove an op from the message map
mzhong1 Jan 24, 2020
9957a99
Update the add_map_op function to update the map if it already has an…
mzhong1 Jan 24, 2020
82bd8f3
Get a specific disk hashmap
mzhong1 Jan 24, 2020
5336b74
Added channels and edited disk-manager's add_disk and listen function…
mzhong1 Jan 27, 2020
50a46a3
Changed disk-manager handler for all functions to use threadpool/DEAL…
mzhong1 Jan 27, 2020
72f3456
Swapped DEALER with STREAM in disk-manager
mzhong1 Jan 29, 2020
9d4b09d
Added handling of sending messages from Bynar to disk-manager
mzhong1 Jan 30, 2020
6fd3fb8
Added macros, handling add_disk
mzhong1 Jan 30, 2020
4767549
Fix a little client stuff
mzhong1 Jan 30, 2020
c3a59d6
Some client fixing + macros
mzhong1 Jan 31, 2020
02a9ac4
Updated Zmq version
mzhong1 Jan 31, 2020
770147f
fix notify?
mzhong1 Jan 31, 2020
bf586c5
Filter states in returned state machines and clippy fixes
mzhong1 Feb 3, 2020
f444936
Filter disks for WaitingForReplacement and not in-progress
mzhong1 Feb 3, 2020
479ade6
Switched to STREAM/STREAM system
mzhong1 Feb 5, 2020
5e50318
Fix bynar-client to use new socket system
mzhong1 Feb 5, 2020
51e6041
parse_from_bytes grabs messages from the end of the byte vector given…
mzhong1 Feb 6, 2020
08bce43
change remove/safeto-remove to be stricter with config/non-lvm bluest…
mzhong1 Feb 6, 2020
6e7b3a5
Fix macro use for parsing opresults
mzhong1 Feb 6, 2020
3528e8c
filter state machines for the disks that need replacement + add in th…
mzhong1 Feb 7, 2020
91ec3b8
Push SafeToRemove + Remove to queue
mzhong1 Feb 7, 2020
c9936c2
Error's cannot be cloned, so change check_for_failed_disk to error ou…
mzhong1 Feb 7, 2020
397e153
handle add and remove return values
mzhong1 Feb 7, 2020
04f690f
Add new message handler to main function
mzhong1 Feb 10, 2020
85d3d61
Add functions to create a req_map for disk-manager + check if ops are…
mzhong1 Feb 10, 2020
96ce4b0
Added request map to disk-manager, skip repeat requests
mzhong1 Feb 10, 2020
eacbe7b
Add behavior to output correct message if skipping an operation becau…
mzhong1 Feb 10, 2020
6737dde
fix the return values to not error incorrectly
mzhong1 Feb 12, 2020
7c5f52a
Fix Error handling for bynar-client'
mzhong1 Feb 12, 2020
b2a9a96
added checks before removing partitions/unmounting partitions
mzhong1 Feb 12, 2020
1d332c5
SafeToRemove returning error is expected, disk path might not be remo…
mzhong1 Feb 12, 2020
7f6efe2
Handle looping and fix error out, Change handle all_finished to check…
mzhong1 Feb 17, 2020
b345ff6
Add the Successful call but SafeToRemove is false to all_finished check
mzhong1 Feb 17, 2020
2587893
Test without ticket erroring out
mzhong1 Feb 18, 2020
cb6459f
error out on ticket creation failure
mzhong1 Feb 18, 2020
780fbdc
Working on Unit tests, added condition to state machine so itonly ski…
mzhong1 Feb 19, 2020
6b71682
Added unit testing for filters and creating the request map
mzhong1 Feb 19, 2020
b1109b0
Unit tests for create_msg_map, creation without partitions, with part…
mzhong1 Feb 19, 2020
dfc2111
Rearranged the code in add_or_update_op for smaller function + easier…
mzhong1 Feb 20, 2020
e030a1c
Moved repeat code into seperate key request function, squished down r…
mzhong1 Feb 20, 2020
2b1621d
Add unit tests for the other req map functions
mzhong1 Feb 20, 2020
6947fda
check filtering waiting for replacement disks from state machine output
mzhong1 Feb 21, 2020
0dad151
Add unit tests for getting the right partitions/disks associated with…
mzhong1 Feb 21, 2020
07240d8
added test for removing duplicates/sorting and reformatted files to d…
mzhong1 Feb 24, 2020
6036e1a
squish tests down
mzhong1 Feb 24, 2020
0d2c817
Fix all finish check for SafeToRemove handling and add unit testing
mzhong1 Feb 25, 2020
a7e03d3
use new function get_request_keys instead
mzhong1 Feb 25, 2020
55d2107
Clippy and code squishing
mzhong1 Feb 26, 2020
2facc6f
pull out repeat behavior into smaller functions for reduced complexity
mzhong1 Feb 26, 2020
53df069
Removed repeat code and replaced with function call
mzhong1 Feb 26, 2020
c80adb2
pulled out repeat behavior into a macro
mzhong1 Feb 28, 2020
cc798ff
clean up some redundant code
mzhong1 Feb 28, 2020
3e384cb
Fixed up returning an error instead of ok, and revamped the listen fu…
mzhong1 Feb 28, 2020
f54c709
Fix issue with is_all_finished not correctly determining when all ope…
mzhong1 Mar 2, 2020
5070465
Update toml's GPT crate
mzhong1 Mar 3, 2020
4702c9a
remove unused disk-manager code and update backend
mzhong1 Mar 10, 2020
bbd763f
add comment, small loop sleep, and CentOS package dependencies
mzhong1 Mar 31, 2020
d383c9b
move the smartctl enable as a soft error, since smartctl -H runs rega…
mzhong1 Mar 31, 2020
103e7bf
Fix checking if disk operation already done to account for bluestore …
mzhong1 Apr 1, 2020
50ed446
Add check in osd metadata for non-lvm bluestore osds, remove SkipRepe…
mzhong1 Apr 2, 2020
e13a72c
update TOML to use correct block-utils version and vault crates
mzhong1 Apr 13, 2020
eae5e98
update goji and arrange function to prevent opening Client without use
mzhong1 Apr 20, 2020
4d8be60
Sleep the poll_events so it doesn't spin
mzhong1 Apr 20, 2020
9bfc05a
fix the skip signal handling issue and add sleep to loop
mzhong1 Apr 22, 2020
8413563
remove extraneous sleep
mzhong1 Apr 23, 2020
0d6dc3f
add explicit sleep to macro for polling events
mzhong1 Apr 23, 2020
a43e444
Fix the fd leak
mzhong1 May 4, 2020
af492ac
Update libatasmart
mzhong1 May 7, 2020
73b2ea6
update ceph crates
mzhong1 May 7, 2020
fa685d2
Fix soft error printing so common errors are changed to debug to prev…
mzhong1 May 14, 2020
82e5276
fix JIRA crate version
mzhong1 Jun 12, 2020
7477053
fix remove operation on successful Add so it removes all operations o…
mzhong1 Jun 12, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
666 changes: 335 additions & 331 deletions Cargo.lock

Large diffs are not rendered by default.

11 changes: 6 additions & 5 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "bynar"
version = "0.1.8"
version = "0.2.0"
authors = ["Chris Holcombe <xfactor973@gmail.com>"]
description = "Server remediation as a service"
license = "Apache-2.0"
Expand Down Expand Up @@ -45,14 +45,15 @@ bytes = "*"
ceph = {git = "https://github.com/mzhong1/ceph-rust"}#"~3.0"
chrono = "~0.4"
clap = "~2"
crossbeam = "~0.7"
daemonize = "~0.4"
derive-error = "0.0.4"
dirs = "~2.0"
dmi = {git = "https://github.com/cholcombe973/dmi"}
fstab = "~0.3"
gluster = "~1.0"
goji = { git = "https://github.com/cholcombe973/goji.git" }
gpt = { git = "https://github.com/mzhong1/gpt"}
goji = { git = "https://github.com/cholcombe973/goji.git", rev = "96aef5ea21d8b5ec3d83209de708fa2e13cd5c96" }
gpt = { git = "https://github.com/Quyzi/gpt"}
hashicorp_vault = { git = "https://github.com/cholcombe973/vault-rs.git" }
hostname = "~0.1"
init-daemon = "~0.1"
Expand All @@ -72,7 +73,7 @@ pwd = "~1.3"
r2d2 = "~0.8"
r2d2_postgres = "~0.14"
rayon = "~1.0"
reqwest = "~0.9"
reqwest = {version ="~0.9", features =["native-tls"]}
serde = "~1"
serde_derive = "~1"
serde_json = "~1"
Expand All @@ -83,7 +84,7 @@ tempdir = "~0.3"
time = "~0.1"
uname = "~0.1"
uuid = { version="~0.7", features = ["serde", "v4"]}
zmq = {version="~0.8"}
zmq = {version="~0.9"}



Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,10 @@ CLI command to install all the dependencies:
sudo apt install libzmq3-dev libprotobuf-dev librados2 libatasmart-dev libssl-dev libblkid-dev libudev-dev librados-dev pkg-config libclang-dev llvm libdevmapper-dev liblvm2-dev liblvm2app2.2 gcc clang smartmontools parted
```

CentOS7
```
yum install -y zeromq-devel protobuf-devel librados2 libatasmart-devel openssl-devel libblkid-devel libudev-devel librados2-devel pkgconfig clang-devel zeromq llvm device-mapper-devel gcc clang smartmontools parted
```

### Working Rust environment

Expand Down
8 changes: 8 additions & 0 deletions api/protos/service.proto
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,8 @@ message OpBoolResult {
optional bool value = 2;
// error_msg is set if ERR
optional string error_msg = 3;
// the type of the operation
required Op op_type = 4;
}

message OpStringResult {
Expand All @@ -88,6 +90,8 @@ message OpStringResult {
optional string value = 2;
// error_msg is set if ERR
optional string error_msg = 3;
// the type of the operation
required Op op_type = 4;
}

message OpOutcomeResult{
Expand All @@ -98,6 +102,10 @@ message OpOutcomeResult{
optional bool value = 3;
// error_msg set if ERR
optional string error_msg = 4;
// the type of the operation
required Op op_type = 5;
// the disk/partition the operation was performed on
optional string disk = 6;
}

enum OpOutcome {
Expand Down
Loading