Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(server): Use global config from file if provided #2458

Merged
merged 41 commits into from
Sep 12, 2023
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
dbc71f3
wip
TBS1996 Sep 3, 2023
a320a33
Merge branch 'master' into tor/static_config
TBS1996 Sep 3, 2023
dc8fe7a
wip
TBS1996 Sep 3, 2023
d631912
wip
TBS1996 Sep 4, 2023
c125ec5
wip
TBS1996 Sep 4, 2023
aed5e6e
wip
TBS1996 Sep 4, 2023
153c9f1
wip
TBS1996 Sep 4, 2023
9367f89
take relay modes into account
TBS1996 Sep 5, 2023
9068be4
merge
TBS1996 Sep 5, 2023
1a43645
Merge branch 'master' into tor/static_config
TBS1996 Sep 5, 2023
3776ec6
wip
TBS1996 Sep 5, 2023
2d56213
Merge branch 'tor/static_config' of https://github.com/getsentry/rela…
TBS1996 Sep 5, 2023
761198e
simplify match arms
TBS1996 Sep 6, 2023
c3221b7
move match to spawnhandler
TBS1996 Sep 6, 2023
8fa4521
ref
TBS1996 Sep 6, 2023
387c246
nit
TBS1996 Sep 6, 2023
0fadcea
self request
TBS1996 Sep 6, 2023
749306a
Merge branch 'master' into tor/static_config
TBS1996 Sep 6, 2023
5281b67
ref match
TBS1996 Sep 6, 2023
68b0bbf
move static config load to service
TBS1996 Sep 6, 2023
383d850
Merge branch 'tor/static_config' of https://github.com/getsentry/rela…
TBS1996 Sep 6, 2023
67d3f73
revert sleephandle logic
TBS1996 Sep 6, 2023
43e4b4c
Merge branch 'master' into tor/static_config
TBS1996 Sep 6, 2023
e9b7272
Merge branch 'tor/static_config' of https://github.com/getsentry/rela…
TBS1996 Sep 6, 2023
5e94b76
move globalconfig load
TBS1996 Sep 7, 2023
9e0f3e3
Merge branch 'master' into tor/static_config
TBS1996 Sep 7, 2023
4f87968
add tests
iker-barriocanal Sep 7, 2023
da63e18
undo rename to prevent bad error grouping
iker-barriocanal Sep 7, 2023
a5e9c03
Merge branch 'master' into tor/static_config
iker-barriocanal Sep 7, 2023
a7030d4
update docstrings
iker-barriocanal Sep 7, 2023
a5c0224
update changelog
TBS1996 Sep 11, 2023
a6efd91
Merge branch 'master' into tor/static_config
TBS1996 Sep 11, 2023
99f2acd
address some feedback
iker-barriocanal Sep 11, 2023
20a593e
wip
TBS1996 Sep 12, 2023
17eba98
Merge branch 'tor/static_config' of https://github.com/getsentry/rela…
TBS1996 Sep 12, 2023
a6ada9f
wip
TBS1996 Sep 12, 2023
a69eb2f
wip
TBS1996 Sep 12, 2023
a4f5a14
t->T
TBS1996 Sep 12, 2023
c0e767d
remove test feat/dep
TBS1996 Sep 12, 2023
c9f0ef9
Merge branch 'master' into tor/static_config
TBS1996 Sep 12, 2023
c39f12c
Merge branch 'master' into tor/static_config
TBS1996 Sep 12, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

**Features**:

- Use static global configuration if file is provided. ([#2458](https://github.com/getsentry/relay/pull/2458))
- Add `view_names` to `AppContext` ([#2344](https://github.com/getsentry/relay/pull/2344))
- Tag keys in error events and transaction events can now be up to `200` ASCII characters long. Before, tag keys were limited to 32 characters. ([#2453](https://github.com/getsentry/relay/pull/2453))

Expand Down
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions relay-config/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ num_cpus = "1.13.0"
relay-auth = { path = "../relay-auth" }
relay-common = { path = "../relay-common" }
relay-kafka = { path = "../relay-kafka" }
relay-dynamic-config = { path = "../relay-dynamic-config" }
relay-log = { path = "../relay-log", features = ["init"] }
relay-metrics = { path = "../relay-metrics" }
relay-redis = { path = "../relay-redis" }
Expand Down
11 changes: 11 additions & 0 deletions relay-config/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,14 @@ use std::io::Write;
use std::net::{IpAddr, SocketAddr, ToSocketAddrs};
use std::path::{Path, PathBuf};
use std::str::FromStr;
use std::sync::Arc;
use std::time::Duration;
use std::{env, fmt, fs, io};

use anyhow::Context;
use relay_auth::{generate_key_pair, generate_relay_id, PublicKey, RelayId, SecretKey};
use relay_common::Dsn;
use relay_dynamic_config::GlobalConfig;
use relay_kafka::{
ConfigError as KafkaConfigError, KafkaConfig, KafkaConfigParam, KafkaTopic, TopicAssignments,
};
Expand Down Expand Up @@ -1250,6 +1252,7 @@ impl ConfigObject for ConfigValues {
pub struct Config {
values: ConfigValues,
credentials: Option<Credentials>,
global_config: Option<Arc<GlobalConfig>>,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wasn't 100% sure if i should put it here or not. My reasoning was that if we fail to parse the file, it's better to fail early than later, and it just seems idiomatic within the codebase that we don't do these kind of risky IO-stuff in the services.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a precedent where we defer the deserialization of a static config file to a service:

/// Get filename for static project config.
pub fn project_configs_path(&self) -> PathBuf {
self.path.join("projects")
}

That service reloads the config every 10 seconds though, which I think is overkill. In short, I think your way of doing it is better because it fails early. I'll leave final review to @jan-auer.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I concur with this: We can load the global configs directly from within the actual service like we do it for project configs. That keeps the responsibility of the relay-config crate lower. The overall implementation in this PR does look good, so it would just have to be moved.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alright, I moved it now to the spawn handler, where I'll log an error if the file exists yet it fails to load it.

path: PathBuf,
}

Expand All @@ -1276,6 +1279,7 @@ impl Config {
} else {
None
},
global_config: GlobalConfig::load(&path)?.map(Arc::new),
path: path.clone(),
};

Expand All @@ -1294,6 +1298,7 @@ impl Config {
values: serde_json::from_value(value)
.with_context(|| ConfigError::new(ConfigErrorKind::BadJson))?,
credentials: None,
global_config: None,
path: PathBuf::new(),
})
}
Expand Down Expand Up @@ -1470,6 +1475,11 @@ impl Config {
self.credentials.as_ref()
}

/// Return the optional statically configured global config.
pub fn global_config(&self) -> Option<Arc<GlobalConfig>> {
self.global_config.clone()
}

/// Set new credentials.
///
/// This also writes the credentials back to the file.
Expand Down Expand Up @@ -2022,6 +2032,7 @@ impl Default for Config {
values: ConfigValues::default(),
credentials: None,
path: PathBuf::new(),
global_config: None,
}
}
}
Expand Down
24 changes: 24 additions & 0 deletions relay-dynamic-config/src/global.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
use std::path::{Path, PathBuf};

use serde::{Deserialize, Serialize};

/// A dynamic configuration for all Relays passed down from Sentry.
Expand All @@ -7,3 +9,25 @@ use serde::{Deserialize, Serialize};
#[derive(Default, Clone, Debug, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
pub struct GlobalConfig {}

impl GlobalConfig {
/// The full filename of the global config file, including the file extension.
fn path(base: &Path) -> PathBuf {
base.join("global_config.json")
jan-auer marked this conversation as resolved.
Show resolved Hide resolved
}

/// Loads the [`GlobalConfig`] from a file if it's provided.
jan-auer marked this conversation as resolved.
Show resolved Hide resolved
pub fn load(base: &Path) -> anyhow::Result<Option<Self>> {
let path = Self::path(base);

match path.exists() {
true => {
TBS1996 marked this conversation as resolved.
Show resolved Hide resolved
let file_contents = std::fs::read_to_string(path)?;
let global_config = serde_json::from_str::<GlobalConfig>(file_contents.as_str())?;
TBS1996 marked this conversation as resolved.
Show resolved Hide resolved

Ok(Some(global_config))
}
false => Ok(None),
}
}
}
68 changes: 44 additions & 24 deletions relay-server/src/actors/global_config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ use std::borrow::Cow;
use std::sync::Arc;

use relay_config::Config;
use relay_config::RelayMode;
use relay_dynamic_config::GlobalConfig;
use relay_statsd::metric;
use relay_system::{Addr, AsyncResponse, Controller, FromMessage, Interface, Service};
Expand Down Expand Up @@ -148,8 +149,39 @@ pub struct GlobalConfigService {
impl GlobalConfigService {
/// Creates a new [`GlobalConfigService`].
pub fn new(config: Arc<Config>, upstream: Addr<UpstreamRelay>) -> Self {
let (global_config_watch, _) = watch::channel(Arc::default());
let (internal_tx, internal_rx) = mpsc::channel(1);

let (global_config_watch, _) = match (
config.relay_mode(),
config.has_credentials(),
config.global_config(),
) {
(RelayMode::Proxy | RelayMode::Static, true, None) | (RelayMode::Managed, true, _) => {
relay_log::info!("global config service starting");
// This request will trigger the request intervals when internal_rx receives the
// result from upstream.
Self::request_global_config(upstream.clone(), internal_tx.clone());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not make requests during initialization, as this can block relay startup. We should only do requests once the service is actually started, see my comment below in the spawn_handler.

watch::channel(Arc::new(GlobalConfig::default()))
}
(RelayMode::Proxy | RelayMode::Static, false, None)
| (RelayMode::Managed, false, _) => {
// NOTE(iker): not making a request results in the sleep handler
// not being reset, so no new requests are made.
relay_log::info!("global config service starting with fetching disabled: no credentials configured");
watch::channel(Arc::new(GlobalConfig::default()))
}
(RelayMode::Proxy | RelayMode::Static | RelayMode::Capture, _, Some(global_config)) => {
relay_log::info!("global config service starting with fetching disabled: using static global config");
watch::channel(global_config.clone())
}
(RelayMode::Capture, _, None) => {
relay_log::info!(
"global config service starting with fetching disabled: using default config"
);
watch::channel(Arc::new(GlobalConfig::default()))
}
};
TBS1996 marked this conversation as resolved.
Show resolved Hide resolved

Self {
config,
global_config_watch,
Expand Down Expand Up @@ -185,12 +217,10 @@ impl GlobalConfigService {
///
/// We check if we have credentials before sending,
/// otherwise we would log an [`UpstreamRequestError::NoCredentials`] error.
fn update_global_config(&mut self) {
self.fetch_handle.reset();

let upstream_relay = self.upstream.clone();
let internal_tx = self.internal_tx.clone();

fn request_global_config(
upstream_relay: Addr<UpstreamRelay>,
internal_tx: mpsc::Sender<UpstreamQueryResult>,
) {
TBS1996 marked this conversation as resolved.
Show resolved Hide resolved
tokio::spawn(async move {
metric!(timer(RelayTimers::GlobalConfigRequestDuration), {
let query = GetGlobalConfig::new();
Expand All @@ -209,7 +239,7 @@ impl GlobalConfigService {
/// 1. Whether the request to the upstream was successful.
/// 2. If the request was successful, it then checks whether the returned
/// global config is valid and contains the expected data.
fn handle_result(&mut self, result: UpstreamQueryResult) {
fn handle_upstream_result(&mut self, result: UpstreamQueryResult) {
match result {
Ok(Ok(config)) => {
let mut success = false;
Expand Down Expand Up @@ -253,25 +283,15 @@ impl Service for GlobalConfigService {
tokio::spawn(async move {
let mut shutdown_handle = Controller::shutdown_handle();

relay_log::info!("global config service starting");

if self.config.has_credentials() {
iker-barriocanal marked this conversation as resolved.
Show resolved Hide resolved
// NOTE(iker): if this first request fails it's possible the default
// global config is forwarded. This is not ideal, but we accept it
// for now.
self.update_global_config();
} else {
// NOTE(iker): not making a request results in the sleep handler
// not being reset, so no new requests are made.
relay_log::info!("fetching global configs disabled: no credentials configured");
}

loop {
tokio::select! {
biased;

() = &mut self.fetch_handle => self.update_global_config(),
Some(result) = self.internal_rx.recv() => self.handle_result(result),
() = &mut self.fetch_handle => {
Self::request_global_config(self.upstream.clone(), self.internal_tx.clone());
// Disable new requests interval until we receive the result from upstream.
self.fetch_handle.reset();
}
Some(result) = self.internal_rx.recv() => self.handle_upstream_result(result),
Some(message) = rx.recv() => self.handle_message(message),
_ = shutdown_handle.notified() => self.handle_shutdown(),

Expand Down