Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(hydro_deploy)!: rename integration crates to drop CLI references #1413

Merged
merged 2 commits into from
Aug 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,8 @@ jobs:
${{ inputs.execute && '--execute' || '--no-publish' }}
hydroflow hydroflow_lang hydroflow_macro hydroflow_plus
hydroflow_datalog hydroflow_datalog_core
hydro_deploy hydro_cli hydroflow_cli_integration
hydroflow_plus_cli_integration
hydro_deploy hydro_cli hydroflow_deploy_integration
hydroflow_plus_deploy
stageleft stageleft_macro stageleft_tool
multiplatform_test
env:
Expand Down
4 changes: 2 additions & 2 deletions .idea/hydroflow.iml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

51 changes: 21 additions & 30 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ members = [
"hydro_deploy/core",
"hydro_deploy/hydro_cli",
"hydro_deploy/hydro_cli_examples",
"hydro_deploy/hydroflow_cli_integration",
"hydro_deploy/hydroflow_plus_cli_integration",
"hydro_deploy/hydroflow_deploy_integration",
"hydro_deploy/hydroflow_plus_deploy",
"hydroflow",
"hydroflow_datalog",
"hydroflow_datalog_core",
Expand Down
4 changes: 2 additions & 2 deletions RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ showing that all the changelogs can be modified. Make sure the version bumps loo

```log
[INFO ] Updating crates-io index
[WARN ] Refused to publish 'hydroflow_cli_integration' as as it didn't change.
[INFO ] Will not publish or alter 3 dependent crates: unchanged = 'hydroflow_cli_integration', 'variadics', 'pusherator'
[WARN ] Refused to publish 'hydroflow_deploy_integration' as as it didn't change.
[INFO ] Will not publish or alter 3 dependent crates: unchanged = 'hydroflow_deploy_integration', 'variadics', 'pusherator'
[INFO ] WOULD auto-bump dependent package 'hydroflow_lang' from 0.4.0 to 0.5.0 for publishing
[INFO ] WOULD auto-bump dependent package 'hydroflow_datalog_core' from 0.4.0 to 0.5.0 for publishing, for SAFETY due to breaking package 'hydroflow_lang'
[INFO ] WOULD auto-bump dependent package 'hydroflow_datalog' from 0.4.0 to 0.5.0 for publishing, for SAFETY due to breaking package 'hydroflow_datalog_core'
Expand Down
12 changes: 6 additions & 6 deletions docs/docs/deploy/your-first-deploy.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,20 +26,20 @@ rustup update
cargo test
```

We'll need to add an additional dependency for `hydroflow_cli_integration` to our `Cargo.toml`:
We'll need to add an additional dependency for `hydroflow_deploy_integration` to our `Cargo.toml`:

```toml
[dependencies]
# ...
hydroflow_cli_integration = "0.1.1"
hydroflow_deploy_integration = "0.1.1"
MingweiSamuel marked this conversation as resolved.
Show resolved Hide resolved
```

Let's open up `src/main.rs` in the generated project and write a new `main` function that initializes Hydro Deploy:

```rust
#[hydroflow::main]
async fn main() {
let ports = hydroflow::util::cli::init().await;
let ports = hydroflow::util::deploy::init().await;
}
```

Expand Down Expand Up @@ -77,12 +77,12 @@ Now, we need to wire up the ports. Hydro Deploy uses _named ports_, which can th
Returning briefly to our Hydroflow code, we can then load these ports and use them to send and receive packets:

```rust
use hydroflow_cli_integration::ConnectedDirect;
use hydroflow_deploy_integration::ConnectedDirect;
use hydroflow::hydroflow_syntax;

#[hydroflow::main]
async fn main() {
let ports = hydroflow::util::cli::init().await;
let ports = hydroflow::util::deploy::init().await;

let input_recv = ports
.port("input")
Expand All @@ -97,7 +97,7 @@ async fn main() {
.await
.into_sink();

hydroflow::util::cli::launch_flow(hydroflow_syntax! {
hydroflow::util::deploy::launch_flow(hydroflow_syntax! {
source_iter(["hello".to_string()]) -> dest_sink(output_send);
input = source_stream(input_recv) -> tee();
input -> dest_sink(output_send);
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/hydroflow_plus/quickstart/clusters.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ To deploy this application, we must set up the Hydro Deploy configuration as bef
use std::cell::RefCell;

use hydro_deploy::{Deployment, HydroflowCrate};
use hydroflow_plus_cli_integration::TrybuildHost;
use hydroflow_plus_deploy::TrybuildHost;

#[tokio::main]
async fn main() {
Expand Down
2 changes: 1 addition & 1 deletion hydro_deploy/core/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ dunce = "1.0.4"
dyn-clone = "1"
futures = "0.3.26"
futures-core = "0.3.26"
hydroflow_cli_integration = { path = "../hydroflow_cli_integration", version = "^0.5.2" }
hydroflow_deploy_integration = { path = "../hydroflow_deploy_integration", version = "^0.5.2" }
indicatif = "0.17.8"
inferno = "0.11.20"
memo-map = "0.3.2"
Expand Down
2 changes: 1 addition & 1 deletion hydro_deploy/core/src/custom_service.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ use std::sync::{Arc, OnceLock, Weak};

use anyhow::{bail, Result};
use async_trait::async_trait;
use hydroflow_cli_integration::{ConnectedDirect, ServerPort};
use hydroflow_deploy_integration::{ConnectedDirect, ServerPort};
use tokio::sync::RwLock;

use super::hydroflow_crate::ports::{
Expand Down
2 changes: 1 addition & 1 deletion hydro_deploy/core/src/hydroflow_crate/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ pub enum CrateTarget {
Example(String),
}

/// Specifies a crate that uses `hydroflow_cli_integration` to be
/// Specifies a crate that uses `hydroflow_deploy_integration` to be
/// deployed as a service.
#[derive(Clone)]
pub struct HydroflowCrate {
Expand Down
2 changes: 1 addition & 1 deletion hydro_deploy/core/src/hydroflow_crate/ports.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ use anyhow::Result;
use async_recursion::async_recursion;
use async_trait::async_trait;
use dyn_clone::DynClone;
use hydroflow_cli_integration::ServerPort;
use hydroflow_deploy_integration::ServerPort;
use tokio::sync::RwLock;

use super::HydroflowCrateService;
Expand Down
6 changes: 3 additions & 3 deletions hydro_deploy/core/src/hydroflow_crate/service.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ use std::time::Duration;
use anyhow::{bail, Result};
use async_trait::async_trait;
use futures_core::Future;
use hydroflow_cli_integration::{InitConfig, ServerPort};
use hydroflow_deploy_integration::{InitConfig, ServerPort};
use serde::Serialize;
use tokio::sync::{mpsc, RwLock};

Expand Down Expand Up @@ -254,7 +254,7 @@ impl Service for HydroflowCrateService {
serde_json::to_string::<InitConfig>(&(bind_config, self.meta.clone())).unwrap();

// request stdout before sending config so we don't miss the "ready" response
let stdout_receiver = binary.cli_stdout();
let stdout_receiver = binary.deploy_stdout();

binary.stdin().send(format!("{formatted_bind_config}\n"))?;

Expand Down Expand Up @@ -290,7 +290,7 @@ impl Service for HydroflowCrateService {

let formatted_defns = serde_json::to_string(&sink_ports).unwrap();

let stdout_receiver = self.launched_binary.as_ref().unwrap().cli_stdout();
let stdout_receiver = self.launched_binary.as_ref().unwrap().deploy_stdout();

self.launched_binary
.as_ref()
Expand Down
8 changes: 4 additions & 4 deletions hydro_deploy/core/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ use std::sync::Arc;

use anyhow::Result;
use async_trait::async_trait;
use hydroflow_cli_integration::ServerBindConfig;
use hydroflow_crate::perf_options::PerfOptions;
use hydroflow_deploy_integration::ServerBindConfig;

pub mod deployment;
pub use deployment::Deployment;
Expand Down Expand Up @@ -74,11 +74,11 @@ pub struct ResourceResult {
pub trait LaunchedBinary: Send + Sync {
fn stdin(&self) -> mpsc::UnboundedSender<String>;

/// Provides a oneshot channel for the CLI to handshake with the binary,
/// with the guarantee that as long as the CLI is holding on
/// Provides a oneshot channel to handshake with the binary,
/// with the guarantee that as long as deploy is holding on
/// to a handle, none of the messages will also be broadcast
/// to the user-facing [`LaunchedBinary::stdout`] channel.
fn cli_stdout(&self) -> oneshot::Receiver<String>;
fn deploy_stdout(&self) -> oneshot::Receiver<String>;

fn stdout(&self) -> mpsc::UnboundedReceiver<String>;
fn stderr(&self) -> mpsc::UnboundedReceiver<String>;
Expand Down
12 changes: 6 additions & 6 deletions hydro_deploy/core/src/localhost/launched_binary.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ use crate::LaunchedBinary;
pub struct LaunchedLocalhostBinary {
child: Mutex<async_process::Child>,
stdin_sender: mpsc::UnboundedSender<String>,
stdout_cli_receivers: Arc<Mutex<Option<oneshot::Sender<String>>>>,
stdout_deploy_receivers: Arc<Mutex<Option<oneshot::Sender<String>>>>,
stdout_receivers: Arc<Mutex<Vec<mpsc::UnboundedSender<String>>>>,
stderr_receivers: Arc<Mutex<Vec<mpsc::UnboundedSender<String>>>>,
}
Expand Down Expand Up @@ -55,7 +55,7 @@ impl LaunchedLocalhostBinary {
});

let id_clone = id.clone();
let (stdout_cli_receivers, stdout_receivers) = prioritized_broadcast(
let (stdout_deploy_receivers, stdout_receivers) = prioritized_broadcast(
BufReader::new(child.stdout.take().unwrap()).lines(),
move |s| ProgressTracker::println(format!("[{id_clone}] {s}")),
);
Expand All @@ -67,7 +67,7 @@ impl LaunchedLocalhostBinary {
Self {
child: Mutex::new(child),
stdin_sender,
stdout_cli_receivers,
stdout_deploy_receivers,
stdout_receivers,
stderr_receivers,
}
Expand All @@ -80,11 +80,11 @@ impl LaunchedBinary for LaunchedLocalhostBinary {
self.stdin_sender.clone()
}

fn cli_stdout(&self) -> oneshot::Receiver<String> {
let mut receivers = self.stdout_cli_receivers.lock().unwrap();
fn deploy_stdout(&self) -> oneshot::Receiver<String> {
let mut receivers = self.stdout_deploy_receivers.lock().unwrap();

if receivers.is_some() {
panic!("Only one CLI stdout receiver is allowed at a time");
panic!("Only one deploy stdout receiver is allowed at a time");
}

let (sender, receiver) = oneshot::channel::<String>();
Expand Down
2 changes: 1 addition & 1 deletion hydro_deploy/core/src/localhost/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ use std::sync::Arc;
use anyhow::{Context, Result};
use async_process::{Command, Stdio};
use async_trait::async_trait;
use hydroflow_cli_integration::ServerBindConfig;
use hydroflow_deploy_integration::ServerBindConfig;

use super::{
ClientStrategy, Host, HostTargetType, LaunchedBinary, LaunchedHost, ResourceBatch,
Expand Down
Loading
Loading