Cannot stream logs from a pod

[bloveless]

It looks like streaming/watching logs from a pod is part of this library. I only think that because the follow flag being part of the LogParams here https://github.com/clux/kube-rs/blob/master/src/api/raw.rs#L475

Note: I'm just learning rust and struggling with futures, but I'm going for it anyway.

I have basically the following code, but when I set the follow flag to true (commented out below) then I no longer receive any logs. I expected this because the future is never resolving. I found an IntoStream trait but I'm not sure how to use a stream with this data.

Anyway, I've posted this question around the internet in a few places (https://stackoverflow.com/questions/59521539/converting-a-future-to-a-stream-in-kube-rs-library and https://www.reddit.com/r/learnrust/comments/eg49tx/help_with_futuresstreams_and_the_kubers_library/) but no answer yet. I'm curious if you might be able to help me out a little bit or even tell me if streaming logs is not supported at this time.

use kube::{
    api::Api,
    client::APIClient,
    config,
};
use kube::api::{LogParams, RawApi};
use futures::{FutureExt, Stream, future::IntoStream, StreamExt};

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    std::env::set_var("RUST_LOG", "info,kube=trace");
    let config = config::load_kube_config().await?;
    let client = APIClient::new(config);

    // Manage pods
    let pods = Api::v1Pod(client).within("fritzandandre");
    let mut lp = LogParams::default();
    lp.container = Some("php".to_string());
// lp.follow = true;
    lp.tail_lines = Some(100);

    let log_string = pods.log("fritzandandre-php-0", &lp).await?;
    println!("FnA Log: {}", log_string);

    Ok(())
}

Finally: feel free to point me somewhere else if you know of a more appropriate place to post this question.

Thanks!

[clux]

Hey there. Not surprised there's no answers to this yet, as I don't think the necessary features are there in this library atm:

We only have streaming support hooked up to watch calls, and the internal LoggingObject marker trait is not wired up to anything but a raw GET request that awaits the full body (whatever that means for this GET request). You can see the reqwest interface here: https://github.com/clux/kube-rs/blob/023c785c0c451c7c8b1a3f37e37d3f937ec44467/src/client/mod.rs#L107

Essentially we need to wire that up to a call that handles streaming correctly like request_events. I'll merge a good PR that implements this functionality though. Writing up what is needed below. Thanks for the issue!

[clux]

Disclaimer: I've marked as good first issue but that's only really true if you are willing to dig through a bunch of async mechanics and feel you can understand request_events in the same file. It's not trivial by any means unless you've got a good hold of async.

What's needed:

• LoggingObject trait extensions (possibly)
• Extra client/mod.rs reqwest handler to allow streaming text (can't reuse request_events)
• Logic to use extra handler when follow is true

History: #50
We ended up removing the public Log trait from that later on because we we were able to encapsulate it as an internal marker trait, but the original PR there is more or less left intact.

NB: Also not sure what to do with timeouts on this get request. Maybe client-go has some good behaviour for how long it keeps this open. Maybe it just streams it until it drops? There doesn't seem to be a timeout in kubectl at least..

[bloveless]

Thanks for the reply! I’m glad that I’m not just missing in the library. I’m fairly dedicated to resolving this issue but it isn’t a high priority since it is for simple side project I’m working on. I’ll absolutely try and cobble something together.

I’m new to rust and even newer to rust async but I may be able to figure something out with your response as guidance.

Thanks again!

[clux]

Awesome, appreciate the initiative! Do feel free to prop up an unfinished PR. I can help out and leave comments pretty easily at least.

[jnicholls]

I will take a look at the code and see if I can contribute this addition to the library. In fact, I may be interested in being a second maintainer (after I prove my chops ;) which may help elevate this client to beta status, which requires 2+ active maintainers.

[clux]

Definitely would be welcome. A lot to work on and think about here :-)

[ericmcbride]

I forked this and got it working last night. If no one has a PR coming up, ill have one by the end of the weekend if thats fine. I use this library a ton, and would be more then happy to start contributing to it. I just have to figure out the timeout situation. Ive ran it for 30 minutes and nothing has dropped yet. Ill leave it running a bit longer. I use the go client at my job I can see how it works under the hood no problem.

[jnicholls]

That's awesome Eric! Yes, please submit your work! Very cool.

…

On Fri, Jan 31, 2020 at 3:21 PM Eric McBride ***@***.***> wrote: I forked this and got it working last night. If no one has a PR coming up, ill have one by the end of the weekend if thats fine. I use this library a ton, and would be more then happy to start contributing to it. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#109>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAE7KZ5DFGBLFGLI2P3RXDRASB3PANCNFSM4KBXT45Q> .

[ericmcbride]

Awesome im going to clean up the code a bit, I added to the examples of how it works and such and will more then likely submit it tomorrow.

[clux]

Have taken @ericmcbride 's pr and tweaked it a bit to just re-expose the straight bytestream we get from the API, because I don't have a good solution to newline splitting that makes any sense to me.

Published in kube 0.25.0

[bloveless]

Wahoo! Thanks everyone for contributing to this! I’m excited to check it out!