readNamespacedPodLog with follow=true

[cabrinoob]

I have a problem with the usage of readNamespacedPodLog with the follow parameter set to true.

This function returns a Promise (BlueBird). I'd like to display the logs in real time of a pod in my Angular app but I have no idea of how to use the promise of a streamed response ...

I tried this :

import * as rxnode from 'rx-node' 
...
let logs = await this.k8sApi.readNamespacedPodLog(podName,this.ns,containerName,true)
rxnode.fromReadableStream(logs.response).subscribe((i)={
 ...
})

I saw in the source code that the Promise returns a http.ClientResponse which extends an IncomingMessage which extends a stream.Readable which implements NodeJS.ReadableStream. So thats why I though using .fromReadableStream with the rx-node librarie ... But it does'nt work.

Do you have any advice of how I can use this method (in streaming way) with Javascript?

Thank you

[brendandburns]

The stream should implements events, so you can run:

stream.on('data', (chunk) => { ... })

[cabrinoob]

@brendanburns
Hi,
Do you have a snippet because I'am doing like this :

let obj = await this.k8sApi.readNamespacedPodLog(podName,this.ns,containername,true)
obj.response.on('data',(chunk) => {
    console.log(chunk.toString())
})

And it does'nt work. I'am stucked on the await. I cannot figure out how to mix promise and stream ...

[brendanburns]

I'll try to repro/write an example today... Thanks for the patience!

…

On Fri, Sep 28, 2018, 7:35 AM cabrinoob ***@***.***> wrote: @brendanburns <https://github.com/brendanburns> Hi, Do you have a snippet because I'am doing like this : let obj = await this.k8sApi.readNamespacedPodLog(podName,this.ns,containername,true) obj.response.on('data',(chunk) => { console.log(chunk.toString()) }) And it does'nt work. I'am stucked on the await. I cannot figure out how to mix promise and stream ... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#110 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABOYXt7bkCADVNhzuWuAnUtz3ByuUMEQks5ufjPNgaJpZM4W61P8> .

[marvinosswald]

this issue over at the python client matches with my observations, the promise never resolves with follow true

kubernetes-client/python#199

[brendandburns]

Edit: This example is incorrect, see below

Yeah, I think what you need to do is:

const promise =  this.k8sApi.readNamespacedPodLog(podName,this.ns,containername,true);
obj.response.on('data',(chunk) => {
    console.log(chunk.toString())
})
const obj = await promise;

This is because the log with follow=true never closes the TCP stream, so the promise never completes.

But if you await after you register the event listener, you should get events...

[dkapanidis]

I don't get the example, how is obj used before it is initialized?

I get ReferenceError: obj is not defined trying this

Yeah, I think what you need to do is:

const promise =  this.k8sApi.readNamespacedPodLog(podName,this.ns,containername,true);
obj.response.on('data',(chunk) => {
    console.log(chunk.toString())
})
const obj = await promise;

This is because the log with follow=true never closes the TCP stream, so the promise never completes.

But if you await after you register the event listener, you should get events...

[brendandburns]

Ah, sorry, you are correct, I was skimming and not paying careful enough attention to the code.

I think that this will require a different solution, which probably involves writing some custom code.

Sorry for the mislead.

[juliahw]

@cabrinoob Did you ever find a workaround? I'm stuck on the same issue.

[cabrinoob]

@juliahw : No. I think i'll try to take a look at socket.io with a nodeJS backend or whatever. But for me, this cannot work as expected directly using a promise that returns a stream

[juliahw]

Ok :( I think the GoDaddy Kubernetes client might be able to do this, so I'll give it a go. It's too bad we can't use the officially supported library.

[cabrinoob]

Yes, it could be nice to have the .getStream() implementation of godaddy watch()

[borremosch]

I was facing the same issue and have written a FollowLogs class to be able to follow logs. It is included in #214.

[cabrinoob]

Hi @borremosch, thank you for your PR. Do you have a code sample that shows how to use your class ?

I rapidely read your code and I was asking myself a question : You only specify namespace and podname to get logs, but what if you have a multicontainer pod ? You have to specify the container name too isn't it?

Thank you

[borremosch]

Hey @cabrinoob, you can put the class anywhere in your codebase. A specific container can be specified in the option parameter like so:

new FollowLogs(kubeConfig).followPodLog(
  'pod',
  'namespace',
  newLogs => console.log(newLogs),
  err => console.log(err),
  { container: 'main' }
);

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[depeele]

Without access to the underlying request object, there is no way to directly consume the streaming response that will be generated when follow=true is specified (short of writing a custom, direct request as proposed by @borremosch).

Support could be added by updating/patching the code generated for readNamespacedPodLog() within the return authenticationPromise.then() block to something like:

                if (Object.keys(localVarFormParams).length) {
                    if (localVarUseFormData) {
                        localVarRequestOptions.formData = localVarFormParams;
                    }
                    else {
                        localVarRequestOptions.form = localVarFormParams;
                    }
                }
                return new Promise((resolve, reject) => {
                  if (localVarQueryParameters['follow']) {
                    /* Immediately return the request instance to allow the
                     * caller direct access to the incoming stream.
                     */
                    resolve( {req: localVarRequest(localVarRequestOptions)} );

                  } else {
                    localVarRequest(localVarRequestOptions, (error, response, body) => {
                        if (error) {
                            reject(error);
                        }
                        else {
                            body = models_1.ObjectSerializer.deserialize(body, "string");
                            if (response.statusCode && response.statusCode >= 200 && response.statusCode <= 299) {
                                resolve({ response: response, body: body });
                            }
                            else {
                                reject({ response: response, body: body });
                            }
                        }
                    });
                  }
                });

A diff patch would look like:

--- coreV1Api-orig.js 2020-03-16 14:03:02.000000000 -0400
+++ coreV1Api.js  2020-03-16 14:09:07.000000000 -0400
@@ -12610,6 +12610,13 @@
                     }
                 }
                 return new Promise((resolve, reject) => {
+                  if (localVarQueryParameters['follow']) {
+                    /* Immediately return the request instance to allow the
+                     * caller direct access to the incoming stream.
+                     */
+                    resolve( {req: localVarRequest(localVarRequestOptions)} );
+
+                  } else {
                     localVarRequest(localVarRequestOptions, (error, response, body) => {
                         if (error) {
                             reject(error);
@@ -12624,6 +12631,7 @@
                             }
                         }
                     });
+                  }
                 });
             });
         });

[tettusud]

Any one has solution for it?

[popcornylu]

I used #83 to follow the logs.

[moshfeu]

@popcornylu how? It's not a regular resolved Promise? Can you show a code example?

[izhilenkov]

@moshfeu #83 isn't a Promise at all. It's piping data to your stream.

[kingdevnl]

Yup i'm stuck at this aswell

I realy need a way to follow the logs so I can pipe them to a socket.io connection

[joac]

Hitting this bug... is related to how the client uses the request library. It tries to fetch all the body that is a never ending stream...

[joac]

I made this workaround using node fetch:

import https from 'https';
import request from 'request';
import { KubeConfig } from '@kubernetes/client-node';     
import fetch, { Response } from 'node-fetch';  

// Build the request options
const buildOptions = async (config: KubeConfig url: string) => {                                                                                                                                                                                     
  const draftOptions: request.Options = {                                                                                                                                              
    url,                                                                                                                                                                               
    headers: {                                                                                                                                                                         
      Accept: 'application/json',                                                                                                                                                      
    },                                                                                                                                                                                 
  };                                                                                                                                                                                   
                                                                                                                                                                                       
  await config.applyToRequest(draftOptions);                                                                                                                                           
  const { headers, ca, key, cert } = draftOptions;                                                                                                                                     
  // fetch has a different parameter handling for ssl than request lib, this i
  const httpsAgent = new https.Agent({                                                                                                                                                 
    cert,                                                                                                                                                                              
    ca,                                                                                                                                                                                
    key,                                                                                                                                                                               
  });                                                                                                                                                                                  
  return {                                                                                                                                                                    
    agent: httpsAgent,                                                                                                                                                                 
    headers,                                                                                                                                                                           
  };              
}
export const getLogs = async (config: KubeConfig, namespace: string, podName: string, container: string): Promise<Response> => {                                                                                                                                                                                                                                       
  const cluster = config.getCurrentCluster();                                                                                                                                          
  if (!cluster) {                                                                                                                                                                      
    throw new Error('No current cluster found');                                                                                                                                       
  }                                                                                                                                                                                    
  const server = cluster.server;                                                                                                                                                                                                                                                                                              
  const params = new URLSearchParams({ container, follow: 'true' });                                                                                                                   
                                                                                                                                                                                       
  const url = `${server}/api/v1/namespaces/${encodeURIComponent(namespace)}/pods/${encodeURIComponent(                                                                                 
    podName,                                                                                                                                                                           
  )}/log?${params}`;                                                                                                                                                                   
  const options = await buildOptions(config, url);                                                                                                                                                                  
  return await fetch(url, options);                                                                                                                                                    
}; 

Then you can use the stream available on the response.body

[scho3034]

I have a problem with the usage of readNamespacedPodLog with the follow parameter set to true.

This function returns a Promise (BlueBird). I'd like to display the logs in real time of a pod in my Angular app but I have no idea of how to use the promise of a streamed response ...

I tried this :

import * as rxnode from 'rx-node' 
...
let logs = await this.k8sApi.readNamespacedPodLog(podName,this.ns,containerName,true)
rxnode.fromReadableStream(logs.response).subscribe((i)={
 ...
})

I saw in the source code that the Promise returns a http.ClientResponse which extends an IncomingMessage which extends a stream.Readable which implements NodeJS.ReadableStream. So thats why I though using .fromReadableStream with the rx-node librarie ... But it does'nt work.

Do you have any advice of how I can use this method (in streaming way) with Javascript?

Thank you

How did you dynamically add the podName ?

[abdennour]

The example works fine BTW: https://github.com/kubernetes-client/javascript/blob/master/examples/follow-logs.js