Passing follow=True to read_namespaced_pod_log makes it never return

[yuvipanda]

I'm trying to stream logs out of a running pod.

For a pod named 'test' running in the default namespace,

api = client.CoreV1Api()
api.read_namespaced_pod_log("test", "default")

returns the logs so far. But,

api = client.CoreV1Api()
api.read_namespaced_pod_log("test", "default", follow=True)

just never returns. I can't use it with watch.Watch either since that seems to be for a very different purpose.

[yuvipanda]

It looks like I can pass _preload_content=False and get a stream that works - however this is unituitive and doesn't seem documented in the obvious places.

[mbohlool]

Maybe we can add a follow method to Watch class to set "follow" flag instead of "watch" and stream out strings instead of API objects. but _preload_content need to be documented anyway.

[yuvipanda]

+1 on adding it to Watch (or watch could special case this one?)

…

On Wed, May 3, 2017 at 11:26 AM, Mehdy Bohlool ***@***.***> wrote: Maybe we can add a follow method to Watch class to set "follow" flag instead of "watch" and stream out strings instead of API objects. but _preload_content need to be documented anyway. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#199 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAB23vyGNN2NrMePmgLy81a5qAVLm70Rks5r2Ma9gaJpZM4NIDE6> .

-- Yuvi Panda T http://yuvi.in/blog

[mbohlool]

I am just looking for somebody to do this. If nobody took it and I got free time (maybe not near future :) ), I will do it.

[mbohlool]

Watch class can also use inspect.getargspec to see if there is Watch or Follow parameter and then stream the result accordingly.

[kayibal]

@mbohlool Any update on this? I'd be happy to contribute but would need a few sentences as guideline.

[mbohlool]

It would be nice if you can contribute. Here are the things need to be done:

• _preload_content need to be documented somewhere
• Add a section to Watch class to use inspect.getargspec or some other methods to determine if the passed function has Watch or Follow flag and set the appropriate flag.
• Test this on read_namespaced_pod_log to see if it is working. The return type should be set to string if it is not already that (look like read_namespaced_pod_log return type is str so we should be fine there).

Let me know if you need more. I will be happy to review the code when it is ready.

[hackenzheng]

I added "_preload_content=False" based on the official example , it still has no output. How can i get the the stream output

[mitar]

FYI, read_namespaced_pod_log(self, name, namespace, **kwargs) is the signature of the function, so one cannot really use inspect to see if the function is taking watch or follow.

[mitar]

I made a PR here: kubernetes-client/python-base#93

[mitar]

It works well, but I do not have an idea how can this API support resuming. So if the HTTP connection fails, it seems there is no real way to resume following the log from where you stopped. The only approach I can see is to require timestamp to be added to every line and then use that?

[nakulpathak3]

Yeah for me neither
client.CoreV1Api.read_namespaced_pod_log('my_pod_name', 'default', follow=True, _preload_content=False).read()
nor
client.CoreV1Api.read_namespaced_pod_log('my_pod_name', 'default', follow=True) are working. They both just indefinitely hang or return the logs if the pod is done execution.

@yuvipanda did you do something different from the above?

[nakulpathak3]

Ah so what I did initially was close but the temporary fix that worked for me was -

for line in client.CoreV1Api.read_namespaced_pod_log('<pod_name>', 'default', follow=True, _preload_content=False).stream():
    print(line)

[mitar]

@roycaihw Why was this labeled with "help wanted"? There is already a PR for this.

[roycaihw]

@mitar I was doing a blind conversion from our custom help-needed label to the generic "help wanted" label to make the issues more visible to new contributors: help-needed . Sorry I didn't look into the status in each issue :)

Thanks for fixing this! I see Yu was already reviewing that PR so I will leave it to him. Please let me know if you need anything from me.

[mitar]

Ah, that makes it clearer. Yes, that PR was reviewed, I have to figure how to make tests though. I looked briefly but it wasn't exactly clear to me.

If anyone here has any spare cycles, it would be helpful if you could help me improving that PR by adding tests. I can add you to my fork repo to contribute if you want.

[micw523]

I am looking through the tests and I may be able to help. Let me work through the watch routines and I'll let you know. @mitar

[zq-david-wang]

Watch may not be the correct way to handle live streaming logs.
when 'follow' is set to True, the api would return a HTTPResponse object, and there is no non-blocking api exposed by this object, which means that when client is blocking on reading logs, the connection could only be 'closed' by server (or the client process exited).
For non-blocking reading, I had to write a wrapper to use select.poll(fileno()) to make logs non blocking. (similar to the example for exec which is wrapped by a websocket stream client)

Hope I am not making mistakes here, and i will try to write a PR for this.

[mitar]

@zq-david-wang There are two ways here. One is if you want to handle streaming yourself, but there is also Watch class which you can use to do it for you, in a blocking way. And that one I made a PR for.

So maybe there is something more one could do to improve things here for cases where you do not want to use Watch.

[mitar]

I have added tests to kubernetes-client/python-base#93.

I also opened kubernetes-client/python-base#117. I really think tests should be using proper Kubernetes cluster inside CI and not mocked messages. Because now the author of the code both writes the code and test messages, but there is really not any assurance that test messages align with real Kubernetes messages.

[Moofasax]

Having trouble using this, am i missing something?

v1 = client.CoreV1Api()
w = watch.Watch()
for e in w.stream(v1.read_namespaced_pod_log, name=pod, namespace=namespace, follow=True, tail_lines=1, limit_bytes=560, _preload_content=False): print(e)

Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python3.7/site-packages/kubernetes/watch/watch.py", line 132, in stream resp = func(*args, **kwargs) File "/usr/local/lib/python3.7/site-packages/kubernetes/client/apis/core_v1_api.py", line 18538, in read_namespaced_pod_log (data) = self.read_namespaced_pod_log_with_http_info(name, namespace, **kwargs) File "/usr/local/lib/python3.7/site-packages/kubernetes/client/apis/core_v1_api.py", line 18576, in read_namespaced_pod_log_with_http_info " to method read_namespaced_pod_log" % key TypeError: Got an unexpected keyword argument 'watch' to method read_namespaced_pod_log

Posted on stackoverflow if anyone wanted to answer!

[zakkg3]

for line in client.CoreV1Api.read_namespaced_pod_log('<pod_name>', 'default', follow=True, _preload_content=False).stream():

I had to decode the output:

for line in v1.read_namespaced_pod_log(<pod-ame>,<namespace>,follow = True, _preload_content=False).stream():
   print(line.decode('utf-8'))

/edit
the proper way to do it:

from kubernetes import watch
 w = watch.Watch()
 for line in w.stream(v1.read_namespaced_pod_log, name=<pod-name>, namespace='<namespace>'):
    log.info(line)