Get Intermediate Results from Python Client

[freddyaboulton]

Description

Summary

1. Adds an outputs() method to Job that returns a list with all of the intermediate outputs received from the job at the time it is called.
2. If the endpoint is a generator, result() will only block until the first result is returned

We will add status callbacks in the near future to enable event-based updates as well.

Usage

With the simple count_generator space

generator_client = grc.Client(space="gradio/count_generator")
job = generator_client.predict(10, fn_index=0)
while not job.done():
    time.sleep(1)
    print(job.outputs())

[]
[]
[]
[]
[]
[]
[['0'], ['1']]
[['0'], ['1'], ['2'], ['3']]
[['0'], ['1'], ['2'], ['3'], ['4'], ['5']]
[['0'], ['1'], ['2'], ['3'], ['4'], ['5'], ['6'], ['7']]
[['0'], ['1'], ['2'], ['3'], ['4'], ['5'], ['6'], ['7'], ['8'], ['9']]
[['0'], ['1'], ['2'], ['3'], ['4'], ['5'], ['6'], ['7'], ['8'], ['9']]

With a fake diffusion space

stable_diff = grc.Client(space="freddyaboulton/fake_diffusion")
job = stable_diff.predict(5, fn_index=0)
while not job.done():
    time.sleep(1)
print(job.outputs())

Checklist:

• I have performed a self-review of my own code
• I have added a short summary of my change to the CHANGELOG.md
• My code follows the style guidelines of this project
• I have commented my code in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

A note about the CHANGELOG

Hello 👋 and thank you for contributing to Gradio!

All pull requests must update the change log located in CHANGELOG.md, unless the pull request is labeled with the "no-changelog-update" label.

Please add a brief summary of the change to the Upcoming Release > Full Changelog section of the CHANGELOG.md file and include
a link to the PR (formatted in markdown) and a link to your github profile (if you like). For example, "* Added a cool new feature by [@myusername](link-to-your-github-profile) in [PR 11111](https://github.com/gradio-app/gradio/pull/11111)".

If you would like to elaborate on your change further, feel free to include a longer explanation in the other sections.
If you would like an image/gif/video showcasing your feature, it may be best to edit the CHANGELOG file using the
GitHub web UI since that lets you upload files directly via drag-and-drop.

[gradio-pr-bot]

All the demos for this PR have been deployed at https://huggingface.co/spaces/gradio-pr-deploys/pr-3694-all-demos

[freddyaboulton]

Otherwise the websocket connection would close when sending large files

[abidlabs]

Tiniest of nits: using float("inf") will allow you to not have to import math

[abidlabs]

So technically, this wouldn't work with older versions of Gradio library that are running generators, right? Is there a fallback we can do based on the message we get in WS?

[abidlabs]

For example, I don't think it works with this Space: abidlabs/test-yield which is running 3.23.1b1

[freddyaboulton]

@abidlabs I think it's working for me

[abidlabs]

Hi @freddyaboulton this looks very cool. My main concern is compatibility with older Spaces. I think we should be able to know if we're calling a generator simply by looking at the messages returned from the websocket, just like the frontend does it, right?

One other suggestion: it would be more elegant imo if we didn't have to pass in the deserialize function all the way down through the communicator. Maybe we instead pass it to the Job constructor and run it when we call outputs()? Just a suggestion

[freddyaboulton]

It's possible for the deserialization to error so I'm catching that here and adding the exception string but I'm open to other solutions

[freddyaboulton]

@abidlabs I think it's working for me

[freddyaboulton]

@abidlabs This should be good for another look! I refactored it so that we no longer check from config if the function is a generator.

[abidlabs]

Would add a docstring for this method, along with the expected signature

[abidlabs]

Hey @freddyaboulton,

Very nice PR. Looks good to me, but a couple of clarification questions:

• outputs() returns each of the intermediate outputs as lists, but the final output not enclosed in a list. Concretely, running

import gradio_client as grc

generator_client = grc.Client("gradio/count_generator")
job = generator_client.predict(10, fn_index=0)
while not job.done():
    pass
job.outputs()

Produces:

[['0'], ['1'], ['2'], ['3'], ['4'], ['5'], ['6'], ['7'], ['8'], ['9'], '9']

Why is the 9 repeated? And why is it not enclosed in a list the final 9? It feels to me that the expected output should be:

['0', '1', ... '8', '9']

• For gr.Interface in which an iterator function is passed in, we see an interesting value returned:

['',
  {'visible': False, '__type__': 'update'},
  {'visible': True, '__type__': 'update'}]

That's because clicking the submit button causes not only the inner function to run, but also the submit button to turn into a stop button, etc. All of these updates are returned by the same API endpoint. I think we should refactor gr.Interface to avoid this from happening. We can do this now using .then(), although I think it's fine to do this in another PR.

Abubakar

[freddyaboulton]

@abidlabs Following our synch earlier today, I made the necessary modifications so the output is not stored in a list if there is only one output component.

For count_generator, you should see ['0', ..., '9'] and this I've added a new endpoint to that space called /gen_images that will generate pairs of random images to test a case with more than one output component:

Should be good for a re-review! Thanks again

[abidlabs]

Tested and works great, thanks @freddyaboulton! Awesome PR

[freddyaboulton]

Thank you for the review @abidlabs !