Embedding gradio within a fastapi app #1608
Comments
|
Good question. Yes, gradio launches a fastapi api under the hood. But you might be able to do what you need just using the gradio app, as the gradio app also exposes a prediction route. Specifically, when you launch the Gradio app, you'll have:
You can see the docs for the prediction API by clicking on "View API" at the bottom of the interface page |
|
is the other way around also possible. so i can have finer control on the app and routes? |
|
What are you trying to do that isn't currently possible with Gradio? Edit: I'm wrong. |
|
We access to FastAPI application like this
Not %100 sure but I think both should be possible |
|
here is what I am trying to achieve. I have a prediction endpoint running in a fastapi Of course i can make two separate dockers each running separate fastapi apps, however I want to have tighter coupling and consistency so that my Please also see my additional comments in #1612 |
|
Hi @sciai-ai I took a dive into FastAPI and it turns out that it is actually possible to serve your Gradio app within another FastAPI app quite easily using FastAPI's """
How to launch your Gradio app within another FastAPI app.
Run this from the terminal as you would normally start a FastAPI app: `uvicorn run:app`
and navigate to http://localhost:8000/gradio in your browser to see the Gradio app.
"""
from fastapi import FastAPI
import gradio as gr
CUSTOM_PATH = "/gradio"
app = FastAPI()
@app.get("/")
def read_main():
return {"message": "This is your main app"}
io = gr.Interface(lambda x: "Hello, " + x + "!", "textbox", "textbox")
gradio_app = gr.routes.App.create_app(io)
app.mount(CUSTOM_PATH, gradio_app)This serves the main app on |
|
What if it's a |
same question here.. |
|
It should work the same @willprincehearts |
|
It does not work when we use gradio block especially when I use from fastapi import FastAPI
import gradio as gr
CUSTOM_PATH = "/gradio"
app = FastAPI()
@app.get("/")
def read_main():
return {"message": "This is your main app"}
def flip_text(x):
return x[::-1]
with gr.Blocks as demo:
gr.Markdown(
"""
# Flip Text!
Start typing below to see the output.
"""
)
input = gr.Textbox(placeholder="Flip this text")
output = gr.Textbox()
input.change(fn=flip_text, inputs=input, outputs=output)
gradio_app = gr.routes.App.create_app(demo)
app.mount(CUSTOM_PATH, gradio_app) |
|
I am having the same issue as @rainmaker712 |
|
Hi @cpatrickalves @rainmaker712 are you both using the latest version of Gradio? Please check and if you still find the bug, please create a new issue |
|
Yes, I have same bug with same issue below with the latest version of Gradio.
|
|
Hi @rainmaker712 ! There are two issues with your code snippet:
Using this code works: from fastapi import FastAPI
import gradio as gr
CUSTOM_PATH = "/gradio"
app = FastAPI()
@app.get("/")
def read_main():
return {"message": "This is your main app"}
def flip_text(x):
return x[::-1]
with gr.Blocks() as demo:
gr.Markdown(
"""
# Flip Text!
Start typing below to see the output.
"""
)
input = gr.Textbox(placeholder="Flip this text")
output = gr.Textbox()
input.change(fn=flip_text, inputs=input, outputs=output)
gr.mount_gradio_app(app, demo, path=CUSTOM_PATH) |
|
@freddyaboulton Thank you for your help! Actually, I am having trouble with using fastapi with gradio Chatbot. Here is the example of chatbot from the official gradio tutorial. import gradio as gr
import random
import time
with gr.Blocks() as demo:
chatbot = gr.Chatbot()
msg = gr.Textbox()
clear = gr.ClearButton([msg, chatbot])
def user(user_message, history):
return gr.update(value="", interactive=False), history + [[user_message, None]]
def bot(history):
bot_message = random.choice(["How are you?", "I love you", "I'm very hungry"])
history[-1][1] = ""
for character in bot_message:
history[-1][1] += character
time.sleep(0.05)
yield history
response = msg.submit(user, [msg, chatbot], [msg, chatbot], queue=False).then(
bot, chatbot, chatbot
)
response.then(lambda: gr.update(interactive=True), None, [msg], queue=False)
demo.queue()
demo.launch()I have no idea how to use block & launch with CUSTOM_PATH = "/gradio"
app = FastAPI()
@app.get("/")
def read_main():
return {"message": "This is your main app"}
gr.mount_gradio_app(app, demo, path=CUSTOM_PATH)Thanks for help. |
|
Hi @rainmaker712 ! You can use the queue by calling import gradio as gr
import random
import time
from fastapi import FastAPI
with gr.Blocks() as demo:
chatbot = gr.Chatbot()
msg = gr.Textbox()
clear = gr.ClearButton([msg, chatbot])
def user(user_message, history):
return gr.update(value="", interactive=False), history + [[user_message, None]]
def bot(history):
bot_message = random.choice(["How are you?", "I love you", "I'm very hungry"])
history[-1][1] = ""
for character in bot_message:
history[-1][1] += character
time.sleep(0.05)
yield history
response = msg.submit(user, [msg, chatbot], [msg, chatbot], queue=False).then(
bot, chatbot, chatbot
)
response.then(lambda: gr.update(interactive=True), None, [msg], queue=False)
demo.queue()
CUSTOM_PATH = "/gradio"
app = FastAPI()
@app.get("/")
def read_main():
return {"message": "This is your main app"}
app = gr.mount_gradio_app(app, demo, path=CUSTOM_PATH)You cannot call launch so setting |
|
@freddyaboulton Thank you so much for your advice, It works very well! |
|
Hi ,
may i know why its showing this error. ? |
|
steps to create - @app.post("/app/v1/predict") this ui.py for gradio frontend docker container 2 import gradio as gr def predict(input): demo = gr.Interface(fn=predict, inputs="text", outputs="text") demo.queue().launch(server_name="0.0.0.0", server_port=7000) ' |
|
without docker container both are working fine |
|
Sorry to resurrect a closed issue, and thanks for all the guidance in the thread so far. I've managed to achieve a lot of what I needed to with
I get the error does nothing when I switch to the selected tab, Is there a way to achieve redirection to a FastAPI page when switching gradio tabs? |
|
+1 |
|
Hi @kells1986 !! For 1. Is the queue enabled in your app? When the queue is enabled connections are made via websocket so it may be a bug with how we copy over the session state to the request object |
|
Hey @freddyaboulton, I am also struggling with accessing the FastAPI request information within Gradio. I have been searching for a solution for a few days now and I would really appreciate any tips on how to overcome this issue using JS, as you suggested. Happy to hear from other approaches if any, as well :) |
|
Hi @antonionieto ! What is your issue? If you want to access the FastAPI request within your gradio function, you can just type the input argument as Here is an example that does a redirect to the gradio application using both raw html and a FastAPI from fastapi import FastAPI
from fastapi.responses import HTMLResponse, RedirectResponse
import gradio as gr
import uvicorn
app = FastAPI()
def request(req: gr.Request):
return {k:req.headers[k] for k in req.headers}
print_request = gr.Interface(request, None, "json")
HTML = """
<!DOCTYPE html>
<html>
<h1>Gradio Request Demo</h1>
<p>Click the button to be redirected to the gradio app!</p>
<button onclick="window.location.pathname='/gradio'">Redirect</button>
</html>
"""
@app.get("/")
def read_main():
return HTMLResponse(HTML)
@app.get("/foo")
def redirect():
return RedirectResponse("/gradio")
if __name__ == "__main__":
app = gr.mount_gradio_app(app, print_request, path="/gradio")
uvicorn.run(app, port=8080) |
|
Thanks, @freddyaboulton!! That made the difference. Btw, for those who are not aware of the Event listener compatibility with JS (like me), you can also use the |
|
@abidlabs |
I am looking for something similar also. It would be nice to have some functionality where you can have some sort of depends inject already offered by FastAPI that could look something like this |
|
@young-hun-jo as a workaround I think you can set the demo.auth = ...
app = gr.mount_gradio_app(...) |
I have tried to run the following code, and got a following error, I solved it by defining demo.auth = ("hello", "world")
demo.auth_message = None
app = gr.mount_gradio_app(...) |
I use the method, but it always runing, can anyone help? |
|
|
@freddyaboulton @kimmchii Thanks for your reply. I updated official docs about |
|
|
|
Is there a way you can access the parent app from the Gradio interface: with gr.Block() as demo:
...
def on_click(text):
score = demo.app.state.model.predict(text)
btn = gr.Button()
btn.click(fn=on_click)Parent app from interface import demo
app = FastAPI()
app.state.model = load_ml_model()
app = gr.mount_gradio_app(app, demo, path="gradio")This is more or less the logic I'd like to accomplish. But it seems Any pointer appreciated! |
I am making a single fastapi app where one route leads to prediction, and a second route which leads to the gradio interface. My understanding is that gradio itself launches a fastapi instance. is it possible to achieve this?
The text was updated successfully, but these errors were encountered: