4.0 beta tracking + feedback

[freddyaboulton]

Place to track all of the bugs, nits, and comments found in the 4.0 branch (v4)

Pre launch

• Get the CI green ✅
• python template will replace from PIL import Image with from PIL import
• js server launching commands should use full path to executables, e.g. gradio, python, node
• FileExplorer not template_able
• Unpin version before release for docs
• Create simple templates for form components
• Before Release - Set gradio>=4.0,<5.0 in pyproject.toml file for custom components
• custom component templating not working for some components #5952
• Add a tag to pyprojecttoml to identify a package as a custom component package
• Front-end dev server will use a port that's used by another app
• Use the data_model in the preprocess
• Model3D component can't be used in dev mode
• Overwrite should clear out the previous directory
• Windows build in CI does not work (frontend)
• Add a server to get backwards compatible type info
• Fix docstrings for events
• Fix website build in v4
• Audio streaming out not working yet #6223
• ImageMask not working
• ImagePaint not working
• gr.Image(source='upload', tool='color-sketch') not working
• Tidy up temp file creation in pre/postprocess (particularly Image).
• Empty JSON component looks weird in kitchen_sink_random when cleared
• In the external.py module, base64 is being returned as opposed to a temp file for audio/image etc. UI works but caching examples doesn't.
• Interface/Blocks __call__ method is still calling serialize so it doesn't work
• Have the Fallback component display plain text/number values
• Reduce size of node folder before shipping
• Fix Literal type hint in models
• gradio_component build builds the front-end
• Make sure all the dependencies are absolute and not relative in front-end modules
• Layout components can't be templated

Nice to have

• Create simple templates for Media components
• smoke test for dev server #6221
• The windows tests are only on release #6220
• Document the theming variables better #6224
• Emit warning when template directories are overwritten in gradio cc build (post launch)
• Write (more) unit tests for gradio_component cli
• Example Css for custom components: https://github.com/gradio-app/gradio/pull/5990/files#r1364407102

Definitely post launch

• Be able to template from a git repo/HF space #6222

[freddyaboulton]

Thoughts on creating my first custom component:

• Allow authors to choose the package name. Right now, it defaults to the class name.
• Rename the front-end template files to match the new component name
• Do not overwrite an existing component unless user explicitly asks
• Use via API page is broken
• Add node_modules to gitignore
• Rename gradio_component to gradio
• Non-interactive components can't be templated because we always try to load the dynamic version. Try with Markdown
• Robust component name identification. Can't template off JSON component because the front-end file is called JSON and the backend file is called json_component.

Can we create single-file templates for common components? It's kind of jarring to see so many different files when you're just getting started.

[abidlabs]

Allow authors to choose the package name. Right now, it defaults to the class name.

If we're referring to the python package, perhaps the default could be gradio-{component_name}. This is less likely to already exist on pypi and can make finding gradio-related packages easier

• We should set pydantic version >= 2.0. I was using 1.x and got:

ImportError: cannot import name 'RootModel' from 'pydantic' (/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pydantic/__init__.cpython-311-darwin.so)

• It might be confusing to users that two ports are launched. Maybe we just show the frontend one?

• I think it would be more intuitive if the default demo was an Interface with input on the left, output on the right, and examples at the bottom. For the value of the example, we could use the example_inputs() method of the component

• the process of adding / removing props is cumbersome -- you have to add/remove the prop in at least 4 places (once in the backend, once in the static version of the component, once in the interactive, and once in the shared). Should we refactor the code so that you have a single file for a component, and there are flags for interactive/static? We'd lose dynamic loading, but it would simplify things so much for almost every component and probably also align better with the mental model people have for mapping frontend and backend.

• it would be nice to know which attributes in the backend need to be implemented at the component-level, versus which ones are taken care of by the Blocks container. For example, for the Textbox, a user doesn't have to worry about implementing label, show_label, container, min_width, etc. but this might not be obvious.

• if you run gradio_component dev and there's an error in your code, there's a whole lot of blank lines that are printed in the terminal console, like this:

In my case, my app was raising a ValueError in the python code

• After I was done developing, I didn't know how to "build" the frontend so that I could create other Gradio apps utilizing my newly built component.

[abidlabs]

From @aliabid94:

• also one last thought when I was building out the hostfile component (even though I didn't build it as a custom component) - the css reset made the input checkbox/radio invisible. I had to copy the css from the existing radio component to make it use the theme settings for inputs so they looked the same. I think the reset css should automatically have the input elements. buttons, etc. use the theme css variables instead of stripping away all styling. This way when users create custom components, they will automatically match the theme styling without them having to apply the theme variables

[duerrsimon]

Great work so far!

Few questions here that remain after the demo yesterday:

• What's the best way to integrate another CSS framework like tailwind in the build process? I'd prefer to have multiple components all use the same stylesheet generated by tailwind and not include tailwind multiple times so I guess it would make sense if one could add some global front-end dependencies. I think tailwind supports project set up as monorepos but I am no expert in this.
• Right now looks like all customcomponents will have to be in Svelte, no? Or will it be possible to use e.g React or Vue at a later point as well?
• Gradio uses @gradio/icons for some icons. Would be great to indicate open icon libraries with similar style so that the iconography stays somewhat consistent

Will be working on a few components now to test stuff more:

• A modified model3D that will allow input from databases via keyword or via upload, output will be the Svelte wrapper around 3Dmol.js I built to visualize the molecules.
• I also wanted to build a variant of the dropdown component with cards to provide a bit better explanation of different modes of the app with some rich text and a markdown. Kind of like this:
  

[abidlabs]

Thanks @duerrsimon for this feedback!

What's the best way to integrate another CSS framework like tailwind in the build process? I'd prefer to have multiple components all use the same stylesheet generated by tailwind and not include tailwind multiple times so I guess it would make sense if one could add some global front-end dependencies. I think tailwind supports project set up as monorepos but I am no expert in this.

Tailwind doesn't play well with Gradio themes. I think the better approach would be to use the CSS variables that are also used in the core Gradio components. You can take a look at existing Gradio components to see how to use the CSS variables. Generally, it looks like this: var(--checkbox-border-color-focus). We'll also document these better (cc @hannahblair @pngwn)

Right now looks like all customcomponents will have to be in Svelte, no? Or will it be possible to use e.g React or Vue at a later point as well?

We'll probably stick to Svelte. Even though I'm primarily backend engineer who tries to stay away from frontend as much as possible, I'll say that the learning curve for Svelte is particularly nice (way easier to pick up than React imo). I've added some resources for learning Svelte to our guide: https://github.com/gradio-app/gradio/wiki/Guide-to-Custom-Components-(Beta)

Gradio uses @gradio/icons for some icons. Would be great to indicate open icon libraries with similar style so that the iconography stays somewhat consistent

@pngwn do you have any thoughts on this?

[abidlabs]

Oh it looks like @hannahblair is already addressing the first point in this PR: #5750!

[freddyaboulton]

Looking forward to the new components @duerrsimon! Thanks for the comments/questions.

[freddyaboulton]

text_with_attachments_publish.mov

EDIT: Using this to store assets lol

Uploading PDFDemo.mov…

[freddyaboulton]

Did pretty much all of this. Converted the remaining TODOs to issues and added the custom-components label.

Thanks again for the feedback @duerrsimon !