Allow building ESM based components

[philippjfr]

class Counter(pn.reactive.ReactiveHTML):
    _esm = """
    export function render({ model, el }) {
      let button = document.createElement("button");
      button.classList.add("counter-button");
      button.innerHTML = `count is ${model.count}`;
      button.addEventListener("click", () => {
        model.count += 1
      });
      model.properties.count.change.connect(() => {
        button.innerHTML = `count is ${model.count}`;
      });
      el.appendChild(button);
    }
    """
    _stylesheets=["""
    .counter-button { background-color: #ea580c; }
    .counter-button:hover { background-color: #9a3412; }
    """]
    count = param.Integer(default=0)

counter = Counter()
counter.count = 42
counter

class ConfettiWidget(pn.reactive.ReactiveHTML):
    _esm = """
    import confetti from "https://esm.sh/canvas-confetti@1.6";

    export function render({model, el}) {
      let btn = document.createElement("button");
      btn.classList.add("confetti-button");
      btn.innerHTML = "click me!";
      btn.addEventListener("click", () => {
        confetti();
      });
      el.appendChild(btn);
    }
    """
    _stylesheets = ["""
    .confetti-button { background-color: #ea580c; }
    .confetti-button:hover { background-color: #9a3412; }
    """]
    
ConfettiWidget()

• Add nicer API for listening to model events
• Add ability to specify Path and then dynamically watch the path for changes
• Give users the ability to write Preact components
• Add docs
• Add tests
• Rename folders/ files under doc/how_to/custom_components/reactive_esm when comments have been resolved

[codecov]

Codecov Report

Attention: Patch coverage is 84.91547% with 116 lines in your changes missing coverage. Please review.

Project coverage is 81.11%. Comparing base (270bf15) to head (ac438a7).
Report is 3 commits behind head on main.

Files	Patch %	Lines
panel/custom.py	82.29%	37 Missing ⚠️
panel/tests/ui/test_custom.py	79.64%	34 Missing ⚠️
panel/reactive.py	69.11%	21 Missing ⚠️
panel/viewable.py	81.25%	9 Missing ⚠️
panel/pane/base.py	85.41%	7 Missing ⚠️
panel/io/datamodel.py	83.33%	3 Missing ⚠️
panel/layout/base.py	89.47%	2 Missing ⚠️
panel/util/checks.py	50.00%	2 Missing ⚠️
panel/io/resources.py	95.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5593      +/-   ##
==========================================
- Coverage   81.54%   81.11%   -0.43%     
==========================================
  Files         319      323       +4     
  Lines       46915    47509     +594     
==========================================
+ Hits        38255    38538     +283     
- Misses       8660     8971     +311     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[MarcSkovMadsen]

WTF! Where did data go? Are parameter values now on model? And What about state etc? And remember that after layout and delete life cycle hooks are still needed.

[philippjfr]

?

[philippjfr]

Ah, got you. I was trying to copy the API of anywidget which does not separate data and model. For consistency that's probably not a great idea, so will probably rename back to data.

[MarcSkovMadsen]

Ah, got you. I was trying to copy the API of anywidget which does not separate data and model. For consistency that's probably not a great idea, so will probably rename back to data.

If it was just possible to merge data and model?

[MarcSkovMadsen]

Regarding: "Give users the ability to write Preact components".

Remember the purpose is not to enable preact. The purpose is to make it easy for users to create react components in a simple way. Almost like copy pasting existing examples. And those examples very often use JSX or even typescript/ TSX.

With Preact its complicated to replicate React examples. Skilled React developers can do this via clever configuration of package.json file. But with ReactiveHTML its complicated. Furthermore Preact points to htm for JSX like syntax.

In #5550 and inspired by IpyReact you can see how React, JSX and typescript/ TSX more easily can be supported using Sucrase. I think we should follow this path.

We should either document how to use React + Sucrase with ReactiveHTML + _esm. Or even better create a ready to use ReactBaseComponent.

[MarcSkovMadsen]

Please consider adding the requirements from #5550 to your todo list in the first post 😄

[MarcSkovMadsen]

One open question is also if you can use code in _esm to insert Panel objects? See #5551

[philippjfr]

If it was just possible to merge data and model?

It's a bad idea because a user may define properties that clash with properties that are automatically inherited.

Remember the purpose is not to enable preact. The purpose is to make it easy for users to create react components in a simple way. Almost like copy pasting existing examples. And those examples very often use JSX or even typescript/ TSX.

The idea there is that we already use and export Preact so they can start using it at zero extra cost. This does not preclude actual React based workflows but those will come at the cost of loading React on top of everything else.

One open question is also if you can use code in _esm to insert Panel objects? See #5551

Will have to think about that.

[maximlt]

WTF! Where did data go? Are parameter values now on model? And What about state etc? And remember that after layout and delete life cycle hooks are still needed.

I think we have to be a little more careful with our language. We've heard complaints that we weren't always very welcoming.

[MarcSkovMadsen]

Thanks. It was meant as a joke. But not perceived that way.

[philippjfr]

Progress

(Low frame rate is due to the GIF, not the example running slow.)

[philippjfr]

With the latest version you can now leverage Preact from within the ESM bundle:

import pathlib
import param
import panel as pn

from panel.reactive import ReactiveHTML

class Todo(ReactiveHTML):

    tasks = param.List()

    _esm = pathlib.Path(__file__).parent / 'preact.js'

preact = Todo(width=800, height=600)
    
preact.servable()

function App(props) {
  const [newTask, setNewTask] = useState('');
  const [tasks, setTasks] = useState(props.tasks);

  const addTodo = () => {
    const task = { name: newTask, done: false }
    setTasks([...tasks, task]);
    props.data.tasks = [...tasks, task]
    setNewTask('');
  }

  const onInput = (event) => {
    const { value } = event.target;
    setNewTask(value)
  }

  const onKeyUp = (event) => {
    const { key } = event;
    if (key === 'Enter') {
      addTodo();
    }
  }

  return html`
    <div class="task-container">
      <div class="task-input">
        <input
          placeholder="Add new item..."
          type="text"
          class="task-new"
          value=${newTask}
          onInput=${onInput}
          onKeyUp=${onKeyUp}
        />
        <button class="task-add" onClick=${addTodo}>Add</button>
      </div>
      <ul class="task-list">
        ${tasks.map((task, index) => html`
          <li key=${index} class="task-item" v-for="task in state.tasks">
            <label class="task-item-container">
              <div class="task-checkbox">
                <input type="checkbox" class="opacity-0 absolute" value=${task.done} />
                <svg class="task-checkbox-icon" width=20 height=20 viewBox="0 0 20 20">
                  <path d="M0 11l2-2 5 5L18 3l2 2L7 18z" />
                </svg>
              </div>
              <div class="ml-2">${task.name}</div>
            </label>
          </li>
        `)}
      </ul>
    </div>
  `;
}

export function render({ data, model, el }) {
  return html`<${App} tasks=${data.tasks} data=${data}/>`
}

[philippjfr]

I've got sucrase working now so you can write React code without htm. At only 5kb increase in bundle size that seems worth it.

[philippjfr]

So the example above now works with:

interface Task {
  name: string;
  done: boolean
};

function App() {
  const [newTask, setNewTask] = useState('');
  const [tasks, setTasks] = useState([] as Task[]);

  const addTodo = () => {
    setTasks([...tasks, { name: newTask, done: false }]);
    setNewTask('');
  }

  const onInput = (event: Event) => {
    const { value } = event.target as HTMLInputElement;
    setNewTask(value)
  }

  const onKeyUp = (event: KeyboardEvent) => {
    const { key } = event;
    if (key === 'Enter') {
      addTodo();
    }
  }

  return (
    <div class="task-container">
      <div class="task-input">
        <input
          placeholder="Add new item..."
          type="text"
          class="task-new"
          value={newTask}
          onInput={onInput}
          onKeyUp={onKeyUp}
        />
        <button class="task-add" onClick={addTodo}>Add</button>
      </div>
      <ul class="task-list">
        {tasks.map((task, index) => (
          <li key={index} class="task-item" v-for="task in state.tasks">
            <label class="task-item-container">
              <div class="task-checkbox">
                <input type="checkbox" class="opacity-0 absolute" value={task.done} />
                <svg class="task-checkbox-icon" viewBox="0 0 20 20">
                  <path d="M0 11l2-2 5 5L18 3l2 2L7 18z" />
                </svg>
              </div>
              <div class="ml-2">{task.name}</div>
            </label>
          </li>
        ))}
      </ul>
    </div>
  );
}

export function render({ data, model, el }) {
  const app = <App tasks={data.tasks}/>;
  return app;
}

[philippjfr]

Okay I think this example demonstrates the React API fairly well:

Python

import pathlib
import param
import panel as pn

from panel.reactive import ReactiveHTML

class Example(ReactiveHTML):

    color = param.Color()

    text = param.String()
    
    _esm = pathlib.Path(__file__).parent / 'react_demo.js'

example = Example(text='Hello World!')

pn.Row(pn.Param(example.param, parameters=['color', 'text']), example).servable()

JSX

function App(props) {
  const [color, setColor] = props.state.color
  const [text, setText ] = props.state.text
  const style = {color: color}
  return (
    <div>
      <h1 style={style}>{text}</h1>
      <input
        value={text}
        onChange={e => setText(e.target.value)} 
      />
    </div>
  );
}

export function render({ state }) {
  return <App state={state}/>;
}

The global namespace also contains a React object containing:

React: {
      Component,
      useCallback,
      useContext,
      useEffect,
      useErrorBoundary,
      useLayoutEffect,
      useState,
      useReducer,
      createElement: h
    }

[philippjfr]

I think it's a pretty clean API now. Here's another demo:

reactive_esm_2x.mp4

This demonstrates:

• Bi-directional syncing of state parameters and React state variables
• ESM module importing
• Automatic creation of state hooks with the state argument to the render function
• Availability of the React namespace
• Automatic transpilation of JSX / TypeScript code

That said, I'm now considering whether we should split ReactiveHTML and ReactiveESM.

[MarcSkovMadsen]

... and also considering whether ReactiveESM is the right name?

I don't think AnyWidget name would be right for us because a widget is an input component in our terminology.

But would AnyComponent, BaseComponent, ESMComponent, BaseESMComponent or something similar be a better name? How can we communicate its an equivalent of AnyWidget in our ecosystem supporting any type of component?

The one I like the most is BaseComponent. I would choose AnyComponent if I had to explain it was similar to AnyWidget but supporting more than widgets.

[MarcSkovMadsen]

I think building a MaterialUI component using the ESM and (P)React functionality would be a really good test if you have done this right.

[philippjfr]

Yes, ESMComponent seems like a good name for it. And also yes, I'm working on adding the necessary shims to get mui working.

[MarcSkovMadsen]

I would think twice about putting ESM in the name :-) Focus on what the component can do for the developer, not what technology its build on. Its AnyWidget not ESMWidget because its much easier to explain.

[MarcSkovMadsen]

The downside of carving it out as a separate component is complexity. Will you explain both ReactiveHTML and ...ESM...? What should users choose?

[philippjfr]

The same is true if you overload ReactiveHTML, suddenly you have this monstrosity which does 500 different things.

[MarcSkovMadsen]

Would it be an idea to give ...ESM.. a nice name and focus on that one? Maybe deprecate ReactiveHTML over a long period of time?

[philippjfr]

Yes, in principle. The nice thing about ReactiveHTML is that thanks to the templating you can write a useful component entirely without knowing any Javascript, so I'd be hesitant to deprecate it, but certainly it could be de-emphasized in the docs.

[philippjfr]

ipymolstar also works as-is:

import pathlib

from panel.custom import AnyWidgetComponent
import param
from typing import Optional, List, TypedDict, Any

THEMES = {
    "light": {
        "bg_color": "#F7F7F7",
        "css": (pathlib.Path(__file__).parent / "pdbe_light.css").read_text(),
    }
}


class Color(TypedDict):
    r: int
    g: int
    b: int


# codeieum translation of QueryParam from
# https://github.com/molstar/pdbe-molstar/blob/master/src/app/helpers.ts#L180
class QueryParam(TypedDict, total=False):
    auth_seq_id: Optional[int]
    entity_id: Optional[str]
    auth_asym_id: Optional[str]
    struct_asym_id: Optional[str]
    residue_number: Optional[int]
    start_residue_number: Optional[int]
    end_residue_number: Optional[int]
    auth_residue_number: Optional[int]
    auth_ins_code_id: Optional[str]
    start_auth_residue_number: Optional[int]
    start_auth_ins_code_id: Optional[str]
    end_auth_residue_number: Optional[int]
    end_auth_ins_code_id: Optional[str]
    atoms: Optional[List[str]]
    label_comp_id: Optional[str]
    color: Optional[Color]
    sideChain: Optional[bool]
    representation: Optional[str]
    representationColor: Optional[Color]
    focus: Optional[bool]
    tooltip: Optional[str]
    start: Optional[Any]
    end: Optional[Any]
    atom_id: Optional[List[int]]
    uniprot_accession: Optional[str]
    uniprot_residue_number: Optional[int]
    start_uniprot_residue_number: Optional[int]
    end_uniprot_residue_number: Optional[int]


class ResetParam(TypedDict, total=False):
    camera: Optional[bool]
    theme: Optional[bool]
    highlightColor: Optional[bool]
    selectColor: Optional[bool]


class PDBeMolstar(AnyWidgetComponent):
    _esm = pathlib.Path(__file__).parent / "ipymolstar.js"
    _stylesheets = [pathlib.Path(__file__).parent / "pdbe_light.css"]

    # width = param.String("100%")
    height = param.String("500px")

    molecule_id = param.String()
    custom_data = param.Dict(default=None, allow_None=True)
    assembly_id = param.String()
    default_preset = param.Selector(
        objects=["default", "unitcell", "all-models", "supercell"],
        default="default",
    )
    ligand_view = param.Dict(default=None, allow_None=True)
    alphafold_view = param.Boolean(default=False)
    superposition = param.Boolean(default=False)
    superposition_params = param.Dict(default=None, allow_None=True)
    visual_style = param.Selector(
        objects=[
            "cartoon",
            "ball-and-stick",
            "carbohydrate",
            "ellipsoid",
            "gaussian-surface",
            "molecular-surface",
            "point",
            "putty",
            "spacefill",
        ],
        default=None,
        allow_None=True,
    )
    hide_polymer = param.Boolean(False)
    hide_water = param.Boolean(False)
    hide_heteroatoms = param.Boolean(False)
    hide_carbs = param.Boolean(False)
    hide_non_standard = param.Boolean(False)
    hide_coarse = param.Boolean(False)
    load_maps = param.Boolean(False)
    map_settings = param.Dict(default=None, allow_None=True)
    bg_color = param.String()
    highlight_color = param.String(default="#FF6699")
    select_color = param.String(default="#33FF19")
    lighting = param.Selector(
        objects=["flat", "matte", "glossy", "metallic", "plastic"],
        default=None,
        allow_None=True,
    )
    validation_annotation = param.Boolean(False)
    domain_annotation = param.Boolean(False)
    symmetry_annotation = param.Boolean(False)
    pdbe_url = param.String("https://www.ebi.ac.uk/pdbe/")
    encoding = param.Selector(objects=["bcif", "cif"], default="bcif")
    low_precision_coords = param.Boolean(False)
    select_interaction = param.Boolean(True)
    granularity = param.Selector(
        objects=[
            "element",
            "residue",
            "chain",
            "entity",
            "model",
            "operator",
            "structure",
            "elementInstances",
            "residueInstances",
            "chainInstances",
        ],
        default="residue",
    )
    subscribe_events = param.Boolean(False)
    hide_controls = param.Boolean(False)
    hide_controls_icon = param.Boolean(False)
    hide_expand_icon = param.Boolean(False)
    hide_settings_icon = param.Boolean(False)
    hide_selection_icon = param.Boolean(False)
    hide_animation_icon = param.Boolean(False)
    sequence_panel = param.Boolean(False)
    pdbe_link = param.Boolean(True)
    loading_overlay = param.Boolean(False)
    expanded = param.Boolean(False)
    landscape = param.Boolean(False)
    reactive = param.Boolean(False)

    spin = param.Boolean(False)
    _focus = param.List(default=None, allow_None=True)
    _highlight = param.Dict(default=None, allow_None=True)
    _clear_highlight = param.Boolean(default=False)
    color_data = param.Dict(default=None, allow_None=True)
    _clear_selection = param.Boolean(default=False)
    tooltips = param.Dict(default=None, allow_None=True)
    _clear_tooltips = param.Boolean(default=False)
    _set_color = param.Dict(default=None, allow_None=True)
    _reset = param.Dict(allow_None=True, default=None)
    _update = param.Dict(allow_None=True, default=None)

    _args = param.Dict()

    mouseover_event = param.Dict()

    def __init__(self, theme="light", **kwargs):
        #_css = THEMES[theme]["css"]
        bg_color = kwargs.pop("bg_color", THEMES[theme]["bg_color"])
        super().__init__(bg_color=bg_color, **kwargs)

    def color(
        self,
        data: list[QueryParam],
        non_selected_color=None,
        keep_colors=False,
        keep_representations=False,
    ) -> None:
        """
        Alias for PDBE Molstar's `select` method.

        See https://github.com/molstar/pdbe-molstar/wiki/3.-Helper-Methods for parameter
        details
        """

        self.color_data = {
            "data": data,
            "nonSelectedColor": non_selected_color,
            "keepColors": keep_colors,
            "keepRepresentations": keep_representations,
        }
        self.color_data = None

    def focus(self, data: list[QueryParam]):
        self._focus = data
        self._focus = None

    def highlight(self, data: list[QueryParam]):
        self._highlight = data
        self._highlight = None

    def clear_highlight(self):
        self._clear_highlight = not self._clear_highlight

    def clear_tooltips(self):
        self._clear_tooltips = not self._clear_tooltips

    def clear_selection(self, structure_number=None):
        # move payload to the traitlet which triggers the callback
        self._args = {"number": structure_number}
        self._clear_selection = not self._clear_selection

    # todo make two traits: select_color, hightlight_color
    def set_color(
        self, highlight: Optional[Color] = None, select: Optional[Color] = None
    ):
        data = {}
        if highlight is not None:
            data["highlight"] = highlight
        if select is not None:
            data["select"] = select
        if data:
            self._set_color = data
            self._set_color = None

    def reset(self, data: ResetParam):
        self._reset = data
        self._reset = None

    def update(self, data):
        self._update = data
        self._update = None

view = PDBeMolstar(
    molecule_id="1qyn",
    theme="light",
    hide_water=True,
    visual_style="cartoon",
    spin=False,
    lighting='glossy',
    height="500px",
    width=500
)

view.tooltips = { 'data': [
                        { 'struct_asym_id': 'A', 'tooltip': 'Custom tooltip for chain A' }, 
                        { 'struct_asym_id': 'B', 'tooltip': 'Custom tooltip for chain B' }, 
                        { 'struct_asym_id': 'C', 'tooltip': 'Custom tooltip for chain C' }, 
                        { 'struct_asym_id': 'D', 'tooltip': 'Custom tooltip for chain D' }, 
                    ] }

view.servable()

[philippjfr]

This is now in a state I'm happy with and I'd like to get this merged to iterate further.