feat: Add a daft dashboard to display queries plans and stats

[raunakab]

Overview

This PR introduces a new (statically built and served) dashboard web-application UI.

We have a simple server that binds to port 3238 and does 3 things:

• listens to broadcasts from the daft query engine
• serves an API endpoint
• serves static HTML/CSS/JS files and assets

The static HTML files which are served can then be accessed by pointing your browser towards http::localhost:3238.

Usage

from daft import dashboard

dashboard.launch()

# rest of your daft queries here

Notes

The server process is launched off as an orphaned process. It will live on even after the script that initialized it has completed.

You can kill this process (only manually as of now) by running kill -9 $(lsof -t -i :3238).

Note to reviewers

Although this PR claims to be a big diff, in reality, it's only big because of a lot of generated React UI components (inside of src/daft-dashboard-client).

For backend reviewers, please review src/daft-dashboard-server and daft. For frontend reviewers, please review src/daft-dashboard-client.

Generated files

Please ignore all the generated HTML/CSS/JS files inside of daft/static_dashboard_assets. They are automatically generated by bun run build during prior to release.

These files have to be checked into the repo since they need to be released alongside daft wheels.

[codspeed-hq]

CodSpeed Performance Report

Merging #3790 will improve performances by 11.82%

_{Comparing dashboard (e63cea3) with main (ca36593)}

Summary

⚡ 1 improvements
✅ 26 untouched benchmarks

Benchmarks breakdown

	Benchmark	BASE	HEAD	Change
⚡	test_iter_rows_first_row[100 Small Files]	167 ms	149.3 ms	+11.82%

[jaychia]

On a high level:

1. Why is everything in src/? Previously we had stuff in dashboard/ which made sense because the server and typescript code doesn't need to live inside of Daft itself.

2. How much does this increase the binary size of the Daft wheel by, since we're packaging all this stuff now with the Daft wheel I'm assuming? Do we need to add a new getdaft[dashboard] perhaps?

I think to merge this we should think a little harder about the packaging story. Seems like things are all shoved into the main Daft binary today which isn't ideal. Can we lay out how things are packaged in the PR description?

[jaychia]

Why is Cargo lock changing?

[raunakab]

The src/daft-dashboard-server uses hyper and some other http-frameworks to:

1. Serve the daft broadcast listener
2. Serve the dashboard API
3. Serve the dashboard web server

[jaychia]

This seems like dashboard-specific code. Let's define this outside of dataframe.py if possible, likely some kind of Client exposed by the dashboard module I think.

[raunakab]

Sure, I can update it to live inside of the daft/dashboard.py file.