Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add a daft dashboard to display queries plans and stats #3790

Merged
merged 178 commits into from
Feb 20, 2025
Merged

feat: Add a daft dashboard to display queries plans and stats #3790

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
178 commits
Select commit Hold shift + click to select a range
4290107
Add basic dashboard
raunakab Jan 28, 2025
2485015
Add basic layout-ing
raunakab Jan 29, 2025
f21bf73
Add basic structure for dashboard
raunakab Jan 29, 2025
71d9203
Edit readme
raunakab Jan 29, 2025
0efb166
Add automatic redirection
raunakab Jan 29, 2025
bca7791
Remove favicon
raunakab Jan 29, 2025
0116857
Add basic data tables to queries page
raunakab Jan 30, 2025
dd48959
Add metrics broadcasting flag
raunakab Jan 30, 2025
6a24b34
Merge branch 'main' into dashboard-queries
raunakab Jan 30, 2025
d47c8f2
Edit empty message
raunakab Jan 30, 2025
d73e1b5
Change lints
raunakab Jan 30, 2025
39b372e
Clean up query page; edit fonts
raunakab Jan 30, 2025
9c6c0cf
Add binary lock file
raunakab Jan 30, 2025
682b17e
Reduce font weightage
raunakab Jan 30, 2025
719a705
Remove font-weightage and add new status badge
raunakab Jan 30, 2025
93a8668
Minor edits to sidebar styling
raunakab Jan 31, 2025
726b4e9
Add collapsible ui component
raunakab Jan 31, 2025
1f49f31
Add collapsible to package.json
raunakab Jan 31, 2025
7d9aac8
Merge branch 'main' into dashboard-queries
raunakab Jan 31, 2025
02130b3
Saving changes
raunakab Jan 31, 2025
6b28b64
Add broadcasting logic
raunakab Jan 31, 2025
b8e9257
Remove formatting from previous commit
raunakab Jan 31, 2025
a029cf0
Implement a broadcasting explain function
raunakab Jan 31, 2025
4d47847
Add calls to `explain_broadcast`
raunakab Jan 31, 2025
0443d41
Add broadcasting of plan
raunakab Feb 2, 2025
4cf3112
Merge branch 'main' into broadcast-metrics
raunakab Feb 2, 2025
7316303
Create a new proxy-service inside of dashboard
raunakab Feb 2, 2025
850ef85
Proxy service working
raunakab Feb 3, 2025
0bcd0a2
Add better utf8 validation; change location of exceeds-check
raunakab Feb 4, 2025
66b58cd
Merge branch 'main' into dashboard-queries
raunakab Feb 5, 2025
cda88a9
Add proxy-service project
raunakab Feb 5, 2025
cb8c15a
Remove initial message
raunakab Feb 5, 2025
036b7ed
Merge branch 'broadcast-metrics' into dashboard-queries
raunakab Feb 5, 2025
e667167
Change what's logged to console
raunakab Feb 5, 2025
b654679
Merge branch 'main' into broadcast-metrics
raunakab Feb 5, 2025
0f90721
Use http posting instead of tcp
raunakab Feb 5, 2025
8e3e326
Rename var
raunakab Feb 5, 2025
568f5ce
Update to using decorator instead
raunakab Feb 5, 2025
0517f7c
Add metrics broadcasting to all write functions
raunakab Feb 5, 2025
bee93c2
Edit protocols used in proxy-service
raunakab Feb 5, 2025
5799b5f
Finish daft server
raunakab Feb 6, 2025
94f2adc
Merge branch 'main' into proxy-service
raunakab Feb 6, 2025
54f5508
Move over to http for dashboard server too
raunakab Feb 6, 2025
eaea587
Merge branch 'broadcast-metrics' into dashboard
raunakab Feb 6, 2025
b86da34
Merge branch 'dashboard-queries' into dashboard
raunakab Feb 6, 2025
4d5611b
Send content as json instead of raw text
raunakab Feb 6, 2025
e6aad55
Merge branch 'broadcast-metrics' into dashboard
raunakab Feb 6, 2025
5d270e4
Edit channel receiving logic
raunakab Feb 6, 2025
ea2da47
Merge branch 'proxy-service' into dashboard
raunakab Feb 6, 2025
06406c6
Change name of field
raunakab Feb 6, 2025
cc1d15d
Update field name
raunakab Feb 6, 2025
7a0d376
Merge branch 'broadcast-metrics' into dashboard
raunakab Feb 6, 2025
a96ace4
Edit query rendering
raunakab Feb 6, 2025
2b400f6
Enable CORS at the server level
raunakab Feb 6, 2025
9bced2d
Merge branch 'proxy-service' into dashboard
raunakab Feb 6, 2025
45fce4d
Change CORS policy
raunakab Feb 6, 2025
65bfdfb
Merge branch 'proxy-service' into dashboard
raunakab Feb 6, 2025
fa475d4
Update error in CORS policy
raunakab Feb 6, 2025
6f31428
Merge branch 'proxy-service' into dashboard
raunakab Feb 6, 2025
0f49f42
Only run effect on component load
raunakab Feb 6, 2025
274d7e5
Fix invalid cloning bug for queries list
raunakab Feb 6, 2025
c437507
Merge branch 'proxy-service' into dashboard
raunakab Feb 6, 2025
26d97cd
Update locking mechanism
raunakab Feb 6, 2025
b1a380f
Add planning time start and end fields
raunakab Feb 6, 2025
25d92e1
Update responses
raunakab Feb 6, 2025
74175a5
Edit the structure of data broadcasted on port 3238
raunakab Feb 6, 2025
34faa38
Add more query-metadata fields
raunakab Feb 6, 2025
abb9dd1
Merge branch 'broadcast-metrics' into dashboard
raunakab Feb 6, 2025
4002163
Merge branch 'proxy-service' into dashboard
raunakab Feb 6, 2025
c1a4c71
Simplify panics
raunakab Feb 6, 2025
2b99ef3
Update values passed into broadcast
raunakab Feb 6, 2025
e9beaf4
Merge branches 'proxy-service' and 'broadcast-metrics' into dashboard
raunakab Feb 6, 2025
5e7e2bc
Add planning time checkpoints
raunakab Feb 6, 2025
69edcd5
Merge branch 'broadcast-metrics' into dashboard
raunakab Feb 6, 2025
45a4459
Add printout of query id
raunakab Feb 6, 2025
4dea98d
Merge branch 'broadcast-metrics' into dashboard
raunakab Feb 6, 2025
f385564
Change broadcasting method from context based to env based
raunakab Feb 6, 2025
541e485
Add 'Content-Type' headers
raunakab Feb 6, 2025
aa33f2a
Merge branch 'main' into dashboard
raunakab Feb 6, 2025
bbc61ea
Merge branch 'broadcast-metrics' into dashboard
raunakab Feb 6, 2025
9acbd05
Change how data is serialized
raunakab Feb 6, 2025
164b5f9
Merge branch 'broadcast-metrics' into dashboard
raunakab Feb 6, 2025
6682f5a
Merge branch 'main' into broadcast-metrics
raunakab Feb 6, 2025
9849fe8
Change environment variables for dashboard; add try-except to prevent
raunakab Feb 6, 2025
ca81621
Edit error message
raunakab Feb 6, 2025
8157579
Remove decorator; use simple function invocations instead
raunakab Feb 6, 2025
9cf4bf1
Merge branch 'broadcast-metrics' into dashboard
raunakab Feb 6, 2025
383e546
Move dashboard code to src
raunakab Feb 6, 2025
5363b5d
Change proxy-service into a library and move it to src
raunakab Feb 6, 2025
96dee25
Add python bindings
raunakab Feb 7, 2025
fe85eb8
Merge branch 'main' into proxy-service
raunakab Feb 7, 2025
834736e
Merge branch 'proxy-service' into dashboard
raunakab Feb 7, 2025
6c528fd
Minor style changes
raunakab Feb 7, 2025
e978e40
Add new page for query visualizations
raunakab Feb 7, 2025
ba8e878
Change case of objects
raunakab Feb 8, 2025
671bbc2
Change case in server
raunakab Feb 8, 2025
4b754b4
Improve error handling
raunakab Feb 8, 2025
415f2af
Add logging crates to dashboard server
raunakab Feb 9, 2025
584c512
Log and early return instead of panicking on binding failure
raunakab Feb 9, 2025
f701d39
Use `select!` instead of `join!`
raunakab Feb 9, 2025
1a647ee
Fork instead of creating a system daemon
raunakab Feb 9, 2025
a4ca1bc
Edit casing of request object
raunakab Feb 9, 2025
c0ed38e
Format
raunakab Feb 9, 2025
271a848
Allow explicit `any`s to be used
raunakab Feb 9, 2025
713e4bf
Add process exit after forking server
raunakab Feb 9, 2025
90472a7
Change from using daemons to forks (again)
raunakab Feb 9, 2025
c28f763
Add mermaid deps
raunakab Feb 10, 2025
900a7a1
Finish dashboard ui mvp
raunakab Feb 10, 2025
af4a14a
Merge branch 'main' into dashboard
raunakab Feb 10, 2025
50f0cf7
Run linter
raunakab Feb 10, 2025
7286534
Allow ignoring underscore-prefixed variable names
raunakab Feb 10, 2025
51bcdf2
Change static export format
raunakab Feb 10, 2025
f729f97
Add skeleton for serving static html files
raunakab Feb 10, 2025
40b5fb2
Add file server
raunakab Feb 11, 2025
c97e7a8
Change directory name
raunakab Feb 11, 2025
d732a79
Add move command to move built artifacts to root and rename
raunakab Feb 11, 2025
4441091
Merge branch 'main' into dashboard
raunakab Feb 11, 2025
e015f04
Edit which endpoint is hit
raunakab Feb 11, 2025
2dc1631
Change how the query page is displayed
raunakab Feb 11, 2025
eeb8948
Edit CORS policies
raunakab Feb 11, 2025
b62ca15
Change how params are passed
raunakab Feb 11, 2025
12fe9f3
Run linter
raunakab Feb 11, 2025
ffb6146
Add suspense boundary over top of component
raunakab Feb 11, 2025
26321d3
Modify directory redirecting
raunakab Feb 11, 2025
78bd05a
Ignore debug html
raunakab Feb 11, 2025
fdc4a05
Add HTML output
raunakab Feb 11, 2025
40ef159
Move location of html directory
raunakab Feb 11, 2025
76ee6d5
Add static html assets to built wheel
raunakab Feb 11, 2025
02e34ee
Change name of static dashboard assets
raunakab Feb 11, 2025
8f931fa
Add ability to serve configurable path
raunakab Feb 11, 2025
3580862
Merge branch 'main' into dashboard
raunakab Feb 11, 2025
5845d13
Remove unused dep
raunakab Feb 11, 2025
c831fde
Add ability to block on dashboard process
raunakab Feb 12, 2025
2ab5cef
Change name/location of daft-dashboard client
raunakab Feb 12, 2025
ccdc941
Remove unused deps
raunakab Feb 12, 2025
c16f596
Remove more deps
raunakab Feb 12, 2025
9bcb526
Change name of launch function (to be namespaced)
raunakab Feb 12, 2025
e63cea3
Remove unnecessary changes to unrelated files
raunakab Feb 12, 2025
a25f53d
Add different entry points for rust and python
raunakab Feb 12, 2025
0ea02d6
Add ability to shutdown server
raunakab Feb 12, 2025
86e500d
Merge branch 'main' into dashboard
raunakab Feb 12, 2025
88fbae1
Fix error where root web-app paths (`/`) were not being served
raunakab Feb 13, 2025
7f0f7ee
Update endpoints
raunakab Feb 13, 2025
3f92b00
Rebuild dashboard with new changes
raunakab Feb 13, 2025
01c6085
Add dashboard make command
raunakab Feb 13, 2025
600b77f
Fix child process not exiting after spawning and finishing
raunakab Feb 13, 2025
87561c3
Rerun dashboard build
raunakab Feb 13, 2025
a4178f3
Update Makefile (and rerun build)
raunakab Feb 13, 2025
849a48c
Add shutdown button
raunakab Feb 13, 2025
af84a98
Merge branch 'main' into dashboard
raunakab Feb 13, 2025
8e4843d
Separate dashboard into new module
raunakab Feb 13, 2025
1748bad
Move files back
raunakab Feb 13, 2025
45aee13
Resolve circular dependency
raunakab Feb 13, 2025
826f77a
Move static_dashboard_assets from daft to src/daft-dashboard-server
raunakab Feb 14, 2025
017e2f4
Remove unnecessary lines
raunakab Feb 14, 2025
cb0d70b
Move structure again
raunakab Feb 14, 2025
4760fff
Rename dashboard to daft_dashboard
raunakab Feb 16, 2025
875de87
Edit error messages; add type-checking block
raunakab Feb 17, 2025
7266c64
Merge branch 'main' into dashboard
raunakab Feb 17, 2025
1560c81
Add new makefile rules; re-run static assets build
raunakab Feb 17, 2025
4842b3a
Remove native binding
raunakab Feb 17, 2025
93af9a9
Fix query plan displaying bug
raunakab Feb 17, 2025
43815d3
Change favicon
raunakab Feb 19, 2025
cf16f0f
Remove home tab; make cells change cursor type to pointer
raunakab Feb 19, 2025
29954a9
Merge branch 'main' into dashboard
raunakab Feb 19, 2025
0a2c1bc
Remove simdutf dependency
raunakab Feb 19, 2025
cb8488b
Rebuild static assets
raunakab Feb 19, 2025
3827f4f
Create a CLI; modify rust internals; remove shutdown button in UI
raunakab Feb 19, 2025
41653fe
Add noops to launch/shutdown commands
raunakab Feb 19, 2025
a739967
Edit child process exit code
raunakab Feb 19, 2025
78d8053
Remove global state and instead put it inside of the loops
raunakab Feb 19, 2025
3ea9afe
Add noop flag to broadcasting logic to ignore already launched error
raunakab Feb 19, 2025
a54370d
Increase zoom of mermaid plan
raunakab Feb 19, 2025
f5af285
Re-run static assets build
raunakab Feb 19, 2025
c5d46d3
Fix test failures (due to faulty imports)
raunakab Feb 19, 2025
3924398
Add try-except
raunakab Feb 19, 2025
3dc2cb1
Make pyo3 dep optional
raunakab Feb 20, 2025
26e322d
Move `fork` crate to be used on Unix builds only
raunakab Feb 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ repos:
hooks:
- id: mypy
additional_dependencies: [types-requests, types-PyYAML, types-tabulate]
files: daft
files: daft, daft_dashboard
exclude: daft/pickle/.*\.py

- repo: https://github.com/pre-commit/pre-commit-hooks
Expand Down Expand Up @@ -57,6 +57,7 @@ repos:
- id: codespell
additional_dependencies:
- tomli
exclude: static-dashboard-assets

- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
Expand Down
Loading
Loading