Skip to content

Commit

Permalink
Merge pull request #397 from duyet/chore/update-update
Browse files Browse the repository at this point in the history
chore: update charts /failed-queries, fix CodeDialog
  • Loading branch information
duyet authored Nov 4, 2024
2 parents 7569b0f + 9835393 commit 617dbf3
Show file tree
Hide file tree
Showing 6 changed files with 100 additions and 57 deletions.
104 changes: 65 additions & 39 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Features:
- Useful tools: Zookeeper data exploration, query EXPLAIN, kill queries, etc.
- Visualization metric charts: queries and resource usage, number of merges/mutation, merge performance, query performance, etc.

**Demo** (*The ClickHouse server running on my homelab so can be slow sometimes*):
**Demo** (_The ClickHouse server running on my homelab so can be slow sometimes_):

- [clickhouse-monitoring.vercel.app](https://clickhouse-monitoring.vercel.app)
- [clickhouse.duyet.net](https://clickhouse.duyet.net)
Expand All @@ -33,19 +33,73 @@ To get the project up and running on your local machine, follow these steps:
1. Clone the repository
2. Install dependencies using `npm install` or `yarn install`
3. Create a `.env.local` file by copying the `.env.example` file and filling in the required environment variables:
- `CLICKHOUSE_HOST`: ClickHouse host(s), for example `http://localhost:8123` or `http://ch-1:8123,http://ch-2:8123`
- `CLICKHOUSE_NAME`: (Optional) Name of ClickHouse instance, must match the number of hosts in `CLICKHOUSE_HOST`, for example `localhost` or `ch-1,ch-2`.
- `CLICKHOUSE_USER`: ClickHouse user with permission to query the `system` database.
- `CLICKHOUSE_PASSWORD`: ClickHouse password for the specified user.
- `CLICKHOUSE_MAX_EXECUTION_TIME`: [`max_execution_time`](https://clickhouse.com/docs/en/operations/settings/query-complexity#max-execution-time) timeout in seconds. Default is `10`.
- `CLICKHOUSE_TZ`: ClickHouse server timezone. Default: `''`.
- `NEXT_QUERY_CACHE_TTL`: TTL of [`unstable_cache`](https://nextjs.org/docs/app/api-reference/functions/unstable_cache) - cache the results of most charts to speed up and reuse them across multiple requests. Default: `86400`.
- `NEXT_PUBLIC_LOGO`: (Optional) HTTP path to logo image.
- `EVENTS_TABLE_NAME`: The table name for storing dashboard self-tracking events. Default: `system.monitoring_events`

- `CLICKHOUSE_HOST`: ClickHouse host(s), for example `http://localhost:8123` or `http://ch-1:8123,http://ch-2:8123`
- `CLICKHOUSE_NAME`: (Optional) Name of ClickHouse instance, must match the number of hosts in `CLICKHOUSE_HOST`, for example `localhost` or `ch-1,ch-2`.
- `CLICKHOUSE_USER`: ClickHouse user with permission to query the `system` database.
- `CLICKHOUSE_PASSWORD`: ClickHouse password for the specified user.
- `CLICKHOUSE_MAX_EXECUTION_TIME`: [`max_execution_time`](https://clickhouse.com/docs/en/operations/settings/query-complexity#max-execution-time) timeout in seconds. Default is `10`.
- `CLICKHOUSE_TZ`: ClickHouse server timezone. Default: `''`.
- `NEXT_QUERY_CACHE_TTL`: TTL of [`unstable_cache`](https://nextjs.org/docs/app/api-reference/functions/unstable_cache) - cache the results of most charts to speed up and reuse them across multiple requests. Default: `86400`.
- `NEXT_PUBLIC_LOGO`: (Optional) HTTP path to logo image.
- `EVENTS_TABLE_NAME`: The table name for storing dashboard self-tracking events. Default: `system.monitoring_events`

4. Run the development server with `npm run dev` or `yarn dev`
5. Open [http://localhost:3000](http://localhost:3000) in your browser to see the dashboard.

## ClickHouse Requirements

### 1. Monitoring user role

Suggested role for "monitoring" user must have these privileges on `system` database:

```xml
# File: users.d/monitoring_role.xml
<clickhouse>
<users>
<monitoring>
<password><!-- define password here --></password>
<profile>monitoring_profile</profile>
<networks><ip>::/0</ip></networks>
<grants>
<query>GRANT monitoring_role</query>
</grants>
</monitoring>
</users>

<roles>
<monitoring_role>
<grants>
<query>REVOKE ALL ON *.*</query>
<query>GRANT SELECT,SHOW,dictGet,REMOTE ON *.*</query>
<query>GRANT SELECT,INSERT,ALTER,CREATE,DROP,TRUNCATE,OPTIMIZE,SHOW,dictGet ON system.*</query>
<query>GRANT CREATE TEMPORARY TABLE ON *.*</query>
</grants>
</monitoring_role>
</roles>
</clickhouse>
```

`CREATE TEMPORARY TABLE` is needed because the UI using `FROM merge(system, '^query_log')` allows retrieving all the data from old tables that were renamed during the upgrade.

### 2. Monitoring user profile

```xml
# File: users.d/monitoring_profile.xml
<clickhouse>
<profiles>
<monitoring_profile>
<allow_experimental_analyzer>1</allow_experimental_analyzer>

<!-- Optional: query cache to avoid hit too much queries on database -->
<use_query_cache>1</use_query_cache>
<query_cache_ttl>50</query_cache_ttl>
<query_cache_max_entries>0</query_cache_max_entries>
</monitoring_profile>
</profiles>
</clickhouse>
```

## Deployment

### 1. Vercel
Expand Down Expand Up @@ -94,34 +148,6 @@ EOF
helm install -f values.yaml clickhouse-monitoring-release duyet/clickhouse-monitoring
```

#### Suggested role for "monitoring" user

```xml
# File: users.d/monitoring_role.xml
<clickhouse>
<users>
<monitoring>
<!-- define password here -->
<profile>default</profile>
<networks><ip>::/0</ip></networks>
<grants>
<query>GRANT monitoring_role</query>
</grants>
</monitoring>
</users>

<roles>
<monitoring_role>
<grants>
<query>REVOKE ALL ON *.*</query>
<query>GRANT SELECT,SHOW,dictGet,REMOTE ON *.*</query>
<query>GRANT SELECT,INSERT,ALTER,CREATE,DROP,TRUNCATE,OPTIMIZE,SHOW,dictGet ON system.*</query>
</grants>
</monitoring_role>
</roles>
</clickhouse>
```

## Feedback and Contributions

Feedback and contributions are welcome! Feel free to open issues or submit pull requests.
Expand All @@ -132,4 +158,4 @@ See [LICENSE](LICENSE).

---

![Alt](https://repobeats.axiom.co/api/embed/830f9ce7ba9e7a42f93630e2581506ca34c84067.svg "Repobeats analytics image")
![Alt](https://repobeats.axiom.co/api/embed/830f9ce7ba9e7a42f93630e2581506ca34c84067.svg 'Repobeats analytics image')
18 changes: 14 additions & 4 deletions app/[host]/[query]/queries/failed-queries.ts
Original file line number Diff line number Diff line change
Expand Up @@ -112,18 +112,28 @@ export const failedQueriesConfig: QueryConfig = {
[
'failed-query-count',
{
title: 'Failed Queries over last 14 days',
title: 'Failed Queries over last 7 days (Exception Types)',
interval: 'toStartOfHour',
lastHours: 24 * 14,
lastHours: 24 * 7,
showLegend: false,
},
],
'break',
[
'failed-query-count-by-user',
{
title: 'Failed Queries over last 14 days by Users',
title: 'Failed Queries over last 7 days (Users)',
interval: 'toStartOfHour',
lastHours: 24 * 14,
lastHours: 24 * 7,
showLegend: false,
},
],
[
'failed-query-count-by-user',
{
title: 'Failed Queries over last 24 hours (Users)',
interval: 'toStartOfHour',
lastHours: 24,
showLegend: false,
},
],
Expand Down
2 changes: 1 addition & 1 deletion components/charts/failed-query-count.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ export async function ChartFailedQueryCount({
className,
chartClassName,
chartCardContentClassName,
lastHours = 24,
lastHours = 24 * 7,
showXAxis = true,
showLegend = false,
showCartesianGrid = true,
Expand Down
26 changes: 15 additions & 11 deletions components/data-table/cells/code-dialog-format.tsx
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
import { DialogDescription, DialogTitle } from '@radix-ui/react-dialog'
import { SizeIcon } from '@radix-ui/react-icons'
import dedent from 'dedent'

import {
Dialog,
DialogContent,
Expand All @@ -6,9 +10,6 @@ import {
} from '@/components/ui/dialog'
import { formatQuery } from '@/lib/format-readable'
import { cn } from '@/lib/utils'
import { DialogDescription, DialogTitle } from '@radix-ui/react-dialog'
import { SizeIcon } from '@radix-ui/react-icons'
import dedent from 'dedent'

export interface CodeDialogOptions {
dialog_title?: string
Expand Down Expand Up @@ -76,18 +77,21 @@ export function CodeDialogFormat({ value, options }: CodeDialogFormatProps) {
<SizeIcon className="size-4 flex-none" role="open-dialog" />
</div>
</DialogTrigger>
<DialogContent className={cn('max-w-fit', options?.dialog_classname)}>
{(options?.dialog_title || options?.dialog_description) && (
<DialogHeader>
<DialogTitle>{options.dialog_title}</DialogTitle>
<DialogDescription>{options.dialog_description}</DialogDescription>
</DialogHeader>
)}
<DialogContent
className={cn('max-w-fit', options?.dialog_classname)}
aria-describedby={options?.dialog_description}
>
<DialogHeader>
<DialogTitle>{options?.dialog_title}</DialogTitle>
<DialogDescription>{options?.dialog_description}</DialogDescription>
</DialogHeader>

<div>
<code className="whitespace-pre-wrap text-wrap text-sm text-stone-500">
{typeof content === 'string' ? (
<pre>{dedent(content)}</pre>
<pre className="whitespace-pre-wrap text-wrap">
{dedent(content)}
</pre>
) : (
content
)}
Expand Down
5 changes: 4 additions & 1 deletion components/related-charts.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,10 @@ export async function RelatedCharts({
}

return (
<ServerComponentLazy key={name} fallback={<ChartSkeleton />}>
<ServerComponentLazy
key={name + props.title}
fallback={<ChartSkeleton />}
>
<Chart
className={cn('w-full p-0 shadow-none', className)}
chartClassName="h-44"
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
"e2e:headless": "start-server-and-test dev http://localhost:3000 \"cypress run --e2e\"",
"component": "cypress open --component",
"component:headless": "cypress run --component",
"fmt": "prettier --write \"**/*.{ts,tsx,mdx,json}\" --cache",
"fmt": "prettier --write \"**/*.{ts,tsx,md,mdx,json}\" --cache",
"test": "jest --coverage",
"jest": "jest --coverage"
},
Expand Down

1 comment on commit 617dbf3

@vercel
Copy link

@vercel vercel bot commented on 617dbf3 Nov 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please sign in to comment.