Skip to content

Commit

Permalink
feat: Distributed Operational UI (#16097)
Browse files Browse the repository at this point in the history
  • Loading branch information
cyriltovena authored Feb 14, 2025
1 parent d1e0fa7 commit dbf2bef
Show file tree
Hide file tree
Showing 285 changed files with 29,287 additions and 55,806 deletions.
104 changes: 104 additions & 0 deletions .cursor/rules/frontend.mdc
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
---
description: building frontend code in react and typescript
globs: *.tsx,*.ts,*.css
---
You are an expert in TypeScript, Node.js, react-dom router, React, Shadcn UI, Radix UI and Tailwind.

# Key Principles

- Write concise, technical TypeScript code with accurate examples.
- Use functional and declarative programming patterns; avoid classes.
- Prefer iteration and modularization over code duplication.
- Use descriptive variable names with auxiliary verbs (e.g., isLoading, hasError).
- Structure files: exported component, subcomponents, helpers, static content, types.

# Component Composition

Break down complex UIs into smaller, more manageable parts, which can be easily reused across different parts of the application.

- Reusability: Components can be easily reused across different parts of the application, making it easier to maintain and update the UI.
- Modularity: Breaking the UI down into smaller, more manageable components makes it easier to understand and work with, particularly for larger and more complex applications.
- Separation of Concerns: By separating the UI into smaller components, each component can focus on its own specific functionality, making it easier to test and debug.
- Code Maintainability: Using smaller components that are easy to understand and maintain makes it easier to make changes and update the application over time.

Avoid large components with nested rendering functions


# State

- useState - for simpler states that are independent
- useReducer - for more complex states where on a single action you want to update several pieces of state
- context + hooks = good state management don't use other library
- prefix -context and -provider for context and respective provider.

# Naming Conventions

- Use lowercase with dashes for directories (e.g., components/auth-wizard).
- Favor named exports for components.

# TypeScript Usage

- Use TypeScript for all code; prefer interfaces over types.
- Avoid enums; use maps instead.
- Use functional components with TypeScript interfaces.

# Syntax and Formatting

- Use the "function" keyword for pure functions.
- Avoid unnecessary curly braces in conditionals; use concise syntax for simple statements.
- Use declarative JSX.
- Use arrow functions for improving code readability and reducing verbosity, especially for smaller functions like event handlers or callback functions.
- Avoid using inline styles

## Project Structure

Colocate things as close as possible to where it's being used

```
src/
├── components/ # React components
│ ├── ui/ # Shadcn UI components
│ │ ├── errors/ # Error handling components
│ │ └── breadcrumbs/ # Navigation breadcrumbs
│ ├── shared/ # Shared components used across pages
│ │ └── {pagename}/ # Page-specific components
│ ├── common/ # Truly reusable components
│ └── features/ # Complex feature modules
│ └── theme/ # Theme-related components and logic
├── pages/ # Page components and routes
├── layout/ # Layout components
├── hooks/ # Custom React hooks
├── contexts/ # React context providers
├── lib/ # Utility functions and constants
└── types/ # TypeScript type definitions
```

DON'T modify shadcn component directly they are stored in src/components/ui/*


# UI and Styling

- Use Shadcn UI, Radix, and Tailwind for components and styling.
- Implement responsive design with Tailwind CSS.

# Performance Optimization

- Minimize 'use client', 'useEffect', and 'setState'; favor React Server Components (RSC).
- Wrap client components in Suspense with fallback.
- Use dynamic loading for non-critical components.
- Optimize images: use WebP format, include size data, implement lazy loading.

# Key Conventions

- Use 'nuqs' for URL search parameter state management.
- Optimize Web Vitals (LCP, CLS, FID).
- Limit 'use client':

# Unit Tests

Unit testing is done to test individual components of the React application involving testing the functionality of each component in isolation to ensure that it works as intended.

- Use Jest for unit testing.
- Write unit tests for each component.
- Use React Testing Library for testing components.
- Use React Testing Library for testing components.
5 changes: 3 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ dlv
rootfs/
dist
!pkg/dataobj/explorer/dist
!pkg/ui/frontend/dist
*coverage.txt
*test_results.txt
.DS_Store
Expand All @@ -37,8 +38,8 @@ dist
pkg/loki/wal
tools/lambda-promtail/main
tools/dev/kafka/data/
pkg/dataobj/explorer/ui/node_modules/*
pkg/dataobj/explorer/ui/.vite/*
pkg/ui/frontend/node_modules/*
pkg/ui/frontend/.vite/*

# Submodule added by `act` CLI
_shared-workflows-dockerhub-login
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,7 @@ cmd/loki/loki-debug:
CGO_ENABLED=0 go build $(DEBUG_GO_FLAGS) -o $@ ./$(@D)

ui-assets:
make -C pkg/dataobj/explorer/ui build
make -C pkg/ui/frontend build
###############
# Loki-Canary #
###############
Expand Down
4 changes: 2 additions & 2 deletions cmd/loki/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@ FROM node:22-alpine AS ui-builder
RUN apk add --no-cache make
COPY . /src/loki
WORKDIR /src/loki
RUN make -C pkg/dataobj/explorer/ui build
RUN make -C pkg/ui/frontend build

# Go build stage
FROM golang:${GO_VERSION} AS build
ARG IMAGE_TAG
COPY . /src/loki
COPY --from=ui-builder /src/loki/pkg/dataobj/explorer/dist /src/loki/pkg/dataobj/explorer/dist
COPY --from=ui-builder /src/loki/pkg/ui/frontend/dist /src/loki/pkg/ui/frontend/dist
WORKDIR /src/loki
RUN make clean && make BUILD_IN_CONTAINER=false IMAGE_TAG=${IMAGE_TAG} loki

Expand Down
36 changes: 36 additions & 0 deletions docs/sources/shared/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,42 @@ Pass the `-config.expand-env` flag at the command line to enable this way of set
# Configures the server of the launched module(s).
[server: <server>]

ui:
# Name to use for this node in the cluster.
# CLI flag: -ui.node-name
[node_name: <string> | default = "<hostname>"]

# IP address to advertise in the cluster.
# CLI flag: -ui.advertise-addr
[advertise_addr: <string> | default = ""]

# Name of network interface to read address from.
# CLI flag: -ui.interface
[interface_names: <list of strings> | default = [<private network interfaces>]]

# How frequently to rejoin the cluster to address split brain issues.
# CLI flag: -ui.rejoin-interval
[rejoin_interval: <duration> | default = 15s]

# Number of initial peers to join from the discovered set.
# CLI flag: -ui.cluster-max-join-peers
[cluster_max_join_peers: <int> | default = 3]

# Name to prevent nodes without this identifier from joining the cluster.
# CLI flag: -ui.cluster-name
[cluster_name: <string> | default = ""]

# Enable using a IPv6 instance address.
# CLI flag: -ui.enable-ipv6
[enable_ipv6: <boolean> | default = false]

discovery:
# List of peers to join the cluster. Supports multiple values separated by
# commas. Each value can be a hostname, an IP address, or a DNS name (A/AAAA
# and SRV records).
# CLI flag: -ui.discovery.join-peers
[join_peers: <list of strings> | default = []]

# Configures the distributor.
[distributor: <distributor>]

Expand Down
5 changes: 3 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ require (
github.com/google/uuid v1.6.0
github.com/gorilla/mux v1.8.1
github.com/gorilla/websocket v1.5.3
github.com/grafana/ckit v0.0.0-20250109002736-4ca45886e452
github.com/grafana/cloudflare-go v0.0.0-20230110200409-c627cf6792f2
github.com/grafana/dskit v0.0.0-20241007172036-53283a0f6b41
github.com/grafana/go-gelf/v2 v2.0.1
Expand Down Expand Up @@ -299,13 +300,13 @@ require (
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-hclog v1.6.3 // indirect
github.com/hashicorp/go-immutable-radix v1.3.1 // indirect
github.com/hashicorp/go-msgpack v1.1.5 // indirect
github.com/hashicorp/go-msgpack v0.5.5 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/go-retryablehttp v0.7.7 // indirect
github.com/hashicorp/go-rootcerts v1.0.2 // indirect
github.com/hashicorp/go-sockaddr v1.0.7 // indirect
github.com/hashicorp/go-uuid v1.0.3 // indirect
github.com/hashicorp/memberlist v0.5.1 // indirect
github.com/hashicorp/memberlist v0.5.2 // indirect
github.com/hashicorp/serf v0.10.1 // indirect
github.com/huandu/xstrings v1.5.0 // indirect
github.com/jcmturner/aescts/v2 v2.0.0 // indirect
Expand Down
7 changes: 4 additions & 3 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -614,6 +614,8 @@ github.com/gorilla/sessions v1.2.1/go.mod h1:dk2InVEVJ0sfLlnXv9EAgkf6ecYs/i80K/z
github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/grafana/ckit v0.0.0-20250109002736-4ca45886e452 h1:d/pdVKdLSNUfHUlWsN39OqUI94XgWKOGJdi568yOXmc=
github.com/grafana/ckit v0.0.0-20250109002736-4ca45886e452/go.mod h1:x6HpYv0+NXPJRBbDYA40IcxWHvrrKwgrMe1Mue172wE=
github.com/grafana/cloudflare-go v0.0.0-20230110200409-c627cf6792f2 h1:qhugDMdQ4Vp68H0tp/0iN17DM2ehRo1rLEdOFe/gB8I=
github.com/grafana/cloudflare-go v0.0.0-20230110200409-c627cf6792f2/go.mod h1:w/aiO1POVIeXUQyl0VQSZjl5OAGDTL5aX+4v0RA1tcw=
github.com/grafana/dskit v0.0.0-20241007172036-53283a0f6b41 h1:a4O59OU3FJZ+EJUVnlvvNTvdAc4uRN1P6EaGwqL9CnA=
Expand Down Expand Up @@ -670,8 +672,8 @@ github.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjh
github.com/hashicorp/go-immutable-radix v1.3.1 h1:DKHmCUm2hRBK510BaiZlwvpD40f8bJFeZnpfm2KLowc=
github.com/hashicorp/go-immutable-radix v1.3.1/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60=
github.com/hashicorp/go-msgpack v0.5.3/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM=
github.com/hashicorp/go-msgpack v1.1.5 h1:9byZdVjKTe5mce63pRVNP1L7UAmdHOTEMGehn6KvJWs=
github.com/hashicorp/go-msgpack v1.1.5/go.mod h1:gWVc3sv/wbDmR3rQsj1CAktEZzoz1YNK9NfGLXJ69/4=
github.com/hashicorp/go-msgpack v0.5.5 h1:i9R9JSrqIz0QVLz3sz+i3YJdT7TTSLcfLLzJi9aZTuI=
github.com/hashicorp/go-msgpack v0.5.5/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM=
github.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk=
github.com/hashicorp/go-multierror v1.1.0/go.mod h1:spPvp8C1qA32ftKqdAHm4hHTbPw+vmowP0z+KUhOZdA=
github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo=
Expand Down Expand Up @@ -1497,7 +1499,6 @@ golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3
golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190424220101-1e8e1cfdf96b/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
Expand Down
32 changes: 32 additions & 0 deletions pkg/analytics/handler.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
package analytics

import (
"encoding/json"
"net/http"
"sync"
"time"
)

var (
seed = &ClusterSeed{}
rw sync.RWMutex
)

func setSeed(s *ClusterSeed) {
rw.Lock()
defer rw.Unlock()
seed = s
}

func Handler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
rw.RLock()
defer rw.RUnlock()
report := buildReport(seed, time.Now())
w.Header().Set("Content-Type", "application/json")
if err := json.NewEncoder(w).Encode(report); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
})
}
5 changes: 2 additions & 3 deletions pkg/analytics/reporter.go
Original file line number Diff line number Diff line change
Expand Up @@ -295,6 +295,7 @@ func (rep *Reporter) running(ctx context.Context) error {
}
return nil
}
setSeed(rep.cluster)
rep.startCPUPercentCollection(ctx, time.Minute)
// check every minute if we should report.
ticker := time.NewTicker(reportCheckInterval)
Expand Down Expand Up @@ -352,9 +353,7 @@ func (rep *Reporter) reportUsage(ctx context.Context, interval time.Time) error

const cpuUsageKey = "cpu_usage"

var (
cpuUsage = NewFloat(cpuUsageKey)
)
var cpuUsage = NewFloat(cpuUsageKey)

func (rep *Reporter) startCPUPercentCollection(ctx context.Context, cpuCollectionInterval time.Duration) {
proc, err := process.NewProcess(int32(os.Getpid()))
Expand Down
7 changes: 2 additions & 5 deletions pkg/compactor/compactor.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ import (
"context"
"flag"
"fmt"
"net/http"
"os"
"path/filepath"
"sort"
Expand Down Expand Up @@ -178,6 +177,7 @@ type Compactor struct {
indexCompactors map[string]IndexCompactor
schemaConfig config.SchemaConfig
tableLocker *tableLocker
limits Limits

// Ring used for running a single compactor
ringLifecycler *ring.BasicLifecycler
Expand Down Expand Up @@ -219,6 +219,7 @@ func NewCompactor(cfg Config, objectStoreClients map[config.DayTime]client.Objec
indexCompactors: map[string]IndexCompactor{},
schemaConfig: schemaConfig,
tableLocker: newTableLocker(),
limits: limits,
}

ringStore, err := kv.NewClient(
Expand Down Expand Up @@ -882,10 +883,6 @@ func (c *Compactor) OnRingInstanceStopping(_ *ring.BasicLifecycler)
func (c *Compactor) OnRingInstanceHeartbeat(_ *ring.BasicLifecycler, _ *ring.Desc, _ *ring.InstanceDesc) {
}

func (c *Compactor) ServeHTTP(w http.ResponseWriter, req *http.Request) {
c.ring.ServeHTTP(w, req)
}

func SortTablesByRange(tables []string) {
tableRanges := make(map[string]model.Interval)
for _, table := range tables {
Expand Down
13 changes: 13 additions & 0 deletions pkg/compactor/deletion/delete_request.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package deletion

import (
"strings"
"time"

"github.com/go-kit/log/level"
Expand Down Expand Up @@ -195,3 +196,15 @@ func intervalsOverlap(interval1, interval2 model.Interval) bool {

return true
}

// GetMatchers returns the string representation of the matchers
func (d *DeleteRequest) GetMatchers() string {
if len(d.matchers) == 0 {
return ""
}
var result []string
for _, m := range d.matchers {
result = append(result, m.String())
}
return strings.Join(result, ",")
}
16 changes: 16 additions & 0 deletions pkg/compactor/deletion/request_handler.go
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,10 @@ func NewDeleteRequestHandler(deleteStore DeleteRequestsStore, maxInterval, delet

// AddDeleteRequestHandler handles addition of a new delete request
func (dm *DeleteRequestHandler) AddDeleteRequestHandler(w http.ResponseWriter, r *http.Request) {
if dm == nil {
http.Error(w, "Retention is not enabled", http.StatusBadRequest)
return
}
ctx := r.Context()
userID, err := tenant.TenantID(ctx)
if err != nil {
Expand Down Expand Up @@ -125,6 +129,10 @@ func (dm *DeleteRequestHandler) interval(params url.Values, startTime, endTime m

// GetAllDeleteRequestsHandler handles get all delete requests
func (dm *DeleteRequestHandler) GetAllDeleteRequestsHandler(w http.ResponseWriter, r *http.Request) {
if dm == nil {
http.Error(w, "Retention is not enabled", http.StatusBadRequest)
return
}
ctx := r.Context()
userID, err := tenant.TenantID(ctx)
if err != nil {
Expand Down Expand Up @@ -219,6 +227,10 @@ func deleteRequestStatus(processed, total int) DeleteRequestStatus {

// CancelDeleteRequestHandler handles delete request cancellation
func (dm *DeleteRequestHandler) CancelDeleteRequestHandler(w http.ResponseWriter, r *http.Request) {
if dm == nil {
http.Error(w, "Retention is not enabled", http.StatusBadRequest)
return
}
ctx := r.Context()
userID, err := tenant.TenantID(ctx)
if err != nil {
Expand Down Expand Up @@ -271,6 +283,10 @@ func filterProcessed(reqs []DeleteRequest) []DeleteRequest {

// GetCacheGenerationNumberHandler handles requests for a user's cache generation number
func (dm *DeleteRequestHandler) GetCacheGenerationNumberHandler(w http.ResponseWriter, r *http.Request) {
if dm == nil {
http.Error(w, "Retention is not enabled", http.StatusBadRequest)
return
}
ctx := r.Context()
userID, err := tenant.TenantID(ctx)
if err != nil {
Expand Down
Loading

0 comments on commit dbf2bef

Please sign in to comment.