Skip to content

Commit

Permalink
feat: remove charts directory
Browse files Browse the repository at this point in the history
  • Loading branch information
pkoutsovasilis committed Jan 15, 2025
1 parent e82728e commit 4f23d9c
Show file tree
Hide file tree
Showing 19 changed files with 226 additions and 11 deletions.
2 changes: 2 additions & 0 deletions deploy/helm/elastic-agent/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Chart.lock
./charts
6 changes: 0 additions & 6 deletions deploy/helm/elastic-agent/Chart.lock

This file was deleted.

Binary file not shown.
6 changes: 6 additions & 0 deletions deploy/helm/elastic-agent/examples/eck/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,12 @@ In this example we install the built-in `kubernetes` integration with the defaul
4. Open https://localhost:12000 in your browser
5. Install kubernetes integration through Kibana

4. Build the dependencies of the Helm chart
```console
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm dependency build ../../
```

## Run:
```console
helm install elastic-agent ../../ \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@

In this example we will perform two Helm chart installations, one installing elastic-agent as a Daemonset and the other installing kube-state-metrics with the `autosharding` feature enabled and elastic-agent as a sidecar container. All the agents are managed by [Fleet](https://www.elastic.co/guide/en/fleet/current/manage-agents-in-fleet.html). Such a type of setup is recommended for big k8s clusters, featuring a lot of k8s resources, where scaling of kube-state-metrics extraction is required.

## Prerequisites:
1. Build the dependencies of the Helm chart
```console
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm dependency build ../../
```

## Run:

1. Follow [this guide](https://www.elastic.co/guide/en/fleet/current/install-fleet-managed-elastic-agent.html#elastic-agent-installation-steps) to set up an agent policy and enroll an agent to it. In the policy unselect the "Collect system logs and metrics" options and continue to agent enrollment. Do not download any binary, from the proposed enrollment command just extract the Fleet URL (`--url=$FLEET_URL`) and Enrollment token (`--enrollment-token=$FLEET_TOKEN`).
Expand Down
6 changes: 6 additions & 0 deletions deploy/helm/elastic-agent/examples/fleet-managed/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@ In this example we deploy an Elastic Agent that is managed by [Fleet](https://ww
## Prerequisites:
1. Follow [this guide](https://www.elastic.co/guide/en/fleet/current/install-fleet-managed-elastic-agent.html#elastic-agent-installation-steps) to set up an agent policy and enroll an agent to it. Do not download any binary, from the proposed enrollment command just extract the Fleet URL (`--url=$FLEET_URL`) and Enrollment token (`--enrollment-token=$FLEET_TOKEN`).

2. Build the dependencies of the Helm chart
```console
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm dependency build ../../
```

## Run:

```console
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,19 @@
In this example we install the built-in `kubernetes` integration with the default built-in values.

## Prerequisites:
1. A k8s secret that contains the connection details to an Elasticsearch cluster such as the URL and the API key ([Kibana - Creating API Keys](https://www.elastic.co/guide/en/kibana/current/api-keys.html)):
1. Build the dependencies of the Helm chart
```console
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm dependency build ../../
```
2. A k8s secret that contains the connection details to an Elasticsearch cluster such as the URL and the API key ([Kibana - Creating API Keys](https://www.elastic.co/guide/en/kibana/current/api-keys.html)):
```console
kubectl create secret generic es-api-secret \
--from-literal=api_key=... \
--from-literal=url=...
```

2. `kubernetes` integration assets installed through Kibana ([Kibana - Install and uninstall Elastic Agent integration assets](https://www.elastic.co/guide/en/fleet/current/install-uninstall-integration-assets.html))
3. `kubernetes` integration assets installed through Kibana ([Kibana - Install and uninstall Elastic Agent integration assets](https://www.elastic.co/guide/en/fleet/current/install-uninstall-integration-assets.html))

## Run:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,11 @@ In this example we install the built-in `kubernetes` integration and enable the

2. `redis` integration assets are installed through Kibana ([Kibana - Install and uninstall Elastic Agent integration assets](https://www.elastic.co/guide/en/fleet/current/install-uninstall-integration-assets.html))

3. Build the dependencies of the Helm chart
```console
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm dependency build ../../
```
## Run:
1. Install Helm chart
```console
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,11 @@ In this example we install the built-in `kubernetes` integration with the defaul

2. `kubernetes` integration assets installed through Kibana ([Kibana - Install and uninstall Elastic Agent integration assets](https://www.elastic.co/guide/en/fleet/current/install-uninstall-integration-assets.html))

3. Build the dependencies of the Helm chart
```console
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm dependency build ../../
```
## Run:

#### Public image registry:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,12 @@ In this example we install the built-in `kubernetes` integration and set it to e

2. `kubernetes` integration assets installed through Kibana ([Kibana - Install and uninstall Elastic Agent integration assets](https://www.elastic.co/guide/en/fleet/current/install-uninstall-integration-assets.html))

3. Build the dependencies of the Helm chart
```console
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm dependency build ../../
```

## Run:
```console
helm install elastic-agent ../../ \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,12 @@ In this example we install the built-in `kubernetes` integration and a `nginx` c

2. `kubernetes`, `redis`, and `nginx` integration assets are installed through Kibana ([Kibana - Install and uninstall Elastic Agent integration assets](https://www.elastic.co/guide/en/fleet/current/install-uninstall-integration-assets.html))

3. Build the dependencies of the Helm chart
```console
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm dependency build ../../
```

## Run:
1. Install Helm chart
```console
Expand Down
6 changes: 6 additions & 0 deletions deploy/helm/elastic-agent/examples/netflow-service/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,12 @@ In this example we define a `netflow` custom integration alongside a custom agen

2. `NetFlow Records` integration assets are installed through Kibana

3. Build the dependencies of the Helm chart
```console
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm dependency build ../../
```

## Run:
1. Install Helm chart
```console
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,12 @@ In this example we define a `nginx` custom integration alongside a custom agent

2. `nginx` integration assets are installed through Kibana

3. Build the dependencies of the Helm chart
```console
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm dependency build ../../
```

## Run:
1. Install Helm chart
```console
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,12 @@ In this example we install the built-in `system` integration and specify custom

2. `system` integration assets installed through Kibana ([Kibana - Install and uninstall Elastic Agent integration assets](https://www.elastic.co/guide/en/fleet/current/install-uninstall-integration-assets.html))

3. Build the dependencies of the Helm chart
```console
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm dependency build ../../
```

## Run:
```console
helm install elastic-agent ../../ \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,12 @@ In this example we define a `nginx` custom integration alongside a custom agent
kubectl create clusterrole user-cr --verb=get,list,watch --resource=pods,namespaces,nodes,replicasets,jobs
```

4. Build the dependencies of the Helm chart
```console
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm dependency build ../../
```

## Run:
1. Install Helm chart
```console
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,12 @@ In this example we install the built-in `kubernetes` integration with the defaul
kubectl create serviceaccount user-sa
```

4. Build the dependencies of the Helm chart
```console
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm dependency build ../../
```

## Run:
```console
helm install elastic-agent ../../ \
Expand Down
13 changes: 11 additions & 2 deletions magefile.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ import (
tcommon "github.com/elastic/elastic-agent/pkg/testing/common"
"github.com/elastic/elastic-agent/pkg/testing/define"
"github.com/elastic/elastic-agent/pkg/testing/ess"
"github.com/elastic/elastic-agent/pkg/testing/helm"
"github.com/elastic/elastic-agent/pkg/testing/kubernetes/kind"
"github.com/elastic/elastic-agent/pkg/testing/multipass"
"github.com/elastic/elastic-agent/pkg/testing/ogc"
Expand Down Expand Up @@ -3413,7 +3414,9 @@ type Helm mg.Namespace
// RenderExamples runs the equivalent of `helm template` and `helm lint`
// for the examples of the Elastic Helm chart which are located at
// `deploy/helm/elastic-agent/examples` directory.
func (Helm) RenderExamples() error {
func (h Helm) RenderExamples() error {
mg.SerialDeps(h.BuildDependencies)

settings := cli.New() // Helm CLI settings
actionConfig := &action.Configuration{}

Expand Down Expand Up @@ -3530,7 +3533,9 @@ func (Helm) UpdateAgentVersion() error {
}

// Lint lints the Elastic-Agent Helm chart.
func (Helm) Lint() error {
func (h Helm) Lint() error {
mg.SerialDeps(h.BuildDependencies)

settings := cli.New() // Helm CLI settings
actionConfig := &action.Configuration{}

Expand Down Expand Up @@ -3596,6 +3601,10 @@ func updateYamlFile(path string, keyVal ...struct {
return nil
}

func (Helm) BuildDependencies() error {
return helm.BuildChartDependencies(helmChartPath)
}

func updateYamlNodes(rootNode *yaml.Node, value string, keys ...string) error {
if len(keys) == 0 {
return fmt.Errorf("no keys provided")
Expand Down
130 changes: 130 additions & 0 deletions pkg/testing/helm/helm.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
// Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
// or more contributor license agreements. Licensed under the Elastic License 2.0;
// you may not use this file except in compliance with the Elastic License 2.0.

package helm

import (
"bufio"
"bytes"
"fmt"
"os"

"gopkg.in/yaml.v2"
"helm.sh/helm/v3/pkg/action"
"helm.sh/helm/v3/pkg/cli"
"helm.sh/helm/v3/pkg/downloader"
"helm.sh/helm/v3/pkg/getter"
"helm.sh/helm/v3/pkg/registry"
"helm.sh/helm/v3/pkg/repo"
)

func ensureRepository(repoName, repoURL string, settings *cli.EnvSettings) error {
repoFile := settings.RepositoryConfig
// Load existing repositories
file, err := repo.LoadFile(repoFile)
if err != nil {
return fmt.Errorf("could not load Helm repository config: %w", err)
}

// Check if the repository is already added
for _, entry := range file.Repositories {
if entry.URL == repoURL {
// repository already exists
return nil
}
}

// Add the repository
entry := &repo.Entry{
Name: repoName,
URL: repoURL,
}

chartRepo, err := repo.NewChartRepository(entry, getter.All(settings))
if err != nil {
return fmt.Errorf("could not create repo %s: %w", repoURL, err)
}

_, err = chartRepo.DownloadIndexFile()
if err != nil {
return fmt.Errorf("could not download index file for repo %s: %w", repoURL, err)
}

file.Update(entry)
if err := file.WriteFile(repoFile, 0o644); err != nil {
return fmt.Errorf("could not write Helm repository config: %w", err)
}

return nil
}

func BuildChartDependencies(chartPath string) error {
settings := cli.New()
settings.SetNamespace("")
actionConfig := &action.Configuration{}

chartFile, err := os.ReadFile(fmt.Sprintf("%s/Chart.yaml", chartPath))
if err != nil {
return fmt.Errorf("could not read %s/Chart.yaml: %w", chartPath, err)
}

dependencies := struct {
Entry []struct {
Name string `yaml:"name"`
Repository string `yaml:"repository"`
} `yaml:"dependencies"`
}{}

err = yaml.Unmarshal(chartFile, &dependencies)
if err != nil {
return fmt.Errorf("could not unmarshal %s/Chart.yaml: %w", chartPath, err)
}

for _, dep := range dependencies.Entry {
err := ensureRepository(dep.Name, dep.Repository, settings)
if err != nil {
return err
}
}

err = actionConfig.Init(settings.RESTClientGetter(), settings.Namespace(), "",
func(format string, v ...interface{}) {})
if err != nil {
return fmt.Errorf("failed to init helm action config: %w", err)
}

client := action.NewDependency()

registryClient, err := registry.NewClient(
registry.ClientOptDebug(settings.Debug),
registry.ClientOptEnableCache(true),
registry.ClientOptWriter(os.Stderr),
registry.ClientOptCredentialsFile(settings.RegistryConfig),
)
if err != nil {
return fmt.Errorf("failed to create helm registry client: %w", err)
}

buffer := bytes.Buffer{}

man := &downloader.Manager{
Out: bufio.NewWriter(&buffer),
ChartPath: chartPath,
Keyring: client.Keyring,
SkipUpdate: true,
Getters: getter.All(settings),
RegistryClient: registryClient,
RepositoryConfig: settings.RepositoryConfig,
RepositoryCache: settings.RepositoryCache,
Debug: settings.Debug,
}
if client.Verify {
man.Verify = downloader.VerifyIfPossible
}
err = man.Build()
if err != nil {
return fmt.Errorf("failed to build helm dependencies: %w", err)
}
return nil
}
6 changes: 5 additions & 1 deletion testing/integration/kubernetes_agent_standalone_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ import (
aclient "github.com/elastic/elastic-agent/pkg/control/v2/client"
atesting "github.com/elastic/elastic-agent/pkg/testing"
"github.com/elastic/elastic-agent/pkg/testing/define"
"github.com/elastic/elastic-agent/pkg/testing/helm"
"github.com/elastic/elastic-agent/pkg/testing/tools/fleettools"
)

Expand Down Expand Up @@ -248,8 +249,11 @@ func TestKubernetesAgentHelm(t *testing.T) {
ctx := context.Background()
kCtx := k8sGetContext(t, info)

err := helm.BuildChartDependencies(agentK8SHelm)
require.NoError(t, err, "failed to build helm dependencies")

nodeList := corev1.NodeList{}
err := kCtx.client.Resources().List(ctx, &nodeList)
err = kCtx.client.Resources().List(ctx, &nodeList)
require.NoError(t, err)

schedulableNodeCount, err := k8sSchedulableNodeCount(ctx, kCtx)
Expand Down

0 comments on commit 4f23d9c

Please sign in to comment.