Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implemented an load configuration YAML #36

Merged
merged 3 commits into from
Dec 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ In real-world applications, the complexity and cost can escalate quickly. `K8sBl

### Example Configuration Tasks

#### JSON:

```json
[
{
Expand Down Expand Up @@ -71,6 +73,44 @@ In real-world applications, the complexity and cost can escalate quickly. `K8sBl
]
```

#### YAML:
```yaml
- name: "list-specific-pods"
type: "GetPods"
parameters:
labelSelector: "app=nginx"
fieldSelector: "status.phase=Running"
limit: 10

- name: "list-specific-pods-run"
type: "CrewGetPodsTaskRunner"
parameters:
labelSelector: "app=nginx"
fieldSelector: "status.phase=Running"
limit: 10

- name: "check-health-pods"
type: "CrewCheckHealthPods"
parameters:
labelSelector: "app=nginx"
fieldSelector: "status.phase=Running"
limit: 10

- name: "label-all-pods"
type: "CrewWriteLabelPods"
parameters:
labelKey: "environment"
labelValue: "production"

- name: "update-specific-pod"
type: "CrewWriteLabelPods"
parameters:
podName: "pod-name"
labelKey: "environment"
labelValue: "production"

```

> [!NOTE]
> Support Multiple-Task and a lot's of worker

Expand Down Expand Up @@ -154,6 +194,9 @@ In real-world applications, the complexity and cost can escalate quickly. `K8sBl
> [!NOTE]
> This specialized feature has been successfully integrated.

## Load Configuration Task
- [x] **Load Configuration**: Enhanced the application to load task configurations from a YAML file, improving ease of use and configurability.

## Pod Labeling Logic Enhancement
- [x] **Optimized Pod Labeling**:
- Implemented an optimized pod labeling process that checks existing labels and updates them only if necessary, reducing the number of API calls and improving overall performance.
Expand Down
30 changes: 23 additions & 7 deletions worker/docs.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// Package worker provides a set of tools designed to facilitate the interaction with
// Kubernetes resources from within a cluster. It offers a convenient abstraction for
// managing Kubernetes operations, focusing on pod health checks, pod labeling, and structured logging.
// managing Kubernetes operations, focusing on pod health checks, pod labeling, structured logging,
// and task configuration through YAML or JSON files.
//
// The package is intended for applications running as pods within Kubernetes clusters
// and leverages in-cluster configuration to establish a clientset for API interactions.
Expand All @@ -20,15 +21,21 @@
// when necessary, reducing API calls and improving performance. It also includes
// retry logic to handle intermittent API errors.
//
// - Configuration loading from YAML files has been added, enhancing the flexibility
// and configurability of task management within the worker processes.
//
// # Functions
//
// - NewKubernetesClient: Creates a new Kubernetes clientset configured for in-cluster
// communication with the Kubernetes API server.
//
// - CrewWorker: Orchestrates a worker process to perform tasks such as health checks
// and labeling of pods within a specified namespace. It includes retry logic to handle transient
// errors and respects cancellation and timeout contexts. Structured logging is used
// to provide detailed contextual information.
// - CrewWorker: Orchestrates a worker process to perform tasks such as health checks,
// labeling of pods, and other configurable tasks within a specified namespace. It includes
// retry logic to handle transient errors and respects cancellation and timeout contexts.
// Structured logging is used to provide detailed contextual information.
//
// - LoadTasksFromYAML: Loads task configurations from a YAML file, allowing for
// dynamic task management based on external configuration.
//
// - CrewGetPods: Retrieves all pods within a given namespace, logging the attempt
// and outcome of the operation.
Expand All @@ -47,7 +54,8 @@
// Initialize the Kubernetes client using NewKubernetesClient, then leverage the client
// to perform operations such as retrieving and processing pods within a namespace.
// Contexts are used to manage the lifecycle of the worker processes, including graceful
// shutdowns and cancellation.
// shutdowns and cancellation. Task configurations can be loaded from a YAML file for
// enhanced flexibility.
//
// Example:
//
Expand All @@ -59,8 +67,13 @@
// ctx, cancel := context.WithCancel(context.Background())
// defer cancel() // Ensure cancellation is called to free resources
//
// tasks, err := worker.LoadTasksFromYAML("tasks.yaml")
// if err != nil {
// // Handle error
// }
//
// resultsChan := make(chan string)
// go worker.CrewWorker(ctx, clientset, namespace, resultsChan)
// go worker.CrewWorker(ctx, clientset, namespace, tasks, resultsChan)
//
// // Process results as they come in
// for result := range resultsChan {
Expand All @@ -81,6 +94,9 @@
// - Pod Labeling Logic has been enhanced to perform more efficiently by minimizing
// unnecessary API calls, and it now includes robust error handling and retry mechanisms.
//
// - Configuration management has been improved by enabling the loading of task
// configurations from YAML files, offering greater versatility and ease of use.
//
// # TODO
//
// - Extend the functionality of the CrewWorker function to support a wider range
Expand Down
19 changes: 19 additions & 0 deletions worker/tasks_crew.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import (
"github.com/H0llyW00dzZ/K8sBlackPearl/navigator"
"github.com/H0llyW00dzZ/go-urlshortner/logmonitor/constant"
"go.uber.org/zap"
"gopkg.in/yaml.v2"
corev1 "k8s.io/api/core/v1"
v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
Expand Down Expand Up @@ -223,3 +224,21 @@ func LoadTasksFromJSON(filePath string) ([]Task, error) {

return tasks, nil
}

// LoadTasksFromYAML reads a YAML file containing an array of Task objects, unmarshals it,
// and returns a slice of Task structs. It returns an error if the file cannot be read or
// the YAML cannot be unmarshaled into the Task structs.
func LoadTasksFromYAML(filePath string) ([]Task, error) {
file, err := os.ReadFile(filePath)
if err != nil {
return nil, err
}

var tasks []Task
err = yaml.Unmarshal(file, &tasks)
if err != nil {
return nil, err
}

return tasks, nil
}