[8.16](backport #6260) [k8s]: restructure kubernetes integration tests

[mergify]

What does this PR do?

This PR lays the groundwork for Kubernetes hints integration tests. The related commits of the former will be included in a follow-up PR (6fa8ffb, 9a48d02) as I want to keep this PR around 500 lines (excluding go.mod and NOTICE files). These changes address the need to avoid duplicating code logic, as hints tests require deploying plain Kubernetes YAML files and checking the agent status independently. Key updates include:

• Centralized Context: Introduced k8sContext to standardize Kubernetes test setup.
• Refactored Helpers: Streamlined functions for Kubernetes operations (e.g., k8sCreateObjects, k8sDeleteObjects, k8sWaitForReady) with better error handling.
• Agent Health Validation: Enhanced agent status checks with k8sCheckAgentStatus.
• Readiness Checker: Leveraged Helm's ReadyChecker to validate readiness of Kubernetes objects (e.g., pods, deployments, daemonSets, statefulSets, etc.).

These updates improve code readability, modularity, and reusability, reducing the need for duplicate logic in k8s tests.

Why is it important?

• Avoiding Duplication: Eliminates the need for repeated logic in k8s tests.
• Improved Modularity: Encapsulates reusable functionality for easier maintenance and test extension.
• Readiness Assurance: Ensures Kubernetes objects are fully ready before test execution, reducing test flakiness.

Checklist

• My code follows the style guidelines of this project.
• I have commented my code, particularly in hard-to-understand areas.
• [ ] I have made corresponding changes to the documentation.
• [ ] I have made corresponding changes to the default configuration files.
• I have added tests that prove my fix is effective or that my feature works.
• [ ] I have added an entry in ./changelog/fragments using the changelog tool.
• I have added an integration test or an E2E test.

Disruptive User Impact

No disruptive changes. Enhancements are internal to Kubernetes integration tests.

How to test this PR locally

mage integration:auth
PLATFORMS=linux/arm64 EXTERNAL=true SNAPSHOT=true PACKAGES=docker mage -v package 
INSTANCE_PROVISIONER=kind STACK_PROVISIONER=stateful K8S_VERSION=v1.31.1 SNAPSHOT=true mage integration:kubernetes

Related issues

• Relates Include hints autodiscovery configuration inside integration tests for k8s #5660

  ---

  This is an automatic backport of pull request [k8s]: restructure kubernetes integration tests #6260 done by Mergify.

[mergify]

Cherry-pick of 79caf55 has failed:

On branch mergify/bp/8.16/pr-6260
Your branch is up to date with 'origin/8.16'.

You are currently cherry-picking commit 79caf557d.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   NOTICE.txt
	modified:   testing/integration/kubernetes_agent_service_test.go
	modified:   testing/integration/kubernetes_agent_standalone_test.go

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)
	both modified:   go.mod
	deleted by us:   testing/integration/otel_helm_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

[pkoutsovasilis]

requires this one to be merged first

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube