Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

Prevent updating Tyk regardless of changes in each reconciliation #571

Merged
merged 32 commits into from
Mar 3, 2023

Conversation

buraksekili
Copy link
Collaborator

@buraksekili buraksekili commented Feb 22, 2023

Description

Users raised possible performance issues in the reconciliation cycle of Tyk Operator. While reconciling a large amount of resources, Operator makes redundant external API calls to Tyk Gateway / Dashboard. This PR compares hashes of an existing resource and an updated resource and prevents calling Update API of GW/Dashboard if no changes are made.

Related Issue

https://tyktech.atlassian.net/browse/TT-7610

Motivation and Context

By default, Operator triggers reconciliation in 10hrs approximately. So, if you have a large number of resources in your cluster, Operator tries to reconcile each of your resources in the cluster even though you haven't changed anything. With the current flow, Operator sends Update requests to Tyk Gateway / Dashboard regardless of changes in resources. This causes issues if your GW is running under certain resource limitations.

Test Coverage For This Change

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If PRing from your fork, don't come from your master!
  • Make sure you are making a pull request against our master branch (left side). Also, it would be best if you started your change off our latest master.
  • Make sure you are updating CHANGELOG.md based on your changes.
  • My change requires a change to the documentation.
    • If you've changed APIs, describe what needs to be updated in the documentation.
  • I have updated the documentation accordingly.
  • If you've changed API models, please update CRDs.
    • make manifests
    • make helm
  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • Check your code additions will not fail linting checks:
    • gofmt -s -w .
    • go vet ./...
    • golangci-lint run

Signed-off-by: Burak Sekili <buraksekili@gmail.com>
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
…kTechnologies/tyk-operator into fix/TT-6389/recreate-nonexistent-apis
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
…timization

Signed-off-by: Burak Sekili <buraksekili@gmail.com>
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
@buraksekili buraksekili marked this pull request as ready for review February 23, 2023 08:50
@buraksekili buraksekili requested a review from a team as a code owner February 23, 2023 08:50
@buraksekili buraksekili requested review from komalsukhani and andrei-tyk and removed request for a team February 23, 2023 08:50
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
…TykTechnologies/tyk-operator into fix/TT-7610/reconciliation-optimization
… cases since the hashing logic is moved to controllers

Signed-off-by: Burak Sekili <buraksekili@gmail.com>
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
…break links between ApiDefinition CRs and other resources such as SecurityPolicy CRs

Signed-off-by: Burak Sekili <buraksekili@gmail.com>
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
…gh no changes happen based on hashing result

Signed-off-by: Burak Sekili <buraksekili@gmail.com>
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
Signed-off-by: Burak Sekili <buraksekili@gmail.com>
@sonarqubecloud
Copy link

sonarqubecloud bot commented Mar 2, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

9.8% 9.8% Coverage
0.0% 0.0% Duplication

@singhpr singhpr merged commit 0ae1ed1 into master Mar 3, 2023
@singhpr singhpr deleted the fix/TT-7610/reconciliation-optimization branch March 3, 2023 10:43
buger pushed a commit that referenced this pull request May 22, 2024
* Recreate nonexistent ApiDefinitions during update calls

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Update CHANGELOG.md

* Replace curl requests with k8s port-forwards

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Open connection in init function

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Update changelog

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Check hashes of ApiDefinitions before updating them

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Update test cases based on new update call

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Remove redundant log messages

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Update CHANGELOG based on new changes

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Move hashing logic to reconciler

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Add test cases for hashing helper function. Bring back to client test cases since the hashing logic is moved to controllers

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Fix linter in test cases

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Add simple test case to check Update functionality

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Add ignore hashes to k8s specific fields

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Revert client test cases to initial version

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Ignore operatorContext

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Add hashing mechanism to Policy controller as well

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Remove extra line to fix linter

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Send reload request before manipulating k8s status

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Remove predicate function to disable subresource updates - this will break links between ApiDefinition CRs and other resources such as SecurityPolicy CRs

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Revert latest commit

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Update securitypolicy controller. so that, it updates links even though no changes happen based on hashing result

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* Stop running tests in unsupported versions

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

* change parameter type to pointer

Signed-off-by: Burak Sekili <buraksekili@gmail.com>

---------

Signed-off-by: Burak Sekili <buraksekili@gmail.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants