Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Concurrent modification error when creating a Service #6837

Closed
rhuss opened this issue Feb 13, 2020 · 4 comments
Closed

Concurrent modification error when creating a Service #6837

rhuss opened this issue Feb 13, 2020 · 4 comments
Labels
area/API API objects and controllers kind/bug Categorizes issue or PR as related to a bug.
Milestone

Comments

@rhuss
Copy link
Contributor

rhuss commented Feb 13, 2020

What version of Knative?

0.12.0

Expected Behavior

We would expect that the creation of a Service works without error

Actual Behavior

Sometimes (not always), when creating a very simple service like with kn service create hello --image gcr.io/knative-samples/helloworld-go we end up with the following events for the created Knative Service:

Events:
  Type     Reason         Age                From                Message
  ----     ------         ----               ----                -------
  Normal   Created        10m                service-controller  Created Configuration "hello"
  Normal   Created        10m                service-controller  Created Route "hello"
  Normal   Updated        10m (x3 over 10m)  service-controller  Updated Service "hello"
  Warning  InternalError  10m                service-controller  failed to reconcile Configuration: Operation cannot be fulfilled on configurations.serving.knative.dev "hello": the object has been modified; please apply your changes to the latest version and try again 

This ends up in the following service conditions:

 status:
  conditions:
  - lastTransitionTime: "2020-02-12T11:17:47Z"
    status: Unknown
    type: ConfigurationsReady
  - lastTransitionTime: "2020-02-12T11:17:48Z"
    message: Revision "hello-hnzcf-1" referenced in traffic not found.
    reason: RevisionMissing
    status: "False"
    type: Ready
  - lastTransitionTime: "2020-02-12T11:17:48Z"
    message: Revision "hello-hnzcf-1" referenced in traffic not found.
    reason: RevisionMissing
    status: "False"
    type: RoutesReady 

It looks like that an (internal) update of the Configuration clashes with the Configuration's reconciliation. I don't know though, why the Configuration (or service) needs to be updated.

We are using BYO revision name, so that might be the cause of this extra Service / Configuration update causing the collision.

See this client E2E log for a more complete dump of the situation.

Steps to Reproduce the Problem

It's flaky, but just try kn service create hello --image gcr.io/knative-samples/helloworld-go and in some case you get that behaviour (kn uses BYO revision names by default)

@rhuss rhuss added the kind/bug Categorizes issue or PR as related to a bug. label Feb 13, 2020
@knative-prow-robot knative-prow-robot added the area/API API objects and controllers label Feb 13, 2020
@navidshaikh
Copy link

Seeing following event as well with same error Message: Revision "hello-nxvfy-1" referenced in traffic not found.

Events:
  Type     Reason         Age              From                Message
  ----     ------         ----             ----                -------
  Warning  InternalError  5s (x8 over 5s)  service-controller  service: "hello" does not own configuration: "hello

client E2E logs

@dprotaso dprotaso added this to the Serving 0.14.x milestone Mar 4, 2020
@dprotaso dprotaso modified the milestones: Serving 0.16.x, 0.17.x Jul 8, 2020
@mattmoor mattmoor modified the milestones: 0.17.x, 0.18.x Aug 31, 2020
@markusthoemmes
Copy link
Contributor

This has been forwarded since 0.14. @rhuss @navidshaikh are you still seeing this?

@navidshaikh
Copy link

@markusthoemmes : nope, didnt see this lately.

@rhuss
Copy link
Contributor Author

rhuss commented Sep 2, 2020

Me neither. I think we are good to close this and create a new issue in case this should come back.

@rhuss rhuss closed this as completed Sep 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/API API objects and controllers kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

No branches or pull requests

6 participants