Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Doctrine] add support to convert Doctrine events to Enqueue messages #68

Closed
dkarlovi opened this issue May 4, 2017 · 3 comments
Closed

Comments

@dkarlovi
Copy link
Contributor

dkarlovi commented May 4, 2017

Doctrine2 has lifecycle events which trigger on create, update, delete. We can hook into those events and push a new message with enough information a processor might do something with it (for example, clear cache, run webhooks, run long running jobs which waited for that entity to reach that state in the attached state machine, etc).

I've created a sample implementation with usage example from Symfony container (disregard the userToken stuff in the processor, that's a hacky implementation for which I'll open another issue).

It works like this (C/P from gitter):

  1. you create an entity, persist it
  2. Doctrine triggers a postPersist, passing the entity
  3. DoctrineEventConverter creates a new DoctrineEventMessage from the Doctrine event
    (passing event type, entity class and entity PK)
  4. TestProcessor attaches a user token to the message before sending it
    (this is temporary, need a better way to do it, maybe an extension of some sort?)
  5. DoctrineEventConverter sends the message via the producer
  6. <transport>
  7. worker running enqueue:consume picks up the message, passes it back to DoctrineEventConverter
  8. DoctrineEventConverter creates a new DoctrineEventMessage from the PSR message
  9. TestProcessor pulls the user token from the message, stores it to its own tokenStorage and also to gedmo (temporary, also)
  10. DoctrineEventConverter fetches the entity by class+ID, stores it to DoctrineEventMessage and passes it to TestProcessor
  11. TestProcessor appends the string " processed" to the entity name
  12. Doctrine triggers a postUpdate, passing the entity
  13. we start from 3) again, but this time the origin is the worker, not ourselves
  14. if anybody was listening for postUpdate, it would trigger here again (so you can chain events)
  15. it stops once all processors decline to do an entity change
@dkarlovi
Copy link
Contributor Author

dkarlovi commented May 11, 2017

Just a reminder for myself: this approach does NOT work.

If you attach two listeners to postPersist, seeing as the RabbitMQ exchange is "fanout", there will be a Cartesian amount of messages (N = number of listeners) because: N times sent (by each listener) x N times delivered (by RabbitMQ + fanout) = N^2 deliveries.

Doctrine events need to be converted to Enqueue messages from a single dedicated service which triggers all the events (for all listeners).

@makasim
Copy link
Member

makasim commented Aug 28, 2017

This could be implemented as a standalone package. but I prefer not to add such stuff directly in this repo

@makasim makasim closed this as completed Aug 28, 2017
@dkarlovi
Copy link
Contributor Author

@makasim fair enough, but I don't think it's any more involved than Doctrine extensions in the bundle.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants