Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add delivery policy to (HTTP) Triggers #583

Merged
merged 8 commits into from
Nov 11, 2022
Merged

Add delivery policy to (HTTP) Triggers #583

merged 8 commits into from
Nov 11, 2022

Conversation

Annopaolo
Copy link
Collaborator

Configure triggers to allow redelivery of payloads upon HTTP errors.
See #554.

@Annopaolo Annopaolo requested review from bettio and rbino May 13, 2021 09:57
@codecov
Copy link

codecov bot commented May 13, 2021 •
edited
Loading

Codecov Report

Merging #583 (055bfa5) into master (b224992) will decrease coverage by 0.29%.
The diff coverage is 64.16%.

❗ Current head 055bfa5 differs from pull request most recent head ea6d298. Consider uploading reports for the commit ea6d298 to get more accurate results

@@            Coverage Diff             @@
##           master     #583      +/-   ##
==========================================
- Coverage   68.80%   68.51%   -0.30%     
==========================================
  Files         247      258      +11     
  Lines        5527     5904     +377     
==========================================
+ Hits         3803     4045     +242     
- Misses       1724     1859     +135
Impacted Files Coverage Δ
...gement/lib/astarte_realm_management/rpc/handler.ex 16.98% <0.00%> (-3.48%) ⬇️
...b/astarte_realm_management_api/triggers/trigger.ex 7.14% <0.00%> (-0.55%) ⬇️
.../astarte_realm_management_api/triggers/triggers.ex 0.00% <0.00%> (ø)
...agement_api_web/controllers/fallback_controller.ex 35.71% <0.00%> (-9.75%) ⬇️
...t_api_web/controllers/trigger_policy_controller.ex 0.00% <0.00%> (ø)
...api/lib/astarte_realm_management_api_web/router.ex 75.00% <ø> (ø)
...tarte_realm_management_api_web/views/error_view.ex 58.33% <0.00%> (-19.45%) ⬇️
...lm_management_api_web/views/trigger_policy_view.ex 0.00% <0.00%> (ø)
...r_engine/lib/astarte_trigger_engine/application.ex 100.00% <ø> (ø)
...gine/lib/astarte_trigger_engine/events_consumer.ex 0.00% <0.00%> (ø)
... and 19 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@Annopaolo Annopaolo force-pushed the delivery_policy branch 6 times, most recently from a5d2b6e to 4bce3fd Compare May 18, 2021 14:03
@Annopaolo Annopaolo marked this pull request as ready for review May 26, 2021 15:41
@Annopaolo Annopaolo added app:trigger_engine This issue or pull request is about astarte_trigger_engine application enhancement New feature or request labels Sep 29, 2021
rbino
rbino requested changes Sep 30, 2022
View reviewed changes
apps/astarte_data_updater_plant/lib/astarte_data_updater_plant/triggers/policy_retriever.ex Outdated Show resolved Hide resolved
apps/astarte_trigger_engine/test/test_helper.exs Outdated Show resolved Hide resolved
apps/astarte_trigger_engine/lib/astarte_trigger_engine/policy/policy.ex Outdated Show resolved Hide resolved
apps/astarte_trigger_engine/lib/astarte_trigger_engine/config.ex Outdated Show resolved Hide resolved
apps/astarte_data_updater_plant/lib/astarte_data_updater_plant/triggers/policy_retriever.ex Outdated Show resolved Hide resolved
apps/astarte_trigger_engine/lib/astarte_trigger_engine/amqp_consumer/amqp_events_consumer.ex Outdated Show resolved Hide resolved
apps/astarte_trigger_engine/lib/astarte_trigger_engine/amqp_consumer/amqp_events_consumer.ex Outdated Show resolved Hide resolved
apps/astarte_trigger_engine/lib/astarte_trigger_engine/amqp_consumer/amqp_events_consumer.ex Outdated Show resolved Hide resolved
apps/astarte_trigger_engine/lib/astarte_trigger_engine/amqp_consumer/amqp_events_consumer.ex Outdated Show resolved Hide resolved
apps/astarte_trigger_engine/lib/astarte_trigger_engine/amqp_consumer/amqp_consumer_tracker.ex Outdated Show resolved Hide resolved
@Annopaolo Annopaolo marked this pull request as draft October 13, 2022 15:25
@Annopaolo Annopaolo force-pushed the delivery_policy branch 6 times, most recently from 8f1bb94 to e05b155 Compare October 18, 2022 15:09
@Annopaolo Annopaolo force-pushed the delivery_policy branch 2 times, most recently from 040817c to c5d37e7 Compare October 19, 2022 16:13
@Annopaolo
Copy link
Collaborator Author

Annopaolo commented Oct 19, 2022 •
edited
Loading

Merge is blocked until astarte-platform/astarte_core#62 and astarte-platform/astarte_rpc#45 are merged.

We're ready!

@Annopaolo Annopaolo force-pushed the delivery_policy branch 3 times, most recently from 385958f to e1b3a05 Compare October 21, 2022 12:29
@Annopaolo Annopaolo requested review from rbino and bettio November 10, 2022 15:01
Pavinati
Pavinati requested changes Nov 11, 2022
View reviewed changes
apps/astarte_realm_management_api/priv/static/astarte_realm_management_api.yaml Outdated Show resolved Hide resolved
apps/astarte_realm_management_api/priv/static/astarte_realm_management_api.yaml Outdated Show resolved Hide resolved
apps/astarte_realm_management_api/priv/static/astarte_realm_management_api.yaml Outdated Show resolved Hide resolved
apps/astarte_realm_management_api/priv/static/astarte_realm_management_api.yaml Outdated Show resolved Hide resolved
apps/astarte_realm_management_api/priv/static/astarte_realm_management_api.yaml Outdated Show resolved Hide resolved
doc/pages/user/062-using_trigger_delivery_policies.md Outdated Show resolved Hide resolved
doc/pages/user/062-using_trigger_delivery_policies.md Outdated Show resolved Hide resolved
doc/pages/user/062-using_trigger_delivery_policies.md Outdated Show resolved Hide resolved
doc/pages/user/060-using_triggers.md Outdated Show resolved Hide resolved
doc/pages/user/060-using_triggers.md Outdated Show resolved Hide resolved
@astarte-platform astarte-platform deleted a comment from Pavinati Nov 11, 2022
Annopaolo and others added 6 commits November 11, 2022 18:41
@Annopaolo
Realm Management API: add trigger delivery policy to API
8e9f94b 
Allow to install, delete, list HTTP trigger delivery policies.
Allow to install triggers referencing a custom policy.

Signed-off-by: Arnaldo Cesco <arnaldo.cesco@secomind.com>
@Annopaolo
Trigger Engine: add support for trigger policy
46aba85 
Add support for customizable redelivery policies for HTTP triggers.
TriggerEngine retries (or not) to deliver a payload according to the
related trigger policy. Default to not retrying on any error, as not to break
compatibility.

There is currently only one listener on the exchange where DUP publishes data.
In the future, there will be one listener per policy queue.

Signed-off-by: Arnaldo Cesco <arnaldo.cesco@secomind.com>
@Annopaolo
Data Updater Plant: support trigger policies
8387e86 
Add support for customizable redelivery policies for HTTP triggers.
Route HTTP triggers to TriggerEngine according to the related policy.
If no policy is specified, a default one is used.

Signed-off-by: Arnaldo Cesco <arnaldo.cesco@ispirata.com>
@Annopaolo
TriggerEngine: handle messages on mulitple queues, one for each policy
24a9b79 
Signed-off-by: Arnaldo Cesco <arnaldo.cesco@secomind.com>
@Annopaolo
TriggerEngine: remove policy processes when policies are uninstalled
1167876 
Signed-off-by: Arnaldo Cesco <arnaldo.cesco@secomind.com>
@Annopaolo
TriggerEngine: let policy process crash on unexpected error
8367b41 
Policy process did catch unexpected message delivery errors, but had no info
on whether to redeliver the message or not. Let the policy process crash
instead of handling the unexpected error, and let the related AMQPConsumer
process come down with it. This will put handled but unacked messages (if any)
back in the queue.

Signed-off-by: Arnaldo Cesco <arnaldo.cesco@secomind.com>
@Annopaolo Annopaolo requested a review from Pavinati November 11, 2022 17:45
@Annopaolo
Docs: add Trigger Policy docs
461d4b2 
Signed-off-by: Arnaldo Cesco <arnaldo.cesco@secomind.com>
@bettio bettio merged commit d786d5c into astarte-platform:master Nov 11, 2022
@davidebriani davidebriani mentioned this pull request Jan 16, 2023
Closed
4 tasks
This was referenced Jan 17, 2023
Closed
Closed
@Annopaolo Annopaolo mentioned this pull request Apr 13, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bettio bettio bettio approved these changes

@rbino rbino rbino approved these changes

@Pavinati Pavinati Pavinati approved these changes

@davidebriani davidebriani davidebriani approved these changes

Assignees
No one assigned
Labels
app:trigger_engine This issue or pull request is about astarte_trigger_engine application enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@Annopaolo @bettio @rbino @Pavinati @davidebriani