Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A draft of simplifying the service implementation #145

Merged
merged 3 commits into from
Oct 22, 2023
Merged

A draft of simplifying the service implementation #145

merged 3 commits into from
Oct 22, 2023

Conversation

mmmries
Copy link
Collaborator

@mmmries mmmries commented Oct 21, 2023

After reviewing #141 a bit further and reviewing the ADR document, I had a few thoughts about how to simplify and optimize our service implementation:

  • Normalize the user-provided service config into a known structure with guaranteed keys
    • this reduces the need for conditional code later on
  • Allocate :counters during initialization and skip the use of ETS
  • Isolate as much of the knowledge about structure of the service datastructure to one module as possible. Use functions to pull out what is needed for service responder etc.
  • Avoid starting ServiceResponder as an additional process, just add an additional subscription to the ConsumerSupervisor

Note: I wrote this before #144 was opened, but it looks like we were thinking of some of these same things

@mmmries mmmries requested a review from autodidaddict October 21, 2023 19:53
@@ -0,0 +1,56 @@
defmodule EchoService do
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a benchmark to try to measure the impact of these changes, but it turns out this branch only make a ~5% improvement in throughput.

We can get around 13.7k service messages per second on both the main branch and about 14.5k service messages per second on this branch. 🤷


true ->
{:error, "You must provide a module or consuming function for the consumer supervisor"}
with {:ok, state} <- maybe_append_service(state, settings),
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious if you'll like this change. The cond block was a bit tricky to read and I wanted to validate the user-input about the service definition during process startup rather than crashing the process later when the connection became available.

So I used a with block here to show the happy path, and I perform all of the service configuration validation right here (as well as initializing service counters etc). This way our stats will persist across re-connects and if the user has an invalid validation they'll get a helpful error at boot time rather than a process crash later

@@ -189,35 +174,65 @@ defmodule Gnat.ConsumerSupervisor do
end
end

defp initialize_as_manual_consumer(state, connection_pid) do
subscriptions = Enum.map(state.subscription_topics, fn(topic_and_queue_group) ->
defp subscribe_to_topics(%{service: service}, connection_pid) do
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I generally prefer to do pattern matching in my function heads rather than if checks inside the function bodies. I don't feel strongly about this preference, but I like how it looks in this case. What do you think?

{epname, groupname}
def maybe_respond(%{topic: topic} = message, service) do
case String.split(topic, ".") do
["$SRV", @op_ping | rest] -> handle_ping(rest, service, message)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I liked pattern matching to grab the prefix, op and rest or tail this way rather than using List.slice. What do you think?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah this looks much cleaner

Copy link
Collaborator

@autodidaddict autodidaddict left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if there's anything that you want to incorporate from #144 , but this all looks better than what I had. It's been too long since I've written idiomatic Elixir.

@mmmries
Copy link
Collaborator Author

mmmries commented Oct 22, 2023

I pulled in some of the typedoc and function doc improvements from #144 into this PR. Thanks for reviewing and sorry we ended tackling some of the same work. But I was glad to see how our thinking was converging.

I'm on vacation so I suddenly had extra time for this project. I'll try to follow your lead on creating a github release

@mmmries mmmries merged commit 16ab4f5 into main Oct 22, 2023
@mmmries mmmries deleted the simplified branch October 22, 2023 07:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants