Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(api_service): exclude health checks from throttling with ConcurrencyLimitLayer #2320

Merged
merged 7 commits into from
Oct 10, 2024
Merged

fix(api_service): exclude health checks from throttling with ConcurrencyLimitLayer #2320

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
5404185
fix(api_service): exclude health checks from throttling with Concurre…
rymnc Oct 8, 2024
131167b
fix: changelog
rymnc Oct 8, 2024
aa651b8
fix: remove redundant comments
rymnc Oct 8, 2024
aadac76
fix: Cargo fmt
netrome Oct 8, 2024
beb03d9
chore: fix changelog
rymnc Oct 8, 2024
1a66422
fix: throttle only /v1/graphql
rymnc Oct 9, 2024
e3a0a7b
Merge branch 'master' into fix/health-concurrency
rymnc Oct 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ and this project adheres to [Semantic Versioning](http://semver.org/).

## [Unreleased]

## Fixed
- [2320](https://github.com/FuelLabs/fuel-core/issues/2320): Prevent `/health` and `/v1/health` from being throttled by the concurrency limiter.

## [Version 0.38.0]

### Added
Expand Down
14 changes: 10 additions & 4 deletions crates/fuel-core/src/graphql_api/api_service.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -221,17 +221,23 @@ where
.extension(ViewExtension::new())
.finish();

let router = Router::new()
let unthrottled_routes = Router::new()
.route("/v1/health", get(health))
.route("/health", get(health));

let throttled_routes = Router::new()
.route("/v1/playground", get(graphql_playground))
.route("/v1/graphql", post(graphql_handler).options(ok))
.route(
"/v1/graphql-sub",
post(graphql_subscription_handler).options(ok),
)
.route("/v1/metrics", get(metrics))
.route("/v1/health", get(health))
.route("/health", get(health))
.layer(ConcurrencyLimitLayer::new(concurrency_limit))
.layer(ConcurrencyLimitLayer::new(concurrency_limit));
Copy link
Collaborator

@xgreenx xgreenx Oct 9, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we want something like. Because graph-sub also is limited by number of subscriptions, and playground is one time query that returns pre rendered html. Throteling metrics doesn't allow request data=)

image

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Throteling metrics doesn't allow request data=)
are you saying that because metrics are throttled, it affects how graphql and graphql-sub are throttled as well?

why shouldn't we throttle the metrics?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because if we have a lot of queries, it becomes impossible for us to track the state of the node from the Grafana=)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, making the change

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in 1a66422


let router = Router::new()
.merge(unthrottled_routes)
.merge(throttled_routes)
.layer(Extension(schema))
.layer(TraceLayer::new_for_http())
.layer(TimeoutLayer::new(request_timeout))
Expand Down
24 changes: 20 additions & 4 deletions tests/tests/dos.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -385,15 +385,12 @@ async fn complex_queries__2_state_transition_bytecode__query_to_complex() {
}

#[tokio::test]
async fn concurrency_limit_0_prevents_any_queries() {
async fn concurrency_limit_0_prevents_any_throttled_queries() {
// Given
let mut config = Config::local_node();
config.graphql_config.max_concurrent_queries = 0;
config.graphql_config.api_request_timeout = std::time::Duration::from_millis(100);

let query = FULL_BLOCK_QUERY.to_string();
let query = query.replace("$NUMBER_OF_BLOCKS", "40");

let node = FuelService::new_node(config).await.unwrap();
let url = format!("http://{}/v1/graphql", node.bound_address);
let client = reqwest::Client::new();
Expand All @@ -412,6 +409,25 @@ async fn concurrency_limit_0_prevents_any_queries() {
assert_eq!(result.status(), 408);
}

#[tokio::test]
async fn concurrency_limit_0_allows_unthrottled_queries() {
// Given
let mut config = Config::local_node();
// perhaps this shouldn't be called `graphql_config` but `api_config` or something
config.graphql_config.max_concurrent_queries = 0;

let node = FuelService::new_node(config).await.unwrap();
let url = format!("http://{}/v1/health", node.bound_address);
let client = reqwest::Client::new();

// When
let response = client.get(url).send().await;

// Then
let result = response.unwrap();
assert_eq!(result.status(), 200);
}

#[tokio::test(flavor = "multi_thread", worker_threads = 8)]
async fn concurrency_limit_1_prevents_concurrent_queries() {
// Given
Expand Down
Loading