Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor proxy #814

Merged
merged 67 commits into from
Oct 7, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
dee7ae3
Add dynamic-proxy
paulgb Sep 23, 2024
ecd45f8
new proxy is integrated
paulgb Sep 23, 2024
bc40f2c
remove extra imports
paulgb Sep 23, 2024
f1a0649
use appropriate imports
paulgb Sep 23, 2024
3aa829a
basic proxy works
paulgb Sep 23, 2024
cb96f31
integrate connection monitor
paulgb Sep 23, 2024
8ccfb1b
redirect test work
paulgb Sep 23, 2024
768880c
https redirect work
paulgb Sep 23, 2024
b1d281a
proxy progress
paulgb Sep 23, 2024
a3129a2
progress
paulgb Sep 23, 2024
11cb6e2
lifecycle test with dummy drone
paulgb Sep 23, 2024
3b40d79
pass test with no connection token
paulgb Sep 24, 2024
779d1ee
get rid of warnings
paulgb Sep 24, 2024
f6b52da
add test for bad connection string
paulgb Sep 24, 2024
77dada1
add bad gateway test
paulgb Sep 24, 2024
b9a7705
add backend timeout test
paulgb Sep 24, 2024
1e12459
add backend accepts test
paulgb Sep 24, 2024
a367217
subdomain verification
paulgb Sep 24, 2024
00a65ff
remove dead code
paulgb Sep 24, 2024
a2f9781
add x-verified- headers
paulgb Sep 24, 2024
dd2dc07
add proxy_headers tests
paulgb Sep 24, 2024
61a770a
add test of CORS headers in error response
paulgb Sep 24, 2024
f7b5062
valid response has CORS headers
paulgb Sep 24, 2024
27d33e5
x-forwarded-* headers
paulgb Sep 24, 2024
ff50340
x-plane-backend-id header
paulgb Sep 24, 2024
6970b6e
error handling in https redirect service
paulgb Sep 24, 2024
0a34df9
nit
paulgb Sep 24, 2024
29b9ff2
proxy comments
paulgb Sep 24, 2024
a9f8b8e
request docs
paulgb Sep 24, 2024
3740756
server
paulgb Sep 24, 2024
b36046a
upgrade
paulgb Sep 24, 2024
aff6fcd
plane-test deps
paulgb Sep 24, 2024
f5e36fb
more cleanup
paulgb Sep 24, 2024
5f5e400
clean up unwraps
paulgb Sep 24, 2024
e580381
format
paulgb Sep 24, 2024
5c445c1
replace unwraps
paulgb Sep 24, 2024
c16a88a
some pr nits
paulgb Sep 24, 2024
b3c7ab3
use consts for headers
paulgb Sep 24, 2024
ef07e70
undo macos bug fix
paulgb Sep 24, 2024
d1f467c
send server header
paulgb Sep 24, 2024
89c0ea0
use GONE header
paulgb Sep 24, 2024
5ddbffb
return MOVED_PERMANENTLY instead of FOUND
paulgb Sep 24, 2024
c11cb6c
proxy ready endpoint
paulgb Sep 24, 2024
ff8b0b9
fix proxy test
paulgb Sep 24, 2024
79508d5
race comment
paulgb Sep 24, 2024
73d4e42
fix root redirect test
paulgb Sep 24, 2024
74d1661
add static token test
paulgb Sep 25, 2024
d17025a
connection monitor touch test
paulgb Sep 25, 2024
404c031
add test for websocket connection monitor
paulgb Sep 25, 2024
8929e2d
remove access-control-allow-credentials header
paulgb Sep 29, 2024
8331c0b
cleaner assert
paulgb Sep 29, 2024
dac2349
remove access-control-allow-credentials check
paulgb Sep 29, 2024
d558c9a
clippy dynamic-proxy
paulgb Sep 30, 2024
b2efbeb
plane-tests clippy
paulgb Oct 1, 2024
f4c8199
add tests for static token and other PR fixes
paulgb Oct 1, 2024
870236c
mark parts added in plane
paulgb Oct 1, 2024
cb9d5b6
pr
paulgb Oct 1, 2024
8a586fa
more nits from pr
paulgb Oct 1, 2024
8933a3d
complete thought
paulgb Oct 1, 2024
b7449be
use expect
paulgb Oct 7, 2024
0844875
use panic in websocket echo server
paulgb Oct 7, 2024
0e0ec60
nits from pr
paulgb Oct 7, 2024
97b0078
nit
paulgb Oct 7, 2024
dc9d644
unify logging
paulgb Oct 7, 2024
ffa530c
Merge branch 'main' into paul/dis-2702-plane-proxy-refactor-to-latest…
paulgb Oct 7, 2024
11c3d5e
add dynamic-proxy to workspace
paulgb Oct 7, 2024
bf17872
add dynamic-proxy to default-members
paulgb Oct 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,27 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

---

Contains code from hyperium/hyper-util, licensed under the MIT license:

Copyright (c) 2023 Sean McArthur

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
3 changes: 2 additions & 1 deletion dynamic-proxy/src/graceful_shutdown.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ use std::{
};
use tokio::sync::watch;

#[derive(Clone)]
#[derive(Clone)] // Added in Plane
pub struct GracefulShutdown {
tx: watch::Sender<()>,
}
Expand All @@ -34,6 +34,7 @@ impl GracefulShutdown {
})
}

// Added in Plane
pub fn subscribe(&self) -> watch::Receiver<()> {
self.tx.subscribe()
}
Expand Down
15 changes: 9 additions & 6 deletions dynamic-proxy/tests/common/websocket_echo_server.rs
Original file line number Diff line number Diff line change
Expand Up @@ -48,14 +48,17 @@ async fn ws_handler(ws: WebSocketUpgrade) -> impl IntoResponse {

async fn handle_socket(mut socket: WebSocket) {
while let Some(msg) = socket.recv().await {
if let Ok(msg) = msg {
if let Message::Text(text) = msg {
if socket.send(Message::Text(text)).await.is_err() {
break;
match msg {
Ok(msg) => {
if let Message::Text(text) = msg {
if socket.send(Message::Text(text)).await.is_err() {
break;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we panic here as well?

}
}
}
} else {
break;
Err(e) => {
panic!("Error receiving message: {}", e);
}
}
}
}
8 changes: 5 additions & 3 deletions dynamic-proxy/tests/graceful.rs
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,11 @@ async fn test_graceful_shutdown() {
// Create a client and start a POST request without finishing the body
let client = reqwest::Client::new();

let _client = client.clone();
let _url = url.clone();
let response_handle = tokio::spawn(async move { _client.get(&_url).send().await.unwrap() });
let response_handle = {
let client = client.clone();
let url = url.clone();
tokio::spawn(async move { client.get(&url).send().await.unwrap() })
};

tokio::time::sleep(Duration::from_millis(100)).await;

Expand Down
8 changes: 5 additions & 3 deletions dynamic-proxy/tests/graceful_https.rs
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,11 @@ async fn test_graceful_shutdown_https() {
.build()
.unwrap();

let _client = client.clone();
let _url = url.clone();
let response_handle = tokio::spawn(async move { _client.get(&_url).send().await.unwrap() });
let response_handle = {
let client = client.clone();
let url = url.clone();
tokio::spawn(async move { client.get(&url).send().await.unwrap() })
};

tokio::time::sleep(Duration::from_millis(100)).await;

Expand Down
3 changes: 1 addition & 2 deletions dynamic-proxy/tests/test_http_redirect.rs
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,8 @@ async fn do_request(url: &str) -> Response {
url.set_port(Some(port)).unwrap();

// Request to http://foo.bar.baz should redirect to https://foo.bar.baz
let response = get_client().get(url).send().await.unwrap();

response
get_client().get(url).send().await.unwrap()
}

#[tokio::test]
Expand Down
4 changes: 2 additions & 2 deletions plane/plane-tests/tests/common/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@ pub mod docker;
pub mod localhost_resolver;
pub mod proxy_mock;
pub mod resources;
pub mod simple_axum_server; // NOTE: copied from dynamic-proxy (until we merge them back)
pub mod simple_axum_server; // TODO: copied from dynamic-proxy (until we merge them back)
pub mod test_env;
pub mod timeout;
pub mod websocket_echo_server; // NOTE: copied from dynamic-proxy (until we merge them back)
pub mod websocket_echo_server; // TODO: copied from dynamic-proxy (until we merge them back)

pub fn run_test<F, Fut>(name: &str, time_limit: Duration, test_function: F)
where
Expand Down
15 changes: 9 additions & 6 deletions plane/plane-tests/tests/common/websocket_echo_server.rs
Original file line number Diff line number Diff line change
Expand Up @@ -48,14 +48,17 @@ async fn ws_handler(ws: WebSocketUpgrade) -> impl IntoResponse {

async fn handle_socket(mut socket: WebSocket) {
while let Some(msg) = socket.recv().await {
if let Ok(msg) = msg {
if let Message::Text(text) = msg {
if socket.send(Message::Text(text)).await.is_err() {
break;
match msg {
Ok(msg) => {
if let Message::Text(text) = msg {
if socket.send(Message::Text(text)).await.is_err() {
break;
paulgb marked this conversation as resolved.
Show resolved Hide resolved
}
}
}
} else {
break;
Err(e) => {
panic!("Error receiving message: {}", e);
}
}
}
}
2 changes: 1 addition & 1 deletion plane/plane-tests/tests/proxy_connection_monitor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ async fn proxy_marks_backend_as_recently_active(env: TestEnvironment) {
panic!("Backend entry not found");
};
assert_eq!(backend_entry.active_connections, 0);
assert_eq!(backend_entry.had_recent_connection, true);
assert!(backend_entry.had_recent_connection);
}

#[plane_test]
Expand Down
2 changes: 1 addition & 1 deletion plane/src/proxy/proxy_server.rs
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ impl Service<Request<Incoming>> for ProxyState {
let (mut res, upgrade_handler) = match result {
Ok((res, upgrade_handler)) => (res, upgrade_handler),
Err(e) => {
tracing::error!("Error proxying request: {}", e);
tracing::error!(?e, "Error proxying request");
return status_code_to_response(StatusCode::INTERNAL_SERVER_ERROR);
}
};
Expand Down
71 changes: 70 additions & 1 deletion plane/src/proxy/request.rs
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ pub fn get_and_maybe_remove_bearer_token(parts: &mut uri::Parts) -> Option<Beare
// the full incoming path, and the path to proxy to is just `/`.
let (token, path) = match full_path.split_once('/') {
Some((token, path)) => (token, path),
None => (full_path, "/"),
None => (full_path, ""),
};

if token.is_empty() {
Expand Down Expand Up @@ -76,6 +76,8 @@ pub fn get_and_maybe_remove_bearer_token(parts: &mut uri::Parts) -> Option<Beare

#[cfg(test)]
mod tests {
use uri::Uri;

use super::*;
use std::str::FromStr;

Expand Down Expand Up @@ -134,4 +136,71 @@ mod tests {
let cluster = ClusterName::from_str("myhost").unwrap();
assert_eq!(subdomain_from_host(host, &cluster), Ok(Some("foobar")));
}

#[test]
fn test_get_and_maybe_remove_bearer_token() {
let url = Uri::from_str("https://example.com/foo/bar").unwrap();
let mut parts = url.into_parts();
assert_eq!(
get_and_maybe_remove_bearer_token(&mut parts),
Some(BearerToken::from("foo".to_string()))
);
assert_eq!(
parts.path_and_query,
Some(PathAndQuery::from_str("/bar").unwrap())
);
}

#[test]
fn test_get_and_maybe_remove_bearer_token_ends_no_slash() {
let url = Uri::from_str("https://example.com/foo").unwrap();
let mut parts = url.into_parts();
assert_eq!(
get_and_maybe_remove_bearer_token(&mut parts),
Some(BearerToken::from("foo".to_string()))
);
assert_eq!(
parts.path_and_query,
Some(PathAndQuery::from_str("/").unwrap())
);
}

#[test]
fn test_get_and_maybe_remove_bearer_token_ends_in_slash() {
let url = Uri::from_str("https://example.com/foo/").unwrap();
let mut parts = url.into_parts();
assert_eq!(
get_and_maybe_remove_bearer_token(&mut parts),
Some(BearerToken::from("foo".to_string()))
);
assert_eq!(
parts.path_and_query,
Some(PathAndQuery::from_str("/").unwrap())
);
}

#[test]
fn test_get_and_maybe_remove_bearer_token_no_token() {
let url = Uri::from_str("https://example.com/").unwrap();
let mut parts = url.into_parts();
assert_eq!(get_and_maybe_remove_bearer_token(&mut parts), None);
assert_eq!(
parts.path_and_query,
Some(PathAndQuery::from_str("/").unwrap())
);
}

#[test]
fn test_get_and_maybe_remove_bearer_token_static_token() {
let url = Uri::from_str("https://example.com/s.foo/bar").unwrap();
let mut parts = url.into_parts();
assert_eq!(
get_and_maybe_remove_bearer_token(&mut parts),
Some(BearerToken::from("s.foo".to_string()))
);
assert_eq!(
parts.path_and_query,
Some(PathAndQuery::from_str("/s.foo/bar").unwrap())
);
}
}
Loading