Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-10487 [FlightRPC][C++] Header-based auth in clients #8724

Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
c2ba2c7
[1] Initial commit to support client header authentication in C++
lyndonbauto Nov 18, 2020
ab7d6a3
[1] Added integration test for client header authentication in C++ an…
lyndonbauto Nov 19, 2020
ecb533f
Merge branch 'lyndon/flight-auth-redesign-cpp' of https://github.com/…
lyndonbauto Nov 19, 2020
73be3e7
[1] Updates for linting
lyndonbauto Nov 19, 2020
74ef8ea
[1] Adding missed file.
lyndonbauto Nov 19, 2020
29a3192
[1] Adding fix for Java lint errors.
lyndonbauto Nov 19, 2020
fac0bd0
[1] Added a couple comments
lyndonbauto Nov 19, 2020
7fd1279
[1] Minor comment fixes
lyndonbauto Nov 19, 2020
0b5a08e
[1] Addressed pull request comments, still need to address comments a…
lyndonbauto Nov 23, 2020
6861cf0
[1] Added unit test.
lyndonbauto Nov 23, 2020
01a134b
[1] Fixed linting issues
lyndonbauto Nov 23, 2020
de78c6b
[1] Fixing linting issues
lyndonbauto Nov 23, 2020
c44698f
[1] Removed some extra spaces at the end of some lines.
lyndonbauto Nov 23, 2020
e975fd8
[1] Correcting linting issues.
lyndonbauto Nov 24, 2020
37889fa
[1] Minor cmake fix
lyndonbauto Nov 24, 2020
e7ac27c
[1] Trying different cmake spacing.
lyndonbauto Nov 24, 2020
ba7cb9f
[1] Trying different cmake
lyndonbauto Nov 24, 2020
1426252
[1] Addressed code review comments.
lyndonbauto Nov 24, 2020
516d993
[1] Removing integration test and reverting some cmake changes.
lyndonbauto Nov 24, 2020
3000ecb
[1] Removed some no longer used functionality.
lyndonbauto Nov 24, 2020
1de10fa
[1] Added improved testing and fixed linting.
lyndonbauto Nov 24, 2020
065af4a
[1] Fixed lint issue
lyndonbauto Nov 24, 2020
d4da03b
Merge branch 'master' of https://github.com/apache/arrow into jduo/ly…
lyndonbauto Nov 25, 2020
911fcc7
[1] Updating submodule
lyndonbauto Nov 25, 2020
f41edce
[1] Minor documentation fixes.
lyndonbauto Nov 25, 2020
6b6fbbe
[1] Fixed casting issue on some builds
lyndonbauto Nov 25, 2020
199b655
[1] Added missing parameter for documentation
lyndonbauto Nov 25, 2020
47aa581
[1] Fixing cast.
lyndonbauto Nov 25, 2020
477d865
[1] Moving std:: from toupper call because it causes break in some bu…
lyndonbauto Nov 25, 2020
1cc3fdb
[1] Adding missed std remove
lyndonbauto Nov 25, 2020
d27465d
[1] Fixed linting issue.
lyndonbauto Nov 25, 2020
d21006f
[1] Updated test return error properly and to check for error
lyndonbauto Nov 25, 2020
6cd8a45
[1] Fixed linting issue.
lyndonbauto Nov 25, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
[1] Updates for linting
  • Loading branch information
lyndonbauto committed Nov 19, 2020
commit 73be3e72b9e1d8d8bae46fcda87a3064994ace1a
7 changes: 4 additions & 3 deletions cpp/src/arrow/flight/client.cc
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ struct ClientRpc {
std::chrono::system_clock::now() + options.timeout);
context.set_deadline(deadline);
}
for (auto metadata: options.metadata) {
for (auto metadata : options.metadata) {
context.AddMetadata(metadata.first, metadata.second);
}
}
Expand Down Expand Up @@ -998,7 +998,8 @@ class FlightClient::FlightClientImpl {
return Status::OK();
}

Status AuthenticateBasicToken(std::string username, std::string password, std::pair<std::string, std::string>* bearer_token) {
Status AuthenticateBasicToken(std::string username, std::string password,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should username/password be passed by const ref?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

std::pair<std::string, std::string>* bearer_token) {
// Add bearer token factory to middleware so it can intercept the bearer token.
middleware.push_back(std::make_shared<ClientBearerTokenFactory>(bearer_token));

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks odd to create the shared pointer after you've passed in the raw pointer....it seems like the method itself should take a shared pointer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The raw pointer is unpopulated, so it's passed to the BearerTokenFactory's constructor, which stores it and populated it when it receives the bearer token. I could make the client pass the whole factory in with the bearer token already inside it, but it's more work and requires they understand what's going on more than they otherwise would need to.

ClientRpc rpc({});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may actually want to allow passing call options so things like timeouts can be set.

Expand Down Expand Up @@ -1227,7 +1228,7 @@ Status FlightClient::Authenticate(const FlightCallOptions& options,
}

Status FlightClient::AuthenticateBasicToken(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we make this arrow::Result<std::pair<>>?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I will make this change.

std::string username, std::string password,
std::string username, std::string password,
std::pair<std::string, std::string>* bearer_token) {
return impl_->AuthenticateBasicToken(username, password, bearer_token);
}
Expand Down
8 changes: 5 additions & 3 deletions cpp/src/arrow/flight/client.h
Original file line number Diff line number Diff line change
Expand Up @@ -194,10 +194,12 @@ class ARROW_FLIGHT_EXPORT FlightClient {
std::unique_ptr<ClientAuthHandler> auth_handler);

/// \brief Authenticate to the server using the given handler.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no handler in play here.

/// \param[in] options Per-RPC options
/// \param[in] auth_handler The authentication mechanism to use
/// \param[in] username Username to use
/// \param[in] password Password to use
/// \param[in] bearer_token Bearer token retreived if applicable
/// \return Status OK if the client authenticated successfully
Status AuthenticateBasicToken(std::string username, std::string password, std::pair<std::string, std::string>* bearer_token);
Status AuthenticateBasicToken(std::string username, std::string password,
std::pair<std::string, std::string>* bearer_token);

/// \brief Perform the indicated action, returning an iterator to the stream
/// of results, if any
Expand Down
38 changes: 21 additions & 17 deletions cpp/src/arrow/flight/client_header_auth_middleware.cc
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,14 @@ namespace flight {

std::string base64_encode(const std::string& input);

ClientBearerTokenMiddleware::ClientBearerTokenMiddleware(std::pair<std::string, std::string>* bearer_token_)
ClientBearerTokenMiddleware::ClientBearerTokenMiddleware(
std::pair<std::string, std::string>* bearer_token_)
: bearer_token(bearer_token_) { }

void ClientBearerTokenMiddleware::SendingHeaders(AddCallHeaders* outgoing_headers) { }

void ClientBearerTokenMiddleware::ReceivedHeaders(const CallHeaders& incoming_headers) {
void ClientBearerTokenMiddleware::ReceivedHeaders(
const CallHeaders& incoming_headers) {
// Grab the auth token if one exists.
auto bearer_iter = incoming_headers.find(AUTH_HEADER);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const?

if (bearer_iter == incoming_headers.end()) {
Expand All @@ -43,9 +45,9 @@ namespace flight {
// Check if the value of the auth token starts with the bearer prefix, latch the token.
std::string bearer_val = bearer_iter->second.to_string();
if (bearer_val.size() > BEARER_PREFIX.size()) {
bool hasPrefix = std::equal(bearer_val.begin(), bearer_val.begin() + BEARER_PREFIX.size(), BEARER_PREFIX.begin(),
[] (const char& char1, const char& char2) {
return (std::toupper(char1) == std::toupper(char2));
bool hasPrefix = std::equal(bearer_val.begin(), bearer_val.begin() + BEARER_PREFIX.size(), BEARER_PREFIX.begin(),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like they use snake_case instead of camelCase.

[] (const char& char1, const char& char2) {
return (std::toupper(char1) == std::toupper(char2));
}
);
if (hasPrefix) {
Expand All @@ -55,7 +57,7 @@ namespace flight {
}

void ClientBearerTokenMiddleware::CallCompleted(const Status& status) { }

void ClientBearerTokenFactory::StartCall(const CallInfo& info, std::unique_ptr<ClientMiddleware>* middleware) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better to pass a reference instead of a pointer?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't this is a method of the base class, I think it's done this way to allow you to assign a new unique pointer to it without exposing the other middlewares they are already holding in their vector.

*middleware = std::unique_ptr<ClientBearerTokenMiddleware>(new ClientBearerTokenMiddleware(bearer_token));

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

std::make_unique?

}
Expand All @@ -68,28 +70,28 @@ namespace flight {
std::string string_format(const std::string& format, const Args... args) {
// Check size requirement for new string and increment by 1 for null terminator.
size_t size = std::snprintf(nullptr, 0, format.c_str(), args ...) + 1;
if(size <= 0){
throw std::runtime_error("Error during string formatting. Format: '" + format + "'.");
if(size <= 0){

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: spacing between if (, ){

throw std::runtime_error("Error during string formatting. Format: '" + format + "'.");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And Arrow disallows exceptions.

}

// Create buffer for new string and write string in.
std::unique_ptr<char[]> buf(new char[size]);
std::unique_ptr<char[]> buf(new char[size]);
std::snprintf(buf.get(), size, format.c_str(), args...);

// Convert to std::string, subtracting size by 1 to trim null terminator.
return std::string(buf.get(), buf.get() + size - 1);
}

void AddBasicAuthHeaders(grpc::ClientContext* context, const std::string& username, const std::string& password) {
const std::string formatted_credentials = string_format("%s:%s", username.c_str(), password.c_str());
context->AddMetadata(AUTH_HEADER, BASIC_PREFIX + base64_encode(formatted_credentials));
}

std::string base64_encode(const std::string& input) {
static const std::string base64_chars =
static const std::string base64_chars =

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't exist in the codebase already?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I found it. Will remove this.

"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
auto get_encoded_length = [] (const std::string& in) {
return 4 * ((in.size() + 2) / 3);
auto get_encoded_length = [] (const std::string& in) {
return 4 * ((in.size() + 2) / 3);
};
auto get_overwrite_count = [] (const std::string& in) {
const std::string::size_type remainder = in.length() % 3;
Expand All @@ -110,10 +112,12 @@ namespace flight {
encoded.push_back(base64_chars[(octriple >> j * 6) & 0x3F]);
}
}

// Round up to nearest multiple of 3 and replace characters at end based on rounding.
int overwrite_count = get_overwrite_count(input);
encoded.replace(encoded.length() - overwrite_count, encoded.length(), overwrite_count, '=');
encoded.replace(encoded.length() - overwrite_count,
encoded.length(),
overwrite_count, '=');
return encoded;
}
} // namespace flight
Expand Down
18 changes: 11 additions & 7 deletions cpp/src/arrow/flight/client_header_auth_middleware.h
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@

#pragma once

#include "client_middleware.h"
#include "client_auth.h"
#include "client.h"
#include "arrow/flight/client_middleware.h"
#include "arrow/flight/client_auth.h"
#include "arrow/flight/client.h"

#ifdef GRPCPP_PP_INCLUDE
#include <grpcpp/grpcpp.h>
Expand All @@ -45,11 +45,14 @@ const std::string BASIC_PREFIX = "Basic ";
namespace arrow {
namespace flight {

void ARROW_FLIGHT_EXPORT AddBasicAuthHeaders(grpc::ClientContext* context, const std::string& username, const std::string& password);
void ARROW_FLIGHT_EXPORT AddBasicAuthHeaders(grpc::ClientContext* context,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is internal - it shouldn't be in a public header. (Ditto for the grpc include.)

const std::string& username,
const std::string& password);

class ARROW_FLIGHT_EXPORT ClientBearerTokenMiddleware : public ClientMiddleware {
public:
explicit ClientBearerTokenMiddleware(std::pair<std::string, std::string>* bearer_token_);
explicit ClientBearerTokenMiddleware(
std::pair<std::string, std::string>* bearer_token_);

void SendingHeaders(AddCallHeaders* outgoing_headers);
void ReceivedHeaders(const CallHeaders& incoming_headers);
Expand All @@ -61,11 +64,12 @@ class ARROW_FLIGHT_EXPORT ClientBearerTokenMiddleware : public ClientMiddleware

class ARROW_FLIGHT_EXPORT ClientBearerTokenFactory : public ClientMiddlewareFactory {
public:
explicit ClientBearerTokenFactory(std::pair<std::string, std::string>* bearer_token_) : bearer_token(bearer_token_) {}
explicit ClientBearerTokenFactory(std::pair<std::string, std::string>* bearer_token_)
: bearer_token(bearer_token_) {}

void StartCall(const CallInfo& info, std::unique_ptr<ClientMiddleware>* middleware);
void Reset();

private:
std::pair<std::string, std::string>* bearer_token;
};
Expand Down