diff --git a/.envrc.example b/.envrc.example new file mode 100644 index 000000000..a646ef5e5 --- /dev/null +++ b/.envrc.example @@ -0,0 +1,16 @@ +export GOOGLE_CLOUD_PROJECT="project-name" + +export MYSQL_CONNECTION_NAME="project:region:instance" +export MYSQL_USER="mysql-user" +export MYSQL_PASS="mysql-password" +export MYSQL_DB="mysql-db-name" + +export POSTGRES_CONNECTION_NAME="project:region:instance" +export POSTGRES_USER="postgres-user" +export POSTGRES_PASS="postgres-password" +export POSTGRES_DB="postgres-db-name" + +export SQLSERVER_CONNECTION_NAME="project:region:instance" +export SQLSERVER_USER="sqlserver-user" +export SQLSERVER_PASS="sqlserver-password" +export SQLSERVER_DB="sqlserver-db-name" diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e7877b508..0f410b41d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -7,6 +7,7 @@ ## Table of contents * [Opening an issue](#opening-an-issue) +* [How to run tests](#how-to-run-tests) * [Contributor License Agreements](#contributor-license-agreements) * [Contributor Code of Conduct](#contributor-code-of-conduct) @@ -72,6 +73,50 @@ expect to be impacted, as well as its severity. As a rule of thumb: +## How to run tests + +The test suite includes both unit and integration tests. For macOS and Linux, +there is a depenency on [FUSE][] that must be present on the system. + +### Test Dependencies + +When running tests on macOS and Linux, users will need to first install +[FUSE][]. Windows users may skip this step. + +On Debian, use: + +``` +sudo apt-get install fuse +``` + +On macOS, use: + +``` +brew install --cask osxfuse +``` + +### How to run just unit tests + +``` +go test -tags=skip_sqlserver,skip_mysql,skip_postgres ./... +``` + +### How to run all tests + +To run all integration tests, users will need a Google Cloud project with a +MySQL, PostgreSQL, and SQL Server database. Note: Pull Requests will run these +tests and as a result may be skipped locally if necessary. + +A sample `.envrc.example` file is included in the root directory which documents +which environment variables must be set to run the integration tests. Copy this +example file to `.envrc` at the root of the project, supply all the correct +values for each variable, source the file (`source .envrc`, or consider using +[direnv][]), and then run: + +``` +go test ./... +``` + ## Contributor License Agreements Open-source software licensing is a wonderful arrangement that benefits @@ -142,3 +187,6 @@ maintainers. This Code of Conduct is adapted from the [Contributor Covenant](http://contributor-covenant.org), version 1.2.0, available at [http://contributor-covenant.org/version/1/2/0/](http://contributor-covenant.org/version/1/2/0/) + +[FUSE]: https://www.kernel.org/doc/html/latest/filesystems/fuse.html +[direnv]: https://direnv.net/ diff --git a/README.md b/README.md index b87fd7ca4..d5ee8a64a 100644 --- a/README.md +++ b/README.md @@ -1,145 +1,333 @@ +# Cloud SQL Proxy -## Cloud SQL Proxy -The Cloud SQL Proxy allows a user with the appropriate permissions to connect -to a Second Generation Cloud SQL database without having to deal with IP whitelisting or SSL -certificates manually. It works by opening unix/tcp sockets on the local machine -and proxying connections to the associated Cloud SQL instances when the sockets -are used. - -To build from source, ensure you have [go installed](https://golang.org/doc/install) -and have set [GOPATH](https://github.com/golang/go/wiki/GOPATH). Then, simply do a go get: - - GO111MODULE=on go get github.com/GoogleCloudPlatform/cloudsql-proxy/cmd/cloud_sql_proxy - -The cloud_sql_proxy will be placed in `$GOPATH/bin` after `go get` completes. - -cloud_sql_proxy takes a few arguments to configure what instances to connect to and connection behavior: - -* `-fuse`: requires access to `/dev/fuse` as well as the `fusermount` binary. An - optional `-fuse_tmp` flag can specify where to place temporary files. The - directory indicated by `-dir` is mounted. -* `-instances="project1:region:instance1,project3:region:instance1"`: A comma-separated list - of instances to open inside `-dir`. Also supports exposing a tcp port and renaming the default Unix Domain Sockets; see examples below. - Same list can be provided via INSTANCES environment variable, in case when both are provided - proxy will use command line flag. -* `-instances_metadata=metadata_key`: Usable on [GCE](https://cloud.google.com/compute/docs/quickstart) only. The given [GCE metadata](https://cloud.google.com/compute/docs/metadata) key will be - polled for a list of instances to open in `-dir`. The metadata key is relative from `computeMetadata/v1/`. The format for the value is the same as the 'instances' flag. A hanging-poll strategy is used, meaning that changes to - the metadata value will be reflected in the `-dir` even while the proxy is - running. When an instance is removed from the list the corresponding socket - will be removed from `-dir` as well (unless it was also specified in - `-instances`), but any existing connections to this instance will NOT be - terminated. -* `-ip_address_types=PUBLIC,PRIVATE`: A comma-delimited list of preferred IP - types for connecting to an instance. For example, setting this to PRIVATE will - force the proxy to connect to instances using an instance's associated private - IP. Defaults to `PUBLIC,PRIVATE` -* `-term_timeout=30s`: How long to wait for connections to close before shutting - down the proxy. Defaults to 0. -* `-skip_failed_instance_config`: Setting this flag will allow you to prevent the proxy from terminating when - some instance configurations could not be parsed and/or are unavailable. -* `-log_debug_stdout=true`: This is to log non-error output to stdOut instead of stdErr. For example, if you don't want connection related messages to log as errors, set this flag to true. Defaults to false. -* `-enable_iam_login`: This enables the proxy to use Cloud SQL IAM database authentication. This will cause the proxy to use IAM account credentials for database user authentication. +![CI][ci-badge] +[![Go Reference][pkg-badge]][pkg-docs] + +The [Cloud SQL Proxy][proxy-page] is a binary that provides IAM-based +authorization and encryption when connecting to a Cloud SQL instance. + +See the [Connecting Overview][connection-overview] page for more information on +connecting to a Cloud SQL instance, or the [About the proxy][about-proxy] page +for details on how the Cloud SQL proxy works. + +Note: The Proxy *cannot* provide a network path to a Cloud SQL instance if one +is not already present (e.g., the proxy cannot access a VPC if it does not +already have access to it). + +## Installation + +For 64-bit Linux, run: + +``` +wget https://storage.googleapis.com/cloudsql-proxy/v1.19.2/cloud_sql_proxy.linux.amd64 -O cloud_sql_proxy +chmod +x cloud_sql_proxy +``` + +Releases for additional OS's and architectures and be found on the [releases +page][releases]. + +For alternative distributions, see below under [third party](#third-party). + +### Container Images + +There are containerized versions of the proxy available from the following +Google Cloud Container Registry repositories: + +* `gcr.io/cloudsql-docker/gce-proxy` +* `us.gcr.io/cloudsql-docker/gce-proxy` +* `eu.gcr.io/cloudsql-docker/gce-proxy` +* `asia.gcr.io/cloudsql-docker/gce-proxy` + +Each image is tagged with the associated proxy version. The following tags are +currently supported: + +* `$VERSION` - default image (recommended) +* `$VERSION-alpine` - uses [`alpine:3`](https://hub.docker.com/_/alpine) + as a base image (only supported from v1.17 up) +* `$VERSION-buster` - uses [`debian:buster`](https://hub.docker.com/_/debian) + as a base image (only supported from v1.17 up) + +We recommend using the latest version of the proxy and updating the version +regularly. However, we also recommend pinning to a specific tag and avoid the +latest tag. Note: the tagged version is only that of the proxy. Changes in base +images may break specific setups, even on non-major version increments. As such, +it's a best practice to test changes before deployment, and use automated +rollbacks to revert potential failures. + +### Install from Source + +To install from source, ensure you have the latest version of [Go +installed](https://golang.org/doc/install). + +Then, simply run: + +``` +go get github.com/GoogleCloudPlatform/cloudsql-proxy/cmd/cloud_sql_proxy +``` + +The `cloud_sql_proxy` will be placed in `$GOPATH/bin` after `go get` completes. + +## Usage + +All the following invocations assume valid credentials are present in the +environment. The following examples all reference an `INSTANCE_CONNECTION_NAME`, +which takes the form: `myproject:myregion:myinstance`. To find the +`INSTANCE_CONNECTION_NAME`, run `gcloud sql instances describe ` +where `INSTANCE_NAME` is the name of the database instance. + +### TCP socket example + +``` bash +# Starts the proxy listening on 127.0.0.1:5432 +cloud_sql_proxy -instances==tcp:5432 +``` + +``` bash +# Starts the proxy listening on on port 5432 on *all* interfaces +cloud_sql_proxy -instances==tcp:0.0.0.0:5432 +``` + +### Unix socket example + +``` bash +# The proxy will mount a Unix domain socket at /cloudsql/ +# Note: The directory specified by `-dir` must exist. +cloud_sql_proxy -dir=/cloudsql -instances= +``` + +### Private IP example + +``` +cloud_sql_proxy -instances==tcp:5432 -ip_address_types=PRIVATE +``` + +For more details, see [Private IP Requirements][private-ip]. + +## Credentials + +The Cloud SQL proxy uses a Cloud IAM account to authorize connections against a +Cloud SQL instance. The proxy sources the credentials for these accounts in the +following order: + +1. The `-credential_file` flag +2. The `-token` flag +3. The service account key at the path stored in the + `GOOGLE_APPLICATION_CREDENTIALS` environment variable. +4. The gcloud user credentials (set from `gcloud auth login`) +5. The [Application Default Credentials](https://cloud.google.com/docs/authentication/production) + +Note: Any account connecting to a Cloud SQL database will need one of the +following IAM roles: + +- Cloud SQL Client (preferred) +- Cloud SQL Editor +- Cloud SQL Admin + +Or one may manually assign the following IAM permissions: + +- `cloudsql.instances.connect` +- `cloudsql.instances.get` + +See [Roles and Permissions in Cloud SQL][roles-and-permissions] for details. + +When the proxy authenticates under the Compute Engine VM's default service +account, the VM must have at least the `sqlservice.admin` API scope (i.e., +"https://www.googleapis.com/auth/sqlservice.admin") and the associated project +must have the SQL Admin API enabled. The default service account must also have +at least writer or editor privileges to any projects of target SQL instances. + + +## CLI Flags + +The Cloud SQL Proxy takes a few arguments to configure what instances to connect +to and connection behavior. For a full list of flags supported by the proxy, +use `cloud_sql_proxy -help`. + +### Authentication Flags + +#### `-credential_file` + +Specifies the path to a JSON [service account][service-account] key the proxy +uses to authorize or authenticate connections. + +#### `-token` + +When set, the proxy uses this Bearer token for authorization. + +#### `-enable_iam_login` + +Enables the proxy to use Cloud SQL IAM database authentication. This will cause +the proxy to use IAM account credentials for database user authentication. For +details, see [Overview of Cloud SQL IAM database authentication][iam-auth] + +### Connection Flags + +#### `-instances="project1:region:instance1,project3:region:instance1"` + +A comma-separated list of instances to open inside `-dir`. Also supports +exposing a TCP port and renaming the default Unix Domain Sockets; see examples +below. Same list can be provided via INSTANCES environment variable, in case +when both are provided - proxy will use command line flag. + +**Example** + +Using TCP sockets: + +``` +./cloud_sql_proxy -instances=my-project:us-central1:sql-inst=tcp:3306 & +mysql -u root -h 127.0.0.1 +``` + +Using Unix sockets: + +``` +./cloud_sql_proxy -dir=/cloudsql -instances=my-project:us-central1:sql-inst & +mysql -u root -S /cloudsql/my-project:us-central1:sql-inst +``` + +To specify a custom Unix socket name: + +``` +./cloud_sql_proxy -dir=/cloudsql \ + -instances=my-project:us-central1:sql-inst=unix:custom_socket_name & +mysql -u root -S /cloudsql/custom_socket_name +``` + +To specify a custom location for a Unix socket (overrides `-dir`): + +``` +./cloud_sql_proxy -dir=/cloudsql \ + -instances=my-project:us-central1:sql-inst=unix:/my/custom/sql-socket & +mysql -u root -S /my/custom/sql-socket +``` + +#### `-fuse` + +Requires access to `/dev/fuse` as well as the `fusermount` binary. An optional +`-fuse_tmp` flag can specify where to place temporary files. The directory +indicated by `-dir` is mounted. + +**Example** + +Using `-fuse`, you do not need to specify instance names ahead of time: + +``` +./cloud_sql_proxy -dir=/cloudsql -fuse & +mysql -u root -S /cloudsql/my-project:us-central1:sql-inst +``` + +#### `-instances_metadata=metadata_key` + +Usable on [GCE](https://cloud.google.com/compute/docs/quickstart) only. The +given [GCE metadata](https://cloud.google.com/compute/docs/metadata) key will be +polled for a list of instances to open in `-dir`. The metadata key is relative +from `computeMetadata/v1/`. The format for the value is the same as the +'instances' flag. A hanging-poll strategy is used, meaning that changes to the +metadata value will be reflected in the `-dir` even while the proxy is running. +When an instance is removed from the list the corresponding socket will be +removed from `-dir` as well (unless it was also specified in `-instances`), but +any existing connections to this instance will NOT be terminated. + +**Example** + +``` +./cloud_sql_proxy -dir=/cloudsql \ + -instances_metadata instance/attributes/ & +mysql -u root -S /cloudsql/my-project:us-central1:sql-inst +``` Note: `-instances` and `-instances_metadata` may be used at the same time but are not compatible with the `-fuse` flag. -cloud_sql_proxy authentication can be configured in a few different ways. Those listed higher on the list will override options lower on the list: - -1. `credential_file` flag -2. `token` flag -3. Service account key at path stored in `GOOGLE_APPLICATION_CREDENTIALS` -4. gcloud _user_ credentials (set from `gcloud auth login`) -5. Default Application Credentials via goauth: - - 1. `GOOGLE_APPLICATION_CREDENTIALS` (again) - 2. gcloud _application default_ credentials (set from ` gcloud auth application-default login`) - 3. appengine.AccessToken (for App Engine Go < =1.9) - 4. GCE/GAE metadata credentials - -When the proxy authenticates under the default service account of the -Compute Engine VM it is running on the VM must have at least the -sqlservice.admin API scope ("https://www.googleapis.com/auth/sqlservice.admin") -and the associated project must have the SQL Admin API -enabled. The default service account must also have at least WRITER/EDITOR -priviledges to any projects of target SQL instances. - -Specifying the `-credential_file` flag allows use of the proxy outside of -Google's cloud. Simply [create a new service -account](https://cloud.google.com/sql/docs/mysql/sql-proxy#create-service-account), -download the associated JSON file, and set `-credential_file` to the path of the -JSON file. You can also set the GOOGLE_APPLICATION_CREDENTIALS environment variable -instead of passing this flag. - -## Example invocations: - - ./cloud_sql_proxy -dir=/cloudsql -instances=my-project:us-central1:sql-inst & - mysql -u root -S /cloudsql/my-project:us-central1:sql-inst - - # To retrieve instances from a custom metadata value (only when running on GCE) - ./cloud_sql_proxy -dir=/cloudsql -instances_metadata instance/attributes/ & - mysql -u root -S /cloudsql/my-project:us-central1:sql-inst - - # For -fuse you do not need to specify instance names ahead of time: - ./cloud_sql_proxy -dir=/cloudsql -fuse & - mysql -u root -S /cloudsql/my-project:us-central1:sql-inst - - # For programs which do not support using Unix Domain Sockets, specify tcp: - ./cloud_sql_proxy -dir=/cloudsql -instances=my-project:us-central1:sql-inst=tcp:3306 & - mysql -u root -h 127.0.0.1 - - # For programs which require a certain Unix Domain Socket name: - ./cloud_sql_proxy -dir=/cloudsql -instances=my-project:us-central1:sql-inst=unix:custom_socket_name & - mysql -u root -S /cloudsql/custom_socket_name - - # For programs which require a the Unix Domain Socket at a specific location, set an absolute path (overrides -dir): - ./cloud_sql_proxy -dir=/cloudsql -instances=my-project:us-central1:sql-inst=unix:/my/custom/sql-socket & - mysql -u root -S /my/custom/sql-socket - -## Container Images - -For convenience, we maintain several containerized versions. These images are -currently hosted in the following GCR repositories: - * `gcr.io/cloudsql-docker/gce-proxy` - * `us.gcr.io/cloudsql-docker/gce-proxy` - * `eu.gcr.io/cloudsql-docker/gce-proxy` - * `asia.gcr.io/cloudsql-docker/gce-proxy` - -__Note:__ - -Each image is tagged with the version of the proxy it was released with. The -following tags are currently supported: - * `$VERSION` - default image (recommended) - * `$VERSION-alpine` - uses [`alpine:3`](https://hub.docker.com/_/alpine) as a base image (only supported from v1.17 up) - * `$VERSION-buster` - uses [`debian:buster`](https://hub.docker.com/_/debian) as a base image (only supported from v1.17 up) - -__Note:__ We strongly recommend to always use the latest version of the proxy, -and to update the version regularly. However, we recommend pinning to a -specific tag and avoid the `latest` tag. Additionally, please note that -the tagged version is _only_ that of the proxy - changes in base images may -break specific setups, even on non-major version increments. As such, -it's a best practice to test changes before deployment, and use automated -rollbacks to revert potential failures. +#### `-max_connections` + +If provided, the maximum number of connections to establish before refusing new +connections. Defaults to 0 (no limit). + +### Additional Flags + +#### `-ip_address_types=PUBLIC,PRIVATE` + +A comma-delimited list of preferred IP types for connecting to an instance. For +example, setting this to PRIVATE will force the proxy to connect to instances +using an instance's associated private IP. Defaults to `PUBLIC,PRIVATE` + +#### `-term_timeout=30s` -## To use from Kubernetes: +How long to wait for connections to close before shutting down the proxy. +Defaults to 0. -### Deploying Cloud SQL Proxy as a sidecar container -Follow this [page](https://github.com/GoogleCloudPlatform/cloudsql-proxy/tree/main/examples/kubernetes#run-the-cloud-sql-proxy-as-a-sidecar). See also -[Connecting from Google Kubernetes Engine](https://cloud.google.com/sql/docs/mysql/connect-kubernetes-engine). +#### `-skip_failed_instance_config` +Setting this flag will prevent the proxy from terminating if any errors occur +during instance configuration. Please note that this means some instances may +fail to be set up correctly while others may work if the proxy restarts. + +#### `-log_debug_stdout=true` + +This is to log non-error output to standard out instead of standard error. For +example, if you don't want connection related messages to log as errors, set +this flag to true. Defaults to false. + +## Running as a Kubernetes Sidecar + +See the [example here][sidecar-example] as well as [Connecting from Google +Kubernetes Engine][connect-to-k8s]. + +## Reference Documentation + +- [Cloud SQL][cloud-sql] +- [Cloud SQL Proxy Documentation][proxy-page] +- [Cloud SQL Proxy Quickstarts][quickstarts] +- [Cloud SQL Code Samples][code-samples] +- [Cloud SQL Proxy Package Documentation][pkg-docs] + +## Contributing + +Contributions are welcome. Please, see the [CONTRIBUTING][contributing] document +for details. + +Please note that this project is released with a Contributor Code of Conduct. +By participating in this project you agree to abide by its terms. See +[Contributor Code of Conduct][code-of-conduct] for more information. ## Third Party __WARNING__: _These distributions are not officially supported by Google._ -### Installing via Homebrew +### Homebrew - You can find a formula for with Homebrew [here](https://github.com/tclass/homebrew-cloud_sql_proxy). +There is Homebrew formula for Cloud SQL Proxy [here](https://github.com/tclass/homebrew-cloud_sql_proxy). +### Kubernetes Cluster Service using Helm -### K8s Cluster Service using Helm +Follow these [instructions](https://github.com/rimusz/charts/tree/master/stable/gcloud-sqlproxy). - Follow these [instructions](https://github.com/rimusz/charts/tree/master/stable/gcloud-sqlproxy). - This chart creates a Deployment and a Service, but we recommend deploying the proxy as a sidecar container in your pods. +This chart creates a Deployment and a Service, but we recommend deploying the +proxy as a sidecar container in your pods. ### .Net Proxy Wrapper (Nuget Package) - - Install via Nuget, follow these [instructions](https://github.com/expert1-pty-ltd/cloudsql-proxy#install-via-nuget). + +Install via Nuget, follow these +[instructions](https://github.com/expert1-pty-ltd/cloudsql-proxy#install-via-nuget). + + +[about-proxy]: https://cloud.google.com/sql/docs/mysql/sql-proxy +[ci-badge]: https://storage.googleapis.com/cloud-devrel-public/cloud-sql-connectors/proxy/go1.15_linux.svg +[cloud-sql]: https://cloud.google.com/sql +[code-samples]: https://cloud.google.com/sql/docs/mysql/samples +[code-of-conduct]: CONTRIBUTING.md#contributor-code-of-conduct +[connect-to-k8s]: https://cloud.google.com/sql/docs/mysql/connect-kubernetes-engine +[connection-overview]: https://cloud.google.com/sql/docs/mysql/connect-overview +[contributing]: CONTRIBUTING.md +[iam-auth]: https://cloud.google.com/sql/docs/postgres/authentication +[pkg-badge]: https://pkg.go.dev/badge/github.com/GoogleCloudPlatform/cloudsql-proxy.svg +[pkg-docs]: https://pkg.go.dev/github.com/GoogleCloudPlatform/cloudsql-proxy +[private-ip]: https://cloud.google.com/sql/docs/mysql/private-ip#requirements_for_private_ip +[proxy-page]: https://cloud.google.com/sql/docs/mysql/sql-proxy +[quickstarts]: https://cloud.google.com/sql/docs/mysql/quickstarts +[releases]: https://github.com/GoogleCloudPlatform/cloudsql-proxy/releases +[roles-and-permissions]: https://cloud.google.com/sql/docs/mysql/roles-and-permissions +[service-account]: https://cloud.google.com/iam/docs/service-accounts +[sidecar-example]: https://github.com/GoogleCloudPlatform/cloudsql-proxy/tree/master/examples/kubernetes#run-the-cloud-sql-proxy-as-a-sidecar +[source-install]: docs/install-from-source.md diff --git a/cmd/cloud_sql_proxy/cloud_sql_proxy.go b/cmd/cloud_sql_proxy/cloud_sql_proxy.go index 36b55f53f..05d976b88 100644 --- a/cmd/cloud_sql_proxy/cloud_sql_proxy.go +++ b/cmd/cloud_sql_proxy/cloud_sql_proxy.go @@ -49,24 +49,37 @@ import ( ) var ( - version = flag.Bool("version", false, "Print the version of the proxy and exit") - verbose = flag.Bool("verbose", true, "If false, verbose output such as information about when connections are created/closed without error are suppressed") + version = flag.Bool("version", false, "Print the version of the proxy and exit") + verbose = flag.Bool("verbose", true, + `If false, verbose output such as information about when connections are +created/closed without error are suppressed`, + ) quiet = flag.Bool("quiet", false, "Disable log messages") logDebugStdout = flag.Bool("log_debug_stdout", false, "If true, log messages that are not errors will output to stdout instead of stderr") - refreshCfgThrottle = flag.Duration("refresh_config_throttle", proxy.DefaultRefreshCfgThrottle, "If set, this flag specifies the amount of forced sleep between successive API calls in order to protect client API quota. Minimum allowed value is "+minimumRefreshCfgThrottle.String()) - checkRegion = flag.Bool("check_region", false, `If specified, the 'region' portion of the connection string is required for + refreshCfgThrottle = flag.Duration("refresh_config_throttle", proxy.DefaultRefreshCfgThrottle, + `If set, this flag specifies the amount of forced sleep between successive +API calls in order to protect client API quota. Minimum allowed value is + `+minimumRefreshCfgThrottle.String(), + ) + checkRegion = flag.Bool("check_region", false, `If specified, the 'region' portion of the connection string is required for Unix socket-based connections.`) // Settings for how to choose which instance to connect to. dir = flag.String("dir", "", "Directory to use for placing Unix sockets representing database instances") - projects = flag.String("projects", "", `Open sockets for each Cloud SQL Instance in the projects specified -(comma-separated list)`) - instances = flag.String("instances", "", `Comma-separated list of fully qualified instances (project:region:name) + projects = flag.String("projects", "", + `Open sockets for each Cloud SQL Instance in the projects specified +(comma-separated list)`, + ) + instances = flag.String("instances", "", + `Comma-separated list of fully qualified instances (project:region:name) to connect to. If the name has the suffix '=tcp:port', a TCP server is opened -on the specified port to proxy to that instance. Otherwise, one socket file per -instance is opened in 'dir'. You may use INSTANCES environment variable -for the same effect. Using both will use value from flag, Not compatible with -fuse`) +on the specified port on localhost to proxy to that instance. It is also possible +to listen on a custom address by providing a host, e.g., '=tcp:0.0.0.0:port'. If +no value is provided for 'tcp', one socket file per instance is opened in 'dir'. +You may use INSTANCES environment variable for the same effect. Using both will +use value from flag, Not compatible with -fuse.`, + ) instanceSrc = flag.String("instances_metadata", "", `If provided, it is treated as a path to a metadata value which is polled for a comma-separated list of instances to connect to. For example, to use the instance metadata value named 'cloud-sql-instances' you would @@ -77,23 +90,46 @@ directory at 'dir' must be empty before this program is started.`) can be removed automatically by this program.`) // Settings for limits - maxConnections = flag.Uint64("max_connections", 0, `If provided, the maximum number of connections to establish before refusing new connections. Defaults to 0 (no limit)`) - fdRlimit = flag.Uint64("fd_rlimit", limits.ExpectedFDs, `Sets the rlimit on the number of open file descriptors for the proxy to the provided value. If set to zero, disables attempts to set the rlimit. Defaults to a value which can support 4K connections to one instance`) - termTimeout = flag.Duration("term_timeout", 0, "When set, the proxy will wait for existing connections to close before terminating. Any connections that haven't closed after the timeout will be dropped") + maxConnections = flag.Uint64("max_connections", 0, + `If provided, the maximum number of connections to establish before refusing +new connections. Defaults to 0 (no limit)`, + ) + fdRlimit = flag.Uint64("fd_rlimit", limits.ExpectedFDs, + `Sets the rlimit on the number of open file descriptors for the proxy to +the provided value. If set to zero, disables attempts to set the rlimit. +Defaults to a value which can support 4K connections to one instance`, + ) + termTimeout = flag.Duration("term_timeout", 0, + `When set, the proxy will wait for existing connections to close before +terminating. Any connections that haven't closed after the timeout will be +dropped`, + ) // Settings for authentication. token = flag.String("token", "", "When set, the proxy uses this Bearer token for authorization.") - tokenFile = flag.String("credential_file", "", `If provided, this json file will be used to retrieve Service Account credentials. -You may set the GOOGLE_APPLICATION_CREDENTIALS environment variable for the same effect.`) - ipAddressTypes = flag.String("ip_address_types", "PUBLIC,PRIVATE", "Default to be 'PUBLIC,PRIVATE'. Options: a list of strings separated by ',', e.g. 'PUBLIC,PRIVATE' ") + tokenFile = flag.String("credential_file", "", + `If provided, this json file will be used to retrieve Service Account +credentials. You may set the GOOGLE_APPLICATION_CREDENTIALS environment +variable for the same effect.`, + ) + ipAddressTypes = flag.String("ip_address_types", "PUBLIC,PRIVATE", + `Default to be 'PUBLIC,PRIVATE'. Options: a list of strings separated by +',', e.g. 'PUBLIC,PRIVATE' `, + ) // Settings for IAM db proxy authentication enableIAMLogin = flag.Bool("enable_iam_login", false, "Enables database user authentication using Cloud SQL's IAM DB Authentication.") - skipInvalidInstanceConfigs = flag.Bool("skip_failed_instance_config", false, `Setting this flag will allow you to prevent the proxy from terminating when - some instance configurations could not be parsed and/or are unavailable.`) + skipInvalidInstanceConfigs = flag.Bool("skip_failed_instance_config", false, + `Setting this flag will allow you to prevent the proxy from terminating +when some instance configurations could not be parsed and/or are +unavailable.`, + ) // Setting to choose what API to connect to - host = flag.String("host", "", "When set, the proxy uses this host as the base API path. Example: https://sqladmin.googleapis.com") + host = flag.String("host", "", + `When set, the proxy uses this host as the base API path. Example: + https://sqladmin.googleapis.com`, + ) ) const ( @@ -126,8 +162,8 @@ Authorization: if they exist. * To configure the proxy using IAM authentication, pass the -enable_iam_login - flag. This will cause the proxy to use IAM account credentials for - database user authentication. + flag. This will cause the proxy to use IAM account credentials for + database user authentication. General: -quiet @@ -135,9 +171,11 @@ General: WARNING: this option disables ALL logging output (including connection errors), which will likely make debugging difficult. The -quiet flag takes precedence over the -verbose flag. + -log_debug_stdout When explicitly set to true, verbose and info log messages will be directed - to stdout as opposed to the default stderr. + to stdout as opposed to the default stderr. + -verbose When explicitly set to false, disable log messages that are not errors nor first-time startup messages (e.g. when new connections are established). @@ -147,28 +185,32 @@ Connection: To connect to a specific list of instances, set the instances parameter to a comma-separated list of instance connection strings. For example: - -instances=my-project:my-region:my-instance + -instances=my-project:my-region:my-instance For connectivity over TCP, you must specify a tcp port as part of the instance string. For example, the following example opens a loopback TCP socket on port 3306, which will be proxied to connect to the instance - 'my-instance' in project 'my-project': + 'my-instance' in project 'my-project'. To listen on other interfaces than + localhost, a custom bind address (e.g., 0.0.0.0) may be provided. For + example: - -instances=my-project:my-region:my-instance=tcp:3306 + -instances=my-project:my-region:my-instance=tcp:3306 + or + -instances=my-project:my-region:my-instance=tcp:0.0.0.0:3306 - When connecting over TCP, the -instances parameter is required. + When connecting over TCP, the -instances parameter is required. To set a custom socket name, you can specify it as part of the instance - string. The following example opens a unix socket in the directory - specified by -dir, which will be proxied to connect to the instance + string. The following example opens a unix socket in the directory + specified by -dir, which will be proxied to connect to the instance 'my-instance' in project 'my-project': - -instances=my-project:my-region:my-instance=unix:custom-socket-name + -instances=my-project:my-region:my-instance=unix:custom-socket-name - To override the -dir parameter, specify an absolute path as shown in the - following example: + To override the -dir parameter, specify an absolute path as shown in the + following example: - -instances=my-project:my-region:my-instance=unix:/my/custom/sql-socket + -instances=my-project:my-region:my-instance=unix:/my/custom/sql-socket Supplying INSTANCES environment variable achieves the same effect. One can use that to keep k8s manifest files constant across multiple environments @@ -185,7 +227,7 @@ Connection: To direct the proxy to allow connections to all instances in specific projects, set the projects parameter: - -projects=my-project + -projects=my-project -fuse If your local environment has FUSE installed, you can specify the -fuse diff --git a/proxy/README.md b/proxy/README.md index affeb9893..9ac2347bd 100644 --- a/proxy/README.md +++ b/proxy/README.md @@ -2,16 +2,32 @@ You can also use the Cloud SQL proxy directly from a Go program. -These packages are primarily used as implementation for the Cloud SQL proxy command, -and may be changed in backwards incompatible ways in the future. +These packages are primarily used as implementation for the Cloud SQL proxy +command, and may be changed in backwards incompatible ways in the future. + +## Usage + +If your program is written in [Go](https://golang.org) you can use the Cloud SQL +Proxy as a library, avoiding the need to start the Proxy as a companion process. + +Alternatively, there are Cloud SQL Connectors for [Java][] and [Python][]. -## To use inside a Go program: -If your program is written in [Go](https://golang.org) you can use the Cloud SQL Proxy as a library, -avoiding the need to start the Proxy as a companion process. ### MySQL -If you're using the the MySQL [go-sql-driver](https://github.com/go-sql-driver/mysql) -you can use helper functions found in the [`proxy/dialers/mysql` package](https://godoc.org/github.com/GoogleCloudPlatform/cloudsql-proxy/proxy/dialers/mysql). See [example usage](https://github.com/GoogleCloudPlatform/cloudsql-proxy/blob/main/tests/dialers_test.go). + +If you're using the MySQL [go-sql-driver][go-mysql] you can use helper +functions found in the [`proxy/dialers/mysql`][mysql-godoc] + +See [example usage](tests/dialers_test.go). ### Postgres -If you're using the the Postgres [lib/pq](https://github.com/lib/pq), you can use the `cloudsqlpostgres` driver from [here](https://github.com/GoogleCloudPlatform/cloudsql-proxy/tree/main/proxy/dialers/postgres). See [example usage](https://github.com/GoogleCloudPlatform/cloudsql-proxy/blob/main/proxy/dialers/postgres/hook_test.go). + +If you're using the Postgres [lib/pq](https://github.com/lib/pq), you can +use the `cloudsqlpostgres` driver from [here](proxy/dialers/postgres). + +See [example usage](proxy/dialers/postgres/hook_test.go). + +[Java]: https://github.com/GoogleCloudPlatform/cloud-sql-jdbc-socket-factory +[Python]: https://github.com/GoogleCloudPlatform/cloud-sql-python-connector +[go-mysql]: https://github.com/go-sql-driver/mysql +[mysql-godoc]: https://pkg.go.dev/github.com/GoogleCloudPlatform/cloudsql-proxy/proxy/dialers/mysql