Better documentation for -max_connections flag

[zepouet]

Feature Description

To be able to limit the number of connections or requests on the database via the proxy.

In the case of a container for which we do not master the code but also the configuration, it would be good to be able to limit, as well as the memory, the CPU ... the fact of calling too much Google Cloud SQL.

For example if you have 20 pods using the same CloudSQL instance with 20 databases, one nasty pod that consumes too much I/O from the DB could penalize other application pods.

In all highly shared environments, it would be a big advantage to protect yourself from this kind of side effect.

Obviously if we have control over the code and the applications, it is the developers' duty to configure the connection pools, but in the opposite case, we have to create x Google Cloud SQL instances to protect ourselves but often customers do not want and cannot.

Alternatives Considered

Either use existing features of MySQL and Postgre drivers or implement this on the proxy side to have something uniform

Additional Context

[zepouet]

@kurtisvg If you need help later about it, it will be nice to participate. Thanks

[kurtisvg]

Hi @zepouet - Can I get a better sense of what specifically you want limited? Do you want _rate_limiting specifically?

The original issue mentions both number of connections and requests. There are currently limits for both - you can limit the number of connections with the -max_connections flag, and the requests made through the sqladmin api are rate limited to 2x per minute. Are you hoping for more control over these limits, or hoping to limit something else?

[Carrotman42]

I believe this request is for "per-client" limits, e.g. it's not sufficient to have a max_connections set to 1000 when you have a rogue pod using up all of those connection slots when it shouldn't be. The request is to be able to say "limit each individual entity connecting to the cloud_sql_proxy to 100 connections" so that one pod cannot prevent other pods from being able to make a connection. I for one think it's better to solve this at the pod level of the rogue process, rather than try to centralize a policy in the Proxy. That is, if I had a rogue process which shouldn't be allowed to make more than 100 connections, then I'd try limiting the number of file descriptors that process is allowed to make. I suppose this may or may not work well depending on the situation, though. To @zepouet: have you tried limiting the count of file descriptors that other "nasty" pod is allowed to make?

…

On Tue, Oct 13, 2020 at 10:50 AM Kurtis Van Gent ***@***.***> wrote: Hi @zepouet <https://github.com/zepouet> - Can I get a better sense of what specifically you want limited? Do you want _rate_limiting specifically? The original issue mentions both number of connections and requests. There are currently limits for both - you can limit the number of connections with the -max_connections flag <https://github.com/GoogleCloudPlatform/cloudsql-proxy/blob/21d19f3fd27dedbd33096b1a6d309daacefe7700/cmd/cloud_sql_proxy/cloud_sql_proxy.go#L80>, and the requests made through the sqladmin api are rate limited to 2x per minute. Are you hoping for more control over these limits, or hoping to limit something else? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#498 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAELF323ZWYFL2EIBWYRQMLSKSAO5ANCNFSM4SL7C3HQ> .

[zepouet]

Thanks @Carrotman42 @kurtisvg

To be honest I didn't test with this option because I didn't know it.
I don't find any references in the documentation and I am silly not to search with help command.
So maybe you solve my problem.

My usecase is a production platform with GKE and one Google Cloud SQL instance (Mysql)
We have many K8S deployments (200) and each deployment has a dedicated database on the same Cloud SQL Instance.
For cost reasons we have WAF in front of a number of pods but we cannot put it in front of all pods.
So if we have a DDOS or just a foreseen performance test (with Gatling or JMeter) on a dedicated website, we could penalize other website due to the common database. The pods use the sidecar pattern with application and cloudsqlproxy.

My idea is to have a way to limit Request and Database connections to isolate less critical websites.
Thanks to you, it seems to have a limit on connection so we could be nice.
Ideally with a request rating, it will be perfect so the sidecar container could play a complete work.
Maybe I am wrong.

A solution would be to create a dedicated Cloud SQL Instance for critical websites but I wonder how to rate limiting a pool of websites.

Maybe add this option to the online documentation for other guy like me :)

Thanks you very much for your comments.
Nicolas

[kurtisvg]

@zepouet here are some broad recommendations:

1. Use different instances over just different databases on the same instance if the database is otherwise unrelated. It's (usually) better to run smaller more isolated instances instead of bigger pooled instances so your failure points are separated.

2. A more engineering heavy option to limit/optimize throughput would be to use a connection or transactional pool (something like ProxySQL or Pgbouncer). You could deploy this as a service with a proxy sidecar, and have different services per deployment (or groups of deployments depending on the situation). This involves a bit more work as you now have to manage things like encryption and cross zone/region traffic to these services. It can also give you other options like specify replicas.

I would still recommend application level connection pooling basically always.

[zepouet]

Thanks @Carrotman42 and @kurtisvg for your help.
Better title and good solution.

Have a nice day and thanks a lot.
Best regards
Nicolas