[Feature Request] Support json configuration file for service limit overrides

[Mortinke]

Feature Request

Support json configuration file for service limit overrides

Feature Description

At the moment, we can override service limits with the -L argument specified in "service_name/limit_name=value" format. For 10 services, the command can quickly exceed 200 characters. I'd be grateful if we could specify these limits in a json configuration file something like that:

service_limits.json

{
    'AutoScaling': {
        'Auto Scaling groups': 100,
        'Launch configurations': 200,
    },
    'EBS': {
        'General Purpose (SSD) volume storage (GiB)': 30720,
        'Magnetic volume storage (GiB)': 30720,
    },
    'EC2': {
        'Running On-Demand EC2 instances': 1000,
        'Running On-Demand c1.medium instances': 1000,
    },
    'ELB': {
        'Active load balancers': 100,
    },
}

Supporting S3 files would be a dream, but isn't necessary.

[jantman]

@Mortinke This is supported via the Python API. The example available at https://github.com/jantman/awslimitchecker/blob/master/docs/examples/check_aws_limits.py (which is based on what we're doing at my current employer) shows how to do that with a dictionary of limit and threshold overrides in the Python source.

There's also a project that provides a web UI for awslimitchecker: https://github.com/Yipit/awslimits

I'm happy to look into adding a bit to the example script if needed, and adding some more links to the example script throughout the documentation to make it easier to find.

But unless someone contributes it as a fully-working pull request with complete code coverage, this isn't a code feature that I'm going to implement myself. As far as I know, most or all users that have need of many overrides are using something at least vaguely like the example script linked above, and providing the overrides via the Python API.

[Mortinke]

Hi @jantman,

thanks for your quick reply. The web UI for awslimitchecker looks interesting, thanks for the link!

As already mentioned in #323, separate wrapper scripts for each account will not scale in our company.
My idea was to read/parse the configuration file at bootstrap and then execute the API calls - similar to your wrapper script.

I'm not a python expert but I can look in other solutions which we currently use e.g. afp-core, SecurityMonkey, aardvark. These tools support configuration files and maybe I'm able to create a pull request for this ticket.

[jantman]

@Mortinke To clarify, I was suggesting a single wrapper script that parses the configuration file.

Why don't we leave this open for now and work on a wrapper that handles multiple accounts with overrides in #323, and in the meantime see if anyone else is interested in merging this into the core.

Comments/feedback from others welcome.

[Mortinke]

Sounds good!

[Mortinke]

Solved with #323! Thanks again!

[jantman]

Great! Any time!

[jantman]

It turns out that my employer now has need for this and is willing to let me develop it during my work hours.

I'll be merging the JSON overrides file support that's present in the docs/examples/check_aws_limits.py and docs/examples/multi-region_multi-account/ examples into awslimitchecker proper.

I'll likely also add support for s3:// URLs to retrieve the override files directly from S3.

[jantman]

This was merged to master in #427 and released as 7.1.0. The release is now live on PyPI and the Docker image should be live on Docker Hub shortly.