Consider supporting import of multiple definition files on boot

[artandor]

Enhancement idea

It would be nice to be able to config multiple definition files in the configuration file. It would make industrialisation a lot easier.
For example, the distributing team would have a default file in which she sets his best practice parameters, users, ... and the business unit would be able to have its own file.

It would also help a team to be sure that its default behaviors are not overwritten by some deployment tools (puppet, ansible, ...)

[michaelklishin]

I don't see how having multiple files would protect against deployment tools doing the wrong thing.

We see similar requests in other areas. It is already possible to have multiple plugin directories, for example.

[artandor]

Thanks for moving it to the right project.

You could have a default definition file and a deployed definition file, for example. If we could specify a weight (priority) to chose which file to execute first and last we could ensure that the default behavior is respected. Mostly usefull for policies.

[michaelklishin]

Thank you for the idea but having weights and priorities sounds complicated. As you know we already have a priority system for policies, some users find it confusing or have no idea it exists.

We don't want to tie ourselves to a specific deployment scenario either. So I think we will provide a way to specify multiple definition files one way or another and import them in order. It can be a directory, for example, or a collection of paths in the config files similar to how you configure e.g. TLS versions or cipher suites (order matters for the latter).

[lukebakken]

The simplest implementation I can think of would be to allow management.load_definitions to point to a single file or to a directory. If a directory, all files will be attempted to be imported in the order returned by file:list_dir/1.

Should an error importing one file stop the process? There is no rollback so I don't see the benefit of stopping, to be honest.

[artandor]

Yeah it's true that priorities are maybe overkill. Loading them in an order is more that enough.

How is the current definition loading working ? Is he executing the management command ? If yes, i could probably try a PR since it shouldn't be excessively complicated to loop over those files.

@lukebakken I guess for this specific use, a rollback procedure could be implemented by dumping the conf before doing anything, and reloading it in case of problem. But that pulls up the problem that the node needs to be alive to dump the conf.

[lukebakken]

Rollback is out of scope for this.

[michaelklishin]

Making management.load_definitions support directory paths and sort files by name is a good idea. Lots of UNIX-like operating systems and tools use this method so at least it won't be surprising.

[michaelklishin]

Definition loading is triggered by a boot step which then runs this function.

[artandor]

Supporting directory path seems better than supporting a list of path indeed.

The modification concerns this part ?

        _    -> case file:read_file(File) of
                    {ok, Body} -> rabbit_log:info(
                                    "Applying definitions from: ~s", [File]),
                                  load_definitions(Body);

where we should add a case where the path is a directory, in which case we loop over files contained in that directory with the load_definitions function ?

[lukebakken]

@artandor please see #671

[artandor]

I'm impressed how fast you guys reacted. Do you know when this will be available to use ? Can i close the issue ?

[michaelklishin]

We haven't merged #671 so please keep the issue open.

[jersonmartinez]

Get a lot of files about JSON Configuration

Edit file rabbitmq.conf add this line: load_definitions = /etc/rabbitmq/definitions/

Create a directory /etc/rabbitmq/definitions/

Insert file definitions.json inside of the new directory, in addition, you can insert other JSON definitions files to be recognized.

Restart service and done!

[michaelklishin]

You don't need to reboot, restarting the RabbitMQ node is all that's necessary.