Docker watcher should back off on authentication errors

[swiatekm]

Currently, the docker watcher will check if the docker socket is accessible by calling the /info endpoint, and will then proceed to consume a stream of events from the /events endpoint. The latter is restarted every second whenever it ends, no matter the reason.

In effect, if the /events endpoint is protected by an authorization plugin, the watcher will continue trying to access it, even if no useful progress can be made, and will continue logging error messages about it. This is far from ideal and should be fixed.

The simplest fix would be to check /events at watcher creation, but the call is unfortunately asynchronous and we'd have to take care to wrap the check in a timeout. In general, we should do an exponential backoff on calls to the watch function, as well as actually checking the error it encountered.

This was raised as an issue for the add_docker_metadata processor in beats and the docker provider in elastic-agent.