Formalize and document the Supervisor's HTTP gateway API

[christophermaier]

This is analogous to the work done in #4823 to clean up the template rendering context data.

The data we expose through the HTTP endpoint is essentially the JSON serialization of several internal data structures. This is not great for all the reasons I laid out in #4761, so please read that for more. There are some particular wrinkles for this work that I'd like to call out explicitly, however.

When rendering Services, we include a lot of information that just isn't necessary. For instance, every service renders out Supervisor-level information, as well as Manager-level information; in addition to being duplicated across every rendered Service, it's also not actually service information! Additionally, since we're automatically deriving our serialization logic, we include a number of implementation-only details. For example, the needs_reload and needs_reconfiguration flags are essentially there to guide the service "state machine" (it's not explicitly implemented as a state machine), and are of no relevance to anyone else.

Additionally, because of how we essentially create a "dummy" Service for all non-running services that have been loaded into the Supervisor, we limit ourselves in other parts of the code. For instance, we could take care of cleanup tasks like removal of PID files automatically in a Drop implementation, but since we end up dropping Service objects when persisting state for the HTTP endpoint, this isn't a great idea (in real-world terms, it would probably be OK, since the only Service objects that get dropped when persisting state are ones that correspond to non-running services, but this still doesn't feel great.)

While we are actually limiting what we render in the case of Services by marking some fields with the serde(skip_serializing) attribute, we're not doing it everywhere we should. Furthermore, it's not actually clear that the things that are being rendered are being rendered in the best form.

The above is focused mainly on Services (and thus, the /services HTTP endpoint), since I happen to have stumbled across this in the context of some work on them, but similar arguments likely hold for our /census and /butterfly endpoints.

We should survey all the information we currently return from the HTTP endpoint and determine what is actually useful to return. We can then formalize this in code with types and serialization implementations that are devoted solely to this use, and decoupled from other internal data structures (again, see #4823 for concrete examples of this). We can then document this formally using JSON schema (as we have done with the template rendering context) as well as HTTP API documentation using something like the OpenAPI specification.

[raskchanky]

Make sure to include the toggling of the sensitive bits from #5550