Draft deployment guidelines

[mpangrazzi]

The purpose of this document is simply to make people more aware of how pipelines can be deployed and called on Hayhooks in different environment (single vs multiple workers).

TODO:

• Basic recommendations on deployment in single vs multiple worker environments
• end-to-end examples (I/O bound vs CPU bound) Update: will add it in a separate PR

[julian-risch]

Looks quite good to me already. I just made four tiny suggestions and then what we have so far is ready to be merged from my side. Let's wait for Stefano's review on Monday.

[mpangrazzi]

@julian-risch thanks! Yes, let's wait for @anakin87 review.

Probably it could worth to add an end-to-end example or tutorial (build a simple pipeline, serialise it to YAML, run Hayhooks and run it), WDYT?

[anakin87]

I like the guidelines.
They should be understandable for someone familiar with these topics and provide a good introduction to others.

• We should probably link them (or copy them?) in our docs: https://docs.haystack.deepset.ai/docs/hayhooks; https://docs.haystack.deepset.ai/docs/deployment
• An end-to-end example would be helpful
• Keep in mind that there is an (unmaintained) demo based on Hayhooks: https://github.com/deepset-ai/haystack-demos/tree/main/qdrant_indexing

[mpangrazzi]

@anakin87 ah, didn't know about that existing demo. I think it would be good to have a similar one here, e.g. on an examples folder or so. I can take care of it.

[mpangrazzi]

@julian-risch @anakin87 I'll merge this and add the examples in a separate PR