Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding new guides for developing and deploying an ML application in S… #36700

Merged
merged 1 commit into from
Jun 29, 2023

Conversation

akshay-anyscale
Copy link
Contributor

…erve and configuring serve deployments.

The change also includes refactoring information in some existing guides.

Why are these changes needed?

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(


These are all the properties available on a Ray Serve deployment. This is also available in the [API reference](../serve/api/doc/ray.serve.deployment_decorator.rst)

The following can be configured through either the serve config file or on the `@serve.deployment` decorator:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few updates should be made to this content:

  • Show a Python code sample and an equivalent config file.
  • Show the usage of .options().
  • Explain the precedence (config > what's in the code).

I think we should also consolidate the content in the "app builder" page here to make it clear how to compose them. WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed I'll add this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the 3 (actually moved them over from the serve config page because I think they're more relevant here). Also updated the serve config page. For the app builder, do you propose merging app builder content entirely into this page?

doc/source/serve/key-concepts.md Outdated Show resolved Hide resolved
doc/source/serve/key-concepts.md Outdated Show resolved Hide resolved
@edoakes
Copy link
Contributor

edoakes commented Jun 22, 2023

Some comments on the "configuring deployments" page. I'm OK with those changes being a follow-up, but I think they'll significantly reduce confusion.

doc/source/serve/configure-serve-deployment.md Outdated Show resolved Hide resolved
doc/source/serve/configure-serve-deployment.md Outdated Show resolved Hide resolved
doc/source/serve/configure-serve-deployment.md Outdated Show resolved Hide resolved
doc/source/serve/configure-serve-deployment.md Outdated Show resolved Hide resolved
doc/source/serve/configure-serve-deployment.md Outdated Show resolved Hide resolved
Copy link
Contributor

@angelinalg angelinalg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly copy edits. There are a few suggestions for clarification mixed in. Great work!!

doc/source/serve/advanced-guides/deploy-vm.md Outdated Show resolved Hide resolved
doc/source/serve/advanced-guides/deploy-vm.md Outdated Show resolved Hide resolved
doc/source/serve/advanced-guides/deploy-vm.md Outdated Show resolved Hide resolved
doc/source/serve/advanced-guides/deploy-vm.md Outdated Show resolved Hide resolved
doc/source/serve/advanced-guides/deploy-vm.md Outdated Show resolved Hide resolved
doc/source/serve/production-guide/handling-dependencies.md Outdated Show resolved Hide resolved
doc/source/serve/production-guide/handling-dependencies.md Outdated Show resolved Hide resolved
doc/source/serve/production-guide/handling-dependencies.md Outdated Show resolved Hide resolved
doc/source/serve/production-guide/handling-dependencies.md Outdated Show resolved Hide resolved
doc/source/serve/develop-and-deploy.md Show resolved Hide resolved
doc/source/serve/develop-and-deploy.md Outdated Show resolved Hide resolved
doc/source/serve/develop-and-deploy.md Outdated Show resolved Hide resolved
@angelinalg
Copy link
Contributor

angelinalg commented Jun 24, 2023

Some content suggestions:

  • Provide guidance on how to determine num_replicas; talk about how it's related to the traffic
  • Should we mention the autoscaler or scheduler?
  • Provide guidance on determining num_gpus; related to size of models
  • Demonstrate the strengths of Ray - if it's something like zero-downtime upgrades, does it makes sense to briefly explain why or when you would care about that
  • The content seems geared towards ML practitioners. The points above may not require explanation for ML practitioners, but should we provide guidance for app developers?
  • Consider using some material (like graphics) from the training module: https://github.com/ray-project/ray-educational-materials/blob/main/Ray_Serve/Multilingual_Chat_with_Ray_Serve.ipynb
  • The definitions of replica and autoscaler in the Glossary don't seem to align with the way they are discussed in the Serve docs. Should we update them?

@akshay-anyscale
Copy link
Contributor Author

Addressed all the comments and updated the examples in the Serve config files page to use the multi-app schema

@akshay-anyscale akshay-anyscale force-pushed the serve_doc_changes branch 3 times, most recently from c195441 to 76ddc36 Compare June 27, 2023 17:22
doc/source/serve/configure-serve-deployment.md Outdated Show resolved Hide resolved
doc/source/serve/configure-serve-deployment.md Outdated Show resolved Hide resolved
doc/source/serve/develop-and-deploy.md Outdated Show resolved Hide resolved
doc/source/serve/develop-and-deploy.md Outdated Show resolved Hide resolved
doc/source/serve/develop-and-deploy.md Show resolved Hide resolved
doc/source/serve/develop-and-deploy.md Outdated Show resolved Hide resolved
doc/source/serve/key-concepts.md Outdated Show resolved Hide resolved
@akshay-anyscale
Copy link
Contributor Author

Updated with new feedback

Copy link
Contributor

@edoakes edoakes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, great improvements, thanks

doc/source/serve/configure-serve-deployment.md Outdated Show resolved Hide resolved
doc/source/serve/configure-serve-deployment.md Outdated Show resolved Hide resolved
doc/source/serve/configure-serve-deployment.md Outdated Show resolved Hide resolved
doc/source/serve/configure-serve-deployment.md Outdated Show resolved Hide resolved
doc/source/serve/configure-serve-deployment.md Outdated Show resolved Hide resolved
doc/source/serve/configure-serve-deployment.md Outdated Show resolved Hide resolved
doc/source/serve/develop-and-deploy.md Outdated Show resolved Hide resolved
doc/source/serve/develop-and-deploy.md Outdated Show resolved Hide resolved
@akshay-anyscale akshay-anyscale force-pushed the serve_doc_changes branch 3 times, most recently from ce6d98b to 0b7511d Compare June 28, 2023 20:09
…erve and configuring serve deployments.

The change also includes refactoring information in some existing guides.

Signed-off-by: Akshay Malik <akshay@anyscale.com>
@edoakes edoakes merged commit 74efb90 into master Jun 29, 2023
@edoakes edoakes deleted the serve_doc_changes branch June 29, 2023 01:23
arvind-chandra pushed a commit to lmco/ray that referenced this pull request Aug 31, 2023
ray-project#36700)

The change also includes refactoring information in some existing guides.

Signed-off-by: e428265 <arvind.chandramouli@lmco.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants