Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[serving] support load multiple version of a model on the same endpoint #1052

Merged
merged 1 commit into from
Jun 26, 2021

Conversation

frankfliu
Copy link
Contributor

  • [serving] Remove GpuAssignmentStrategy.java
  • [serving] support load multiple version of a model on the same endpoint

Description

Brief description of what this PR is about

  • If this change is a backward incompatible change, why must this change be made?
  • Interesting edge cases to note here

@frankfliu frankfliu changed the title version [serving] support load multiple version of a model on the same endpoint Jun 25, 2021
@codecov-commenter
Copy link

codecov-commenter commented Jun 25, 2021

Codecov Report

Merging #1052 (584f758) into master (3b88cf9) will decrease coverage by 0.09%.
The diff coverage is 61.17%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #1052      +/-   ##
============================================
- Coverage     70.12%   70.02%   -0.10%     
- Complexity     5197     5216      +19     
============================================
  Files           509      510       +1     
  Lines         23147    23250     +103     
  Branches       2455     2489      +34     
============================================
+ Hits          16232    16281      +49     
- Misses         5605     5643      +38     
- Partials       1310     1326      +16     
Impacted Files Coverage Δ
.../java/ai/djl/repository/RepositoryFactoryImpl.java 76.74% <0.00%> (-1.83%) ⬇️
...ing/src/main/java/ai/djl/serving/wlm/Endpoint.java 42.22% <42.22%> (ø)
...ng/src/main/java/ai/djl/serving/wlm/ModelInfo.java 77.19% <60.00%> (-3.51%) ⬇️
...a/ai/djl/serving/http/InferenceRequestHandler.java 52.12% <61.11%> (+0.36%) ⬆️
...ving/src/main/java/ai/djl/serving/ModelServer.java 51.46% <61.90%> (+0.80%) ⬆️
...src/main/java/ai/djl/serving/wlm/ModelManager.java 79.62% <66.66%> (-10.03%) ⬇️
.../main/java/ai/djl/serving/wlm/WorkLoadManager.java 64.07% <70.00%> (ø)
.../ai/djl/serving/http/ManagementRequestHandler.java 91.05% <88.88%> (-1.19%) ⬇️
.../serving/src/main/java/ai/djl/serving/wlm/Job.java 66.66% <100.00%> (ø)
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b88cf9...584f758. Read the comment docs.

Change-Id: I41824ed1e7ae2cec7d779e5dc3e821d9a695fabc
@frankfliu frankfliu merged commit 403b3ab into deepjavalibrary:master Jun 26, 2021
@frankfliu frankfliu deleted the version branch June 26, 2021 03:31
frankfliu added a commit to frankfliu/djl that referenced this pull request Jun 27, 2021
…nt (deepjavalibrary#1052)

Change-Id: I41824ed1e7ae2cec7d779e5dc3e821d9a695fabc
Lokiiiiii pushed a commit to Lokiiiiii/djl that referenced this pull request Oct 10, 2023
* [docker] Upgrades to inf2 2.13.2 version

* Fixes torch-neuronx pip install package name
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants