Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NIM on GKE Tutorial #737

Merged
merged 16 commits into from
Jul 29, 2024
Merged

NIM on GKE Tutorial #737

Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
1fa4973
Merge pull request #4 from GoogleCloudPlatform/main
brandonroyal Jul 15, 2024
291f353
.gitignore
brandonroyal Jul 15, 2024
d7bf2e1
added README.md
brandonroyal Jul 15, 2024
e428eed
added pvc.yaml
brandonroyal Jul 15, 2024
fe7e05f
fixed markdown mismatch
brandonroyal Jul 15, 2024
c933269
cleanup
brandonroyal Jul 22, 2024
cf8a9ee
updated prerequisites
brandonroyal Jul 22, 2024
50dbce9
cleanup .gitignore
brandonroyal Jul 23, 2024
c3cc063
fixed incorrect jq reference
brandonroyal Jul 23, 2024
e734565
added ngc prerequisite
brandonroyal Jul 23, 2024
7af8dea
fixed ZONE references
brandonroyal Jul 23, 2024
365d573
removed redundant secrets
brandonroyal Jul 23, 2024
0672094
added namespace and removed context setting command
brandonroyal Jul 23, 2024
d081f4b
cleanup pvc + explaination
brandonroyal Jul 23, 2024
e04ca10
cleanup pvc file save
brandonroyal Jul 23, 2024
b53c8a9
cleanup language reference to local files and delete pvc file
brandonroyal Jul 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
added namespace and removed context setting command
  • Loading branch information
@brandonroyal
brandonroyal committed Jul 23, 2024
commit 0672094112e5e34efcadaf39c2207aef0e12762c
3 changes: 1 addition & 2 deletions tutorials-and-examples/nvidia-nim/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,6 @@ ngc registry model list
4. Create a Kuberntes namespace and switch context to that namespace
```bash
kubectl create namespace nim
kubectl config set-context --current --namespace nim
```

## Deploy a PVC to persist the model
Expand Down Expand Up @@ -128,7 +127,7 @@ helm --namespace nim install demo-nim nim-llm/ --set model.ngcAPIKey=$NGC_CLI_AP
## Test the NIM
1. Expose the service
```bash
kubectl port-forward services/demo-nim-nim-llm 8000
kubectl port-forward --namespace nim services/demo-nim-nim-llm 8000
```

2. Send a test prompt - A100
Expand Down