Skip to content

Commit

Permalink
Remove instanceType field in aks arc doc and rename HCI to Azure Local (
Browse files Browse the repository at this point in the history
#702)

**Reason for Change**:
<!-- What does this PR improve or fix in Kaito? Why is it needed? -->

**Requirements**

- [ ] added unit tests and e2e tests (if applicable).

**Issue Fixed**:
<!-- If this PR fixes GitHub issue 4321, add "Fixes #4321" to the next
line. -->

**Notes for Reviewers**:

---------

Co-authored-by: qfai <quke@microsoft.com>
  • Loading branch information
qfai and qfai authored Dec 7, 2024
1 parent d783693 commit 11f9cd1
Showing 1 changed file with 4 additions and 5 deletions.
9 changes: 4 additions & 5 deletions docs/How-to-use-kaito-in-aks-arc.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,13 @@ Currently KAITO supports models such as Falcon, Phi2, Phi3, Llama2, Llama2Chat,
- We recommend using Linux machine for this feature.
- Your local kubectl environment configured to point to your AKS cluster.
- Run `az aksarc get-credentials --resource-group <ResourceGroupName> --name <ClusterName> --admin` to download the kubeconfig file.
2. Make sure your HCI cluster is enabled with GPU, you can ask your infrastructure administrator to set it up for you. You also need to identify the right VM SKUs for your AKS cluster before creating the node pool. The instruction can be found at [use GPU for compute-intensive workloads](https://learn.microsoft.com/en-us/azure/aks/hybrid/deploy-gpu-node-pool).
2. Make sure your Azure Local cluster is enabled with GPU, you can ask your infrastructure administrator to set it up for you. You also need to identify the right VM SKUs for your AKS cluster before creating the node pool. The instruction can be found at [use GPU for compute-intensive workloads](https://learn.microsoft.com/en-us/azure/aks/hybrid/deploy-gpu-node-pool).
3. Make sure the helm and kubectl are installed in your local machine.
- If you need to install or upgrade, please see instruction from [Install Helm](https://helm.sh/docs/intro/install/).
- If you need to install kubectl, please see instructions from [Install kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/).

## Create a GPU Node Pool
Available GPU sku can be found [here](https://learn.microsoft.com/en-us/azure/aks/hybrid/deploy-gpu-node-pool#supported-vm-sizes)
<details>
<summary><b>Using Azure Portal</b></summary>
<div align="middle">
Expand All @@ -30,7 +31,7 @@ Currently KAITO supports models such as Falcon, Phi2, Phi3, Llama2, Llama2Chat,
<details>
<summary><b>Using Az CLI</b></summary>
<div align="middle">
Run following Az command to provision node pool, available GPU sku can be found <a href="https://learn.microsoft.com/en-us/azure/aks/hybrid/deploy-gpu-node-pool#supported-vm-sizes">here</a>
Run following Az command to provision node pool

```bash
az aksarc nodepool add --name "samplenodepool" --cluster-name "samplecluster" --resource-group "sample-rg" –node-vm-size "samplenodepoolsize" –os-type "Linux"
Expand Down Expand Up @@ -97,7 +98,6 @@ kind: Workspace
metadata:
name: { YourDeploymentName }
resource:
instanceType: Standard_NC12s_v3
labelSelector:
matchLabels:
apps: { YourNodeLabel }
Expand All @@ -115,7 +115,6 @@ kind: Workspace
metadata:
name: workspace-falcon-7b
resource:
instanceType: Standard_NC12s_v3
labelSelector:
matchLabels:
apps: falcon-7b
Expand All @@ -125,7 +124,7 @@ inference:
preset:
name: falcon-7b-instruct
```
2. You need to label your GPU node first, `Kubectl label node samplenode app=YourNodeLabel` and then apply the YAML file
2. You need to label your GPU node first, `Kubectl label node samplenode apps=YourNodeLabel` and then apply the YAML file
`kubectl apply -f sampleyamlfile.yaml`


Expand Down

0 comments on commit 11f9cd1

Please sign in to comment.