diff --git a/docs/How-to-use-kaito-in-aks-arc.md b/docs/How-to-use-kaito-in-aks-arc.md index b2a29f65c..2ac678e68 100644 --- a/docs/How-to-use-kaito-in-aks-arc.md +++ b/docs/How-to-use-kaito-in-aks-arc.md @@ -14,12 +14,13 @@ Currently KAITO supports models such as Falcon, Phi2, Phi3, Llama2, Llama2Chat, - We recommend using Linux machine for this feature. - Your local kubectl environment configured to point to your AKS cluster. - Run `az aksarc get-credentials --resource-group --name --admin` to download the kubeconfig file. -2. Make sure your HCI cluster is enabled with GPU, you can ask your infrastructure administrator to set it up for you. You also need to identify the right VM SKUs for your AKS cluster before creating the node pool. The instruction can be found at [use GPU for compute-intensive workloads](https://learn.microsoft.com/en-us/azure/aks/hybrid/deploy-gpu-node-pool). +2. Make sure your Azure Local cluster is enabled with GPU, you can ask your infrastructure administrator to set it up for you. You also need to identify the right VM SKUs for your AKS cluster before creating the node pool. The instruction can be found at [use GPU for compute-intensive workloads](https://learn.microsoft.com/en-us/azure/aks/hybrid/deploy-gpu-node-pool). 3. Make sure the helm and kubectl are installed in your local machine. - If you need to install or upgrade, please see instruction from [Install Helm](https://helm.sh/docs/intro/install/). - If you need to install kubectl, please see instructions from [Install kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/). ## Create a GPU Node Pool +Available GPU sku can be found [here](https://learn.microsoft.com/en-us/azure/aks/hybrid/deploy-gpu-node-pool#supported-vm-sizes)
Using Azure Portal
@@ -30,7 +31,7 @@ Currently KAITO supports models such as Falcon, Phi2, Phi3, Llama2, Llama2Chat,
Using Az CLI
-Run following Az command to provision node pool, available GPU sku can be found here +Run following Az command to provision node pool ```bash az aksarc nodepool add --name "samplenodepool" --cluster-name "samplecluster" --resource-group "sample-rg" –node-vm-size "samplenodepoolsize" –os-type "Linux" @@ -97,7 +98,6 @@ kind: Workspace metadata: name: { YourDeploymentName } resource: - instanceType: Standard_NC12s_v3 labelSelector: matchLabels: apps: { YourNodeLabel } @@ -115,7 +115,6 @@ kind: Workspace metadata: name: workspace-falcon-7b resource: - instanceType: Standard_NC12s_v3 labelSelector: matchLabels: apps: falcon-7b @@ -125,7 +124,7 @@ inference: preset: name: falcon-7b-instruct ``` -2. You need to label your GPU node first, `Kubectl label node samplenode app=YourNodeLabel` and then apply the YAML file +2. You need to label your GPU node first, `Kubectl label node samplenode apps=YourNodeLabel` and then apply the YAML file `kubectl apply -f sampleyamlfile.yaml`