-
Notifications
You must be signed in to change notification settings - Fork 70
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: Add Adapter Loading Test for E2E Image Preset Workflow (#567)
**Reason for Change**: This PR adds a test for loading a mental health adapter onto the falcon-7b model. And testing to make sure the adapter was loaded onto the model successfully using `inference_api.py` logs.
- Loading branch information
1 parent
8b7aabf
commit 1f06382
Showing
4 changed files
with
130 additions
and
16 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
13 changes: 13 additions & 0 deletions
13
presets/test/manifests/falcon-7b-adapter/falcon-7b-adapter-service.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| apiVersion: v1 | ||
| kind: Service | ||
| metadata: | ||
| name: falcon-7b-adapter | ||
| spec: | ||
| selector: | ||
| app: falcon | ||
| ports: | ||
| - protocol: TCP | ||
| port: 80 | ||
| targetPort: 5000 | ||
| type: LoadBalancer | ||
| publishNotReadyAddresses: true |
71 changes: 71 additions & 0 deletions
71
presets/test/manifests/falcon-7b-adapter/falcon-7b-adapter.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| apiVersion: apps/v1 | ||
| kind: Deployment | ||
| metadata: | ||
| name: falcon-7b-adapter | ||
| spec: | ||
| replicas: 1 | ||
| selector: | ||
| matchLabels: | ||
| app: falcon | ||
| template: | ||
| metadata: | ||
| labels: | ||
| app: falcon | ||
| spec: | ||
| initContainers: | ||
| - name: adapter-init-container | ||
| image: REPO_HERE.azurecr.io/adapter-falcon-7b-amod-mental-health-busybox:0.0.2 | ||
| command: | ||
| - /bin/sh | ||
| - -c | ||
| - | | ||
| mkdir -p /mnt/adapter && cp -r /data/* /mnt/adapter/ | ||
| volumeMounts: | ||
| - name: adapter-volume | ||
| mountPath: /mnt/adapter | ||
| imagePullPolicy: Always | ||
| containers: | ||
| - name: falcon-container | ||
| image: REPO_HERE.azurecr.io/falcon-7b:TAG_HERE | ||
| command: | ||
| - /bin/sh | ||
| - -c | ||
| - accelerate launch --num_processes 1 --num_machines 1 --machine_rank 0 --gpu_ids all inference_api.py --pipeline text-generation --torch_dtype bfloat16 | ||
| resources: | ||
| requests: | ||
| nvidia.com/gpu: 2 | ||
| limits: | ||
| nvidia.com/gpu: 2 # Requesting 2 GPUs | ||
| livenessProbe: | ||
| httpGet: | ||
| path: /healthz | ||
| port: 5000 | ||
| initialDelaySeconds: 600 # 10 Min | ||
| periodSeconds: 10 | ||
| readinessProbe: | ||
| httpGet: | ||
| path: /healthz | ||
| port: 5000 | ||
| initialDelaySeconds: 30 | ||
| periodSeconds: 10 | ||
| volumeMounts: | ||
| - name: dshm | ||
| mountPath: /dev/shm | ||
| - name: adapter-volume | ||
| mountPath: /mnt/adapter | ||
| volumes: | ||
| - name: dshm | ||
| emptyDir: | ||
| medium: Memory | ||
| - name: adapter-volume | ||
| emptyDir: {} # emptyDir volume shared between adapter-init container and main container | ||
| tolerations: | ||
| - effect: NoSchedule | ||
| key: sku | ||
| operator: Equal | ||
| value: gpu | ||
| - effect: NoSchedule | ||
| key: nvidia.com/gpu | ||
| operator: Exists | ||
| nodeSelector: | ||
| pool: falcon7b |