Skip to content

Commit

Permalink
[lora] Update LoRA adapter doc (#2604)
Browse files Browse the repository at this point in the history
  • Loading branch information
xyang16 authored Nov 22, 2024
1 parent 83b6fb7 commit e27d95f
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions serving/docs/adapters.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ You can also specify the base model for an element in the batch by using the emp
curl -X POST http://127.0.0.1:8080/invocations \
-H "Content-Type: application/json" \
-H "X-Amzn-SageMaker-Target-Model: base-1.tar.gz" \
-d '{"inputs": ["How is the weather"], "adapters": ["adapter_1"], "parameters": {"max_new_tokens": 25}}'
-d '{"inputs": "How is the weather", "adapters": "adapter_1", "parameters": {"max_new_tokens": 25}}'
```

### Input Content (and query parameter)
Expand All @@ -150,7 +150,7 @@ This will not work in Amazon SageMaker.
curl -X POST http://127.0.0.1:8080/invocations?adapter=adapter_1 \
-H "Content-Type: application/json" \
-H "X-Amzn-SageMaker-Target-Model: base-1.tar.gz" \
-d '{"inputs": ["How is the weather"], "parameters": {"max_new_tokens": 25}}'
-d '{"inputs": "How is the weather", "parameters": {"max_new_tokens": 25}}'
```

### SageMaker Header
Expand All @@ -163,8 +163,8 @@ This option will work in Amazon SageMaker.
curl -X POST http://127.0.0.1:8080/invocations \
-H "Content-Type: application/json" \
-H "X-Amzn-SageMaker-Target-Model: base-1.tar.gz" \
-H "X-Amzn-SageMaker-Adapter-Identifier: adapter_1"
-d '{"inputs": ["How is the weather"], "parameters": {"max_new_tokens": 25}}'
-H "X-Amzn-SageMaker-Adapter-Identifier: adapter_1" \
-d '{"inputs": "How is the weather", "parameters": {"max_new_tokens": 25}}'
```

## Models with Adapters
Expand Down

0 comments on commit e27d95f

Please sign in to comment.