Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve clarity of model run command's error message #4

Merged
merged 3 commits into from
Oct 9, 2024

Conversation

garman
Copy link
Contributor

@garman garman commented Oct 8, 2024

closes: github/models#322

Adds more detail to the error output when a given model name is either empty or otherwise not found.


Before (taken from issue):

$ gh models run ThisDoesNotExist Mary had a little lamb
Error: bad request
{"error":{"code":"unknown_model","message":"Unknown model: thisdoesnotexist","details":null}}

Usage:
  gh run [model] [prompt] [flags]

Flags:
  -h, --help                   help for run
      --max-tokens string      Limit the maximum tokens for the model response.
      --system-prompt string   Prompt the system.
      --temperature string     Controls randomness in the response, use lower to be more deterministic.
      --top-p string           Controls text diversity by selecting the most probable words until a set probability is reached.


After:

danielgarman:Daniel’s MacBook Pro
╰─\gh-models:models/322-improve-error-messaging-root*:% ./gh-models run "" "mary had a little lamb"
Error: The specified model name is not found. Run 'gh models list' to see available models or 'gh models run' to select interactively.
Usage:
  gh run [model] [prompt] [flags]

Flags:
  -h, --help                   help for run
      --max-tokens string      Limit the maximum tokens for the model response.
      --system-prompt string   Prompt the system.
      --temperature string     Controls randomness in the response, use lower to be more deterministic.
      --top-p string           Controls text diversity by selecting the most probable words until a set probability is reached.

danielgarman:Daniel’s MacBook Pro
╰─\gh-models:models/322-improve-error-messaging-root*:% ./gh-models run "asdfasdf" "mary had a little lamb"
Error: The specified model name is not found. Run 'gh models list' to see available models or 'gh models run' to select interactively.
Usage:
  gh run [model] [prompt] [flags]

Flags:
  -h, --help                   help for run
      --max-tokens string      Limit the maximum tokens for the model response.
      --system-prompt string   Prompt the system.
      --temperature string     Controls randomness in the response, use lower to be more deterministic.
      --top-p string           Controls text diversity by selecting the most probable words until a set probability is reached.

or

danielgarman:Daniel’s MacBook Pro
╰─\gh-models:models/322-improve-error-messaging-root:% ./gh-models run foo "hello world"
Error: The specified model name is not found. Run 'gh models list' to see available models or 'gh models run' to select interactively.
Usage:
  gh run [model] [prompt] [flags]

Flags:
  -h, --help                   help for run
      --max-tokens string      Limit the maximum tokens for the model response.
      --system-prompt string   Prompt the system.
      --temperature string     Controls randomness in the response, use lower to be more deterministic.
      --top-p string           Controls text diversity by selecting the most probable words until a set probability is reached.

@garman garman force-pushed the models/322-improve-error-messaging-root branch from 8162419 to 0fe7680 Compare October 8, 2024 14:34
This applies both for when an empty string or an incorrect model name
is specified. The prior had an error message already, but it was fairly
vague. The latter was using the default error output from later in
execution. This commit makes them consistent and more detailed,
providing two suggestions.
@garman garman force-pushed the models/322-improve-error-messaging-root branch from 0fe7680 to fa846d3 Compare October 8, 2024 14:35
@garman garman marked this pull request as ready for review October 8, 2024 16:18
Copy link
Member

@cheshire137 cheshire137 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable to me! Can you verify that the new error message comes up if you try gh models run foo "hello world"?

@@ -226,15 +226,23 @@ func NewRunCommand() *cobra.Command {
modelName = args[0]
}

noMatchErrorMessage := "The specified model name is not found. Run 'gh models list' to see available models or 'gh models run' to select interactively."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could include the value that was given, like "The specified model name 'foo'".

@garman
Copy link
Contributor Author

garman commented Oct 8, 2024

Can you verify that the new error message comes up if you try gh models run foo "hello world"?

Yep!

danielgarman:Daniel’s MacBook Pro
╰─\gh-models:models/322-improve-error-messaging-root:% ./gh-models run foo "hello world"
Error: The specified model name is not found. Run 'gh models list' to see available models or 'gh models run' to select interactively.
Usage:
  gh run [model] [prompt] [flags]

Flags:
  -h, --help                   help for run
      --max-tokens string      Limit the maximum tokens for the model response.
      --system-prompt string   Prompt the system.
      --temperature string     Controls randomness in the response, use lower to be more deterministic.
      --top-p string           Controls text diversity by selecting the most probable words until a set probability is reached.

@garman garman merged commit ab0f0c4 into main Oct 9, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants