feat(community): Add new LLM provider Novita AI #7231
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,207 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "raw", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "---\n", | ||
| "sidebar_label: Novita\n", | ||
| "---" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# ChatNovita\n", | ||
| "\n", | ||
| "[Novita AI](https://novita.ai/model-api/product/llm-api?utm_source=github_langchain&utm_medium=github_readme&utm_campaign=link): The most cost-effective, stable, and scalable inference platform — delivering production-ready performance at a lower cost.\n", | ||
| "\n", | ||
| "This guide will help you getting started with `ChatNovitaAI` [chat models](/docs/concepts/chat_models). For detailed documentation of all `ChatNovitaAI` features and configurations head to the [API reference](https://novita.ai/docs/model-api/reference/llm/llm.html?utm_source=github_langchain&utm_medium=github_readme&utm_campaign=link)." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Overview\n", | ||
| "\n", | ||
| "### Model features\n", | ||
| "| [Tool calling](../../how_to/tool_calling.ipynb) | [Structured output](../../how_to/structured_output.ipynb) | JSON mode | [Image input](../../how_to/multimodal_inputs.ipynb) | Audio input | Video input | [Token-level streaming](../../how_to/chat_streaming.ipynb) | Native async | [Token usage](../../how_to/chat_token_usage_tracking.ipynb) | [Logprobs](../../how_to/logprobs.ipynb) |\n", | ||
| "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", | ||
| "| ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ |" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Setup\n", | ||
| "\n", | ||
| "To access Novita AI models you'll need to create a Novita account and get an API key.\n", | ||
| "\n", | ||
| "### Credentials\n", | ||
| "\n", | ||
| "Head to [this page](https://novita.ai/settings#key-management?utm_source=github_langchain&utm_medium=github_readme&utm_campaign=link) to sign up to Novita AI and generate an API key. Once you've done this set the NOVITA_API_KEY environment variable:" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "vscode": { | ||
| "languageId": "shellscript" | ||
| } | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "export NOVITA_API_KEY=\"your-api-key\"" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### Installation\n", | ||
| "\n", | ||
| "The LangChain Novita integration lives in the `@langchain-community` package:\n", | ||
| "\n", | ||
| "```{=mdx}\n", | ||
| "import IntegrationInstallTooltip from \"@mdx_components/integration_install_tooltip.mdx\";\n", | ||
| "import Npm2Yarn from \"@theme/Npm2Yarn\";\n", | ||
| "\n", | ||
| "<IntegrationInstallTooltip></IntegrationInstallTooltip>\n", | ||
| "\n", | ||
| "<Npm2Yarn>\n", | ||
| " @langchain/community @langchain/core\n", | ||
| "</Npm2Yarn>\n", | ||
| "```" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Instantiation\n", | ||
| "\n", | ||
| "Now we can instantiate our model object and generate chat completions:" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "vscode": { | ||
| "languageId": "javascript" | ||
| } | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import { ChatNovitaAI } from \"../novita.js\";\n", | ||
| "\n", | ||
| "const llm = new ChatNovitaAI({\n", | ||
| " model: \"gryphe/mythomax-l2-13b\",\n", | ||
| " temperature: 0,\n", | ||
| " // other params...\n", | ||
| "})" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Invocation" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "vscode": { | ||
| "languageId": "javascript" | ||
| } | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "const aiMsg = await llm.invoke([\n", | ||
| " [\n", | ||
| " \"system\",\n", | ||
| " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", | ||
| " ],\n", | ||
| " [\"human\", \"I love programming.\"],\n", | ||
| "])\n", | ||
| "aiMsg" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "vscode": { | ||
| "languageId": "javascript" | ||
| } | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "console.log(aiMsg.content)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Chaining\n", | ||
| "\n", | ||
| "We can [chain](../../how_to/sequence.ipynb) our model with a prompt template like so:" | ||
|
|
||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "vscode": { | ||
| "languageId": "javascript" | ||
| } | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import { ChatPromptTemplate } from \"@langchain/core/prompts\"\n", | ||
| "\n", | ||
| "const prompt = ChatPromptTemplate.fromMessages(\n", | ||
| " [\n", | ||
| " [\n", | ||
| " \"system\",\n", | ||
| " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", | ||
| " ],\n", | ||
| " [\"human\", \"{input}\"],\n", | ||
| " ]\n", | ||
| ")\n", | ||
| "\n", | ||
| "const chain = prompt.pipe(llm);\n", | ||
| "await chain.invoke(\n", | ||
| " {\n", | ||
| " input_language: \"English\",\n", | ||
| " output_language: \"German\",\n", | ||
| " input: \"I love programming.\",\n", | ||
| " }\n", | ||
| ")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## API reference\n", | ||
| "\n", | ||
| "For detailed documentation of Novita AI LLM APIs, head to [Novita AI API reference](https://novita.ai/docs/model-api/reference/llm/llm.html?utm_source=github_langchain&utm_medium=github_readme&utm_campaign=link)\n" | ||
|
There was a problem hiding this comment. We prefer having this link to the LangChain API refs, but this is ok too There was a problem hiding this comment. I'm truly sorry but I don't have more time to write a Novita API ref for LangChain 😭. There was a problem hiding this comment. Oh, no this will be autogenerated don't worry. |
||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "language_info": { | ||
| "name": "python" | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 2 | ||
| } | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| --- | ||
| sidebar_label: Novita | ||
| --- | ||
|
|
||
| import CodeBlock from "@theme/CodeBlock"; | ||
|
|
||
|
There was a problem hiding this comment. You shouldn't have this and the notebook above - the notebook compiles into an |
||
| # ChatNovita | ||
|
|
||
| [Novita AI](https://novita.ai/model-api/product/llm-api?utm_source=github_langchain&utm_medium=github_readme&utm_campaign=link) is the most cost-effective, stable, and scalable inference platform — delivering production-ready performance at a lower cost. | ||
|
|
||
| This guide will help you get started with `ChatNovitaAI` [chat models](/docs/concepts/chat_models). For detailed documentation of all `ChatNovitaAI` features and configurations head to the [API reference](https://novita.ai/docs/model-api/reference/llm/llm.html?utm_source=github_langchain&utm_medium=github_readme&utm_campaign=link). | ||
|
|
||
| ## Setup | ||
|
|
||
| To access Novita AI models you'll need to create a Novita account and get an API key. | ||
|
|
||
| ### Credentials | ||
|
|
||
| Head to [this page](https://novita.ai/settings#key-management?utm_source=github_langchain&utm_medium=github_readme&utm_campaign=link) to sign up to Novita AI and generate an API key. Once you've done this set the NOVITA_API_KEY environment variable: | ||
|
|
||
| ```bash | ||
| export NOVITA_API_KEY="your-api-key" | ||
| ``` | ||
|
|
||
| ### Installation | ||
|
|
||
| The LangChain Novita integration lives in the `@langchain-community` package: | ||
|
|
||
| import IntegrationInstallTooltip from "@mdx_components/integration_install_tooltip.mdx"; | ||
|
|
||
| <IntegrationInstallTooltip></IntegrationInstallTooltip> | ||
|
|
||
| ```bash npm2yarn | ||
| npm install @langchain/community @langchain/core | ||
| ``` | ||
|
|
||
| ## Basic usage | ||
|
|
||
| import Novita from "@examples/models/chat/integration_novita.ts"; | ||
|
|
||
| <CodeBlock language="typescript">{Novita}</CodeBlock> | ||
|
|
||
| ## Chain model calls | ||
|
|
||
| import NovitaChain from "@examples/models/chat/integration_novita_chain.ts"; | ||
|
|
||
| <CodeBlock language="typescript">{NovitaChain}</CodeBlock> | ||
|
|
||
| ## API reference | ||
|
|
||
| For detailed documentation of Novita AI LLM APIs, head to [Novita AI API reference](https://novita.ai/docs/model-api/reference/llm/llm.html?utm_source=github_langchain&utm_medium=github_readme&utm_campaign=link) | ||
|
There was a problem hiding this comment. Remove, see above. The notebook is all you need |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| import { ChatNovita } from "@langchain/community/chat_models/novita"; | ||
| import { HumanMessage } from "@langchain/core/messages"; | ||
|
|
||
| // Use gryphe/mythomax-l2-13b | ||
| const chat = new ChatNovita({ | ||
| apiKey: "YOUR-API-KEY", // In Node.js defaults to process.env.NOVITA_API_KEY | ||
| model: "gryphe/mythomax-l2-13b", // Check available models at https://novita.ai/llm-api | ||
| temperature: 0.3, | ||
| }); | ||
|
|
||
| const messages = [new HumanMessage("Hello")]; | ||
|
|
||
| const res = await chat.invoke(messages); | ||
| /* | ||
| AIMessage { | ||
| content: "Hello! How can I help you today? Is there something you would like to talk about or ask about? I'm here to assist you with any questions you may have.", | ||
| } | ||
| */ | ||
|
|
||
| const res2 = await chat.invoke(messages); | ||
| /* | ||
| AIMessage { | ||
| text: "Hello! How can I help you today? Is there something you would like to talk about or ask about? I'm here to assist you with any questions you may have.", | ||
| } | ||
| */ |
|
There was a problem hiding this comment. Remove, see above. The notebook is all you need |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| import { ChatNovita } from "@langchain/community/chat_models/novita"; | ||
| import { LLMChain } from "langchain/chains"; | ||
| import { PromptTemplate } from "@langchain/core/prompts"; | ||
|
|
||
|
|
||
| const chat = new ChatNovita({ | ||
| apiKey: "YOUR-API-KEY", // In Node.js defaults to process.env.NOVITA_API_KEY | ||
| model: "gryphe/mythomax-l2-13b", // Check available models at https://novita.ai/llm-api | ||
| temperature: 0.3, | ||
| }); | ||
|
|
||
| const prompt = PromptTemplate.fromTemplate( | ||
| "What is a good name for a company that makes {product}?" | ||
| ); | ||
| const chain = new LLMChain({ llm: chat, prompt }); | ||
|
|
||
| const response = await chain.invoke({ product: "colorful socks" }); | ||
|
|
||
| console.log({ response }); | ||
|
|
||
| /* | ||
| { | ||
| text: `I'm not sure what you mean by "colorful socks" but here are some ideas:\n` + | ||
| '\n' + | ||
| '- Sock-it to me!\n' + | ||
| '- Socks Away\n' + | ||
| '- Fancy Footwear' | ||
| } | ||
| */ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe these need to be absolute paths rather than relative
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed