Skip to content

Commit

Permalink
⚡️ Preview (#1228)
Browse files Browse the repository at this point in the history
* 🐛 fix off-by-one errors

* 🚧 swap out button on enter

* 💄 styling and auto-resize

* 💄 box shadow

* 🚧 fix keyboard shortcuts to accept/reject diff

* 💄 improve small interactions

* 💄 loading icon, cancellation logic

* 🐛 handle next.value being undefined

* ✨ latex support

* Bug Fix: Add ternary operator to prevent nonexistant value error (#1052)

* add terniary operator

* Removing logging

* remove comment

---------

Co-authored-by: Justin Milner <jmilner@jmilner-lt2.deka.local>
Co-authored-by: Nate Sesti <33237525+sestinj@users.noreply.github.com>

* 🎨 small formatting change

* 🩹 tweak /edit solution

* ✨ Dropdown to select model

* 🔊 print when SSL verification disabled

* 📌 pin esbuild version to match our hosted binary

* 🔥 remove unused package folder

* 👷 add note about pinning esbuild

* 🚚 rename pkg to binary

* ⚡️ update an important stop word for starcoder2, improve dev data

* 🐛 fix autocomplete bug

* Update completionProvider.ts

Add \r\n\r\n stop to tab completion

* 📌 update package-locks

* 🐛 fix bug in edit prompt

* 🔊 log extension version

* 🐛 handle repo undefined in vscode

* ⏪ revert back to esbuild ^0.17.19 to solve no backend found error with onnxruntime

* 🩹 set default autocomplete temp to 0.01 to be strictly positive

* make the useCopyBuffer option effective (#1062)

* Con-1037: Toggle full screen bug (#1065)

* webview reset

* add warning

---------

Co-authored-by: Justin Milner <jmilner@jmilner-lt2.deka.local>

* Update completionProvider.ts

as @rootedbox suggested

* Resolve conflict, accept branch being merged in (#1076)

* Resolve conflict, accept branch being merged in

* remove accidental .gitignore add

* whoops, put gitignore back

* fix

---------

Co-authored-by: Justin Milner <jmilner@jmilner-lt2.deka.local>

* #1073: update outdated documentation (#1074)

* 🩹 small tweaks to stop words

* Add abstraction for fetch to easily allow using request options (#1059)

* add fetch helper function with request options

* add support for request options for Jira context provider

* Add a new slash command to review code. (#1071)

* Add a new slash command to review code.

* clean code

* 🩹 add new starcoder artifact as stopword

* 💄 slight improvements to inline edit UI

* 🔖 update default models, bump gradle version

* 📝 recommend starcoder2

* 🐛 fix jetbrains encoding issue

* 🩹 don't index site-packages

* 🩹 error handling in JetBrains

* 🐛 fix copy to clipboard in jetbrains

* fix: cursor focus issue causing unwanted return to text area (#1086)

* 📝 mention autocomplete in jetbrains

* 📝 Tab-autocomplete README

* 🔥 remove note about custom ctx providers only being on VS Code

* 📝 docs about http context provider

* 👥 pull request template

* Update from Claude 2 to Claude 3 (#1078)

* 📝 add FAQ about single-line completions

* 📝 update autocomplete docs

* fix cursor focus issue causing unwanted return to text area

---------

Co-authored-by: Nate Sesti <sestinj@gmail.com>
Co-authored-by: Ty Dunn <ty@tydunn.com>
Co-authored-by: Nate Sesti <33237525+sestinj@users.noreply.github.com>

* Update tree-sitter-wasms to 0.1.11 (which includes Solidity)

* Make use of solidity tree-sitter parser

* 🔧 option to disable autocomplete from config.json

* ✨ option to disable streaming with anthropic

* ✅ Test to verify that files are packaged

* Add FIM template for CodeGemma (#1097)

Also pass stop tokens to llama.cpp.

* ✨ customizable rerankers (#1088)

* ✨ customizable rerankers

* 💄 fix early truncation button

* ⚡️ improvements to full text search + reranking

* ⚡️ only use starcoder2 stop words for starcoder2

* ⚡️ crawl code graph for call expressions

* 🚧 starcoder2-7b free trial

* 🚧 free trial client for embeddings and re-ranking

* 🚧 embeddings provider

* ✅ test for presence of files in CI

* 🐛 fixes to reranking

* ✨ new onboarding experience

* ✨ new onboarding experience

* 💄 small tweaks to onboarding

* 🩹 add stopAtLines filter to /edit

* 🐛 clean up vite build errors

* 👷 make vscode external in binary build

* 💄 improved models onboarding for existing users

* 💄 default indexing progress to 0.0

* 🐛 small fixes to reranking

* 👷 clear folders before prepackage

* 👷 say where .vsix is output

* 👷 also download arm packages outside of gh actions

* 🎨 add AbortSignal to indexing

* 🔧 starcoder, not 2 in config_schema

* 🚚 again, starcoder, not 2

* 🐛 fix bug when reranker undefined

* 🩹 fix binary tsc error

* ✨ configure context menu prompts

* 🐛 acknowledge useLegacyCompletionsEndpoint

* 🚑 fix keep existing config option

* 🔊 learn about selection

* ⚡️ improvements to indexing reporting when not in git repo

* 🥅 handle situation where git doesn't exist in workspace

* ✨ support for gemini 1.5 pro

* 🐛 handle embeddingProvider name not found

* ✨ Gemini 1.5 and GPT-4 Turbo

* 👷 fix os, arch undefined in prepackage.js

* ⚡️ better detection of terminal code blocks

* 🧑‍💻 solve tailwind css warnings

* ✨ cmd/ctrl+L to select terminal contents

* 🐛 correctly handle remotes not found

* ✨ allow templating for custom commands

* 🔥 temporarily remove cmd+L to select terminal contents

* 🐛 remove quotes around Ollama stop words

* ✨ add Cohere as Model Provider (#1119)

* 🩹 add gpt-4-turbo to list of chat_only models

* feat: use exponential backoff in llm chat (#1115)

Signed-off-by: inimaz <93inigo93@gmail.com>

* 🩹 update exponential backoff timing

* 💄 spell out Alt in keyboard shortcuts

* 🩹 don't set edit prompt for templateType "none"

* Adds additional ignores for C-fmilies langs (#1129)

Ignored:
- cache directory `.cache`, used by clangd
- dependency files `*o.d`, used by object files
- LLVM and GNU coverage files: `*.profraw`, `*.gcda` and `*.gcno`

* 🔥 temporarily remove problematic expandSnippet import

* 👷 add npx to prefix vsce in build

* 🐛 handle messages sent in multiple parts over stdin

* 🔖 update gradle version

* 🩹 for now, skip onboarding in jetbrains

* 🩹 temporarily don't show use codebase on jetbrains

* 🐛 use system certificates in binary

* 🔖 update jetbrains version

* 🩹 correctly contruct set of certs

* 🔖 bump intellij version to 0.0.45

* 🩹 update to support images for gpt-4-turbo

* 🐛 fix image support autodetection

* ⚡️ again, improve image support autodetection

* 🐛 set supportsCompletions based on useLegacyCompletionsEndpoint model setting

Closes #1132

* 📝 useLegacyCompletionsEndpoint within OpenAI docs

* 🔧 forceCompletionsEndpointType option

* Revert "🔧 forceCompletionsEndpointType option"

This reverts commit dd51fcb.

* 🩹 set default useLegacyCompletionsEndpoint to undefined

* 🩹 look for bedrock credentials in homedir

* 🩹 use title for autodetect

* 🐛 set supportsCompletions based on useLegacyCompletionsEndpoint model setting

Closes #1132

* 📝 useLegacyCompletionsEndpoint within OpenAI docs

* 🩹 set default useLegacyCompletionsEndpoint to undefined

* 🩹 look for bedrock credentials in homedir

* 🩹 use title for autodetect

* Fix slash command params loading

Existing slash commands expect an object named
"params" so mapping to "options" here caused
params to be undefined within the run scope. I
renamed from 'm' to 's' just to avoid potential
confusion with the model property mapping above.

* Add outputDir param to /share

* Enable basic tilde expansion for /share outputDir

* Add ability to specify workspace for /share

* Add datetimestamp to exported session filename

* Use `.`, `./`, or `.\` for current workspace

* Add description of outputDir param for /share

* Ensure replacement only at start of string

* Create user-specified directory if necessary

* Change "Continue" to "Assistant" in export

* Consolidate to single replace regex

* Reformat markdown code blocks

Currently, user-selected code blocks are formatted
with range in file (rif) info on the same line as
the triple backticks, which means that when
exported to markdown they don't have the language
info needed on that line for syntax highlighting.
This update moves the rif info to the following
line as a comment in the language of the file and
with the language info in the correct place.

Before:
```example.ts (3-6)

function fib(n) {
    if (n <= 1) return n;
    return fib(n - 2) + fib(n - 1);
}
```

After:
```ts
// example.ts (3-6)

function fib(n) {
    if (n <= 1) return n;
    return fib(n - 2) + fib(n - 1);
}
```

* Tidy regex to capture filename

* Tidy regex to capture filename

* Ensure adjacent codeblocks separated by newline

* Aesthetic tweaks to output format

* ✨ disableInFiles option for autocomplete

* feat(httpContextProvider): load AC on fetch client (#1150)

Co-authored-by: Bertrand Pinel <bertrand.pinel@pole-emploi.fr>

* ✨ global filewatcher for config.json/ts changes

* 🐛 retry webview requests so that first cmd+L works

* ✨ Improved onboarding experience (#1155)

* 🚸 onboarding improvements

* 🧑‍💻 keyboard shortcuts to toggle autocomplete and open config.json

* ⚡️ improve detection of terminal code blocks

* 🚧 onboarding improvements

* 🚧 more onboarding improvements

* 💄 last session button

* 🚸 show more fallback options in dropdown

* 💄 add sectioning to models page

* 💄 clean up delete model button

* 💄 make tooltip look nicer

* 🚸 download Ollama button

* 💄 local LLM onboarding

* 🐛 select correct terminal on "runCommand" message

* 💄 polish onboarding

* 💚 fix gui build errors

* 📝 add /v1 to OpenAI examples in docs

* 🚑 hotfix for not iterable error

* ✨ add Cohere as Embeddings Provider

* 💄 add llama3 to UI

* 🔥 remove disable indexing

* 🍱 update continue logo

* 🐛 fix language undefined bug

* 🐛 fix merge mistake

* 📝 update mistral models

* ✨ global request options (#1153)

* ✨ global request options

* 🐛 fix jira context provider by injecting fetch

* ✨ request options for embeddings providers

* ✨ add Cohere as Reranker (#1159)

* ♻️ use custom requestOptions with CohereEmbeddingsProvider

* Update preIndexedDocs.ts (#1154)

Add WordPress and WooCommerce as preIndexedDocs.

* 🩹 remove example "outputDir" from default config

* Fix slash command params loading (#1084)

Existing slash commands expect an object named
"params" so mapping to "options" here caused
params to be undefined within the run scope. I
renamed from 'm' to 's' just to avoid potential
confusion with the model property mapping above.

* 🐛 don't index if no open workspace folders

* 💄 improve onboarding language

* 🚸 improve onboarding

* 🐛 stop loading when error

* 💄 replace text in input box

* Respect Retry-After header when available from 429 responses (#1182)

* 🩹 remove dead code for exponential backoff

This has been replaced by the withExponentialBackoff helper

* 🩹 respect Retry-After header when available

* 🚸 update inline tips language

* ✨ input box history

* 📌 update package-locks

* 🔊 log errors in prepackage

* 🐛 err to string

* 📌 pin llama-tokenizer-js

* 📌 update lockfile

* 🚚 change /docs to docs.

* 📦 package win-ca dependencies in binary

* 🔥 remove unpopular models from UI

* 🍱 new logo in jetbrains

* 🎨 use node-fetch everywhere

* 🚸 immediately select newly added models

* 🚸 spell out Alt instead of using symbol

* 🔥 remove config shortcut

* 🐛 fix changing model bug

* 🩹 de-duplicate before adding models

* 🔧 add embeddingsProvider specific request options

* 🎨 refactor to always use node-fetch from LLM

* 🔥 remove duplicate tokens generated

* 🔊 add timestamp to JetBrains logs

* 🎨 maxStopWords for Groq

* 🐛 fix groq provider calling /completions

* 🐛 correctly adhere to LanceDB table name spec

* 🐛 fix sqlite NOT NULL constraint failed error with custom model

* Fix issue where Accept/Reject All only accepts/rejects a single diff hunk. (#1197)

* Fix issues parsing Ollama /api/show endpoint payloads. (#1199)

* ✨ model role for inlineEdit

* 🩹 various small updates

* 🐛 fix openai image support

* 🔖 update gradle version

* 🍱 update jetbrains icon

* 🐛 fix autocomplete in notebook cells

* 🔥 remove unused media

* 🔥 remove unused files

* Fix schema to allow 'AUTODETECT' sentinel for model when provider is 'groq'. (#1203)

* 🐛 small improvements

* Fix issue with @codebase provider when n becomes odd due to a divide by 2 during the full text search portion of the query. (#1204)

* 🐛 add skipLines

* ✨ URLContextProvider

* 🥅 improved error handling for codebase indexing

* 🏷️ use official Git extension types

* ➕ declare vscode.git extension dependency

* ⚡️ use reranker for docs context provider

* 🚸 Use templating in default customCommand

* 🎨 use U+23CE

* 🚸 disable autocomplete in commit message box

* 🩹 add gems to default ignored paths

* ⚡️ format markdown blocks as comments in .ipynb completions

* 🐛 don't strip port in URL

* 🐛 fix "gemini" provider spacing issues

* 📦 update posthog version

* 🏷️ update types.ts

* 🐛 fix copy/paste/cut behavior in VS Code notebooks

* ✨ llama3 prompt template

* 🐛 fix undefined prefix, suffix and language for `/edit` (#1216)

* 🐛 add .bind to fix templating in systemMessage

* 🐛 small improvements to autocomplete

* Update DocsContextProvider.ts (#1217)

I fixed a bug where you were sending the query variable (which holds the base URL of the doc) to the rerank method, and it made no sense to rerank the chunks based on a URL. So I changed it to extras.fullInput because it should rerank based on the user input, which should provide better results.

* 📝 select-provider.md update

* 🐛 fix merge errors

---------

Signed-off-by: inimaz <93inigo93@gmail.com>
Co-authored-by: Justin Milner <42585006+justinmilner1@users.noreply.github.com>
Co-authored-by: Justin Milner <jmilner@jmilner-lt2.deka.local>
Co-authored-by: lmaosweqf1 <138042737+lmaosweqf1@users.noreply.github.com>
Co-authored-by: ading2210 <71154407+ading2210@users.noreply.github.com>
Co-authored-by: Martin Mois <martin.mois@googlemail.com>
Co-authored-by: Tobias Jung <102594442+tobiajung@users.noreply.github.com>
Co-authored-by: Jason Jacobs <nerfnerd@gmail.com>
Co-authored-by: Nithish <83941930+Nithishvb@users.noreply.github.com>
Co-authored-by: Ty Dunn <ty@tydunn.com>
Co-authored-by: Riccardo Schirone <riccardo.schirone@trailofbits.com>
Co-authored-by: postmasters <namnguyen@google.com>
Co-authored-by: Maxime Brunet <max@brnt.mx>
Co-authored-by: inimaz <49730431+inimaz@users.noreply.github.com>
Co-authored-by: SR_team <me@sr.team>
Co-authored-by: Roger Meier <r.meier@siemens.com>
Co-authored-by: Peter Zaback <pzaback@gmail.com>
Co-authored-by: Bertrand P <49525332+Berber31@users.noreply.github.com>
Co-authored-by: Bertrand Pinel <bertrand.pinel@pole-emploi.fr>
Co-authored-by: Jose Vega <bloguea.y.gana@gmail.com>
Co-authored-by: Nejc Habjan <hab.nejc@gmail.com>
Co-authored-by: Chad Yates <cyates@dynfxdigital.com>
Co-authored-by: 小颚虫 <123357481+5eqn@users.noreply.github.com>
  • Loading branch information
1 parent 4db5b6d commit cf4b557
Show file tree
Hide file tree
Showing 154 changed files with 4,240 additions and 16,188 deletions.
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ body:
required: false
- label: I'm not able to find an [open issue](https://github.com/continuedev/continue/issues?q=is%3Aopen+is%3Aissue) that reports the same bug
required: false
- label: I've seen the [troubleshooting guide](https://continue.dev/docs/troubleshooting) on the Continue Docs
- label: I've seen the [troubleshooting guide](https://docs.continue.dev/troubleshooting) on the Continue Docs
required: false
- type: textarea
attributes:
Expand Down Expand Up @@ -58,5 +58,5 @@ body:
attributes:
label: Log output
description: |
Please refer to the [troubleshooting guide](https://continue.dev/docs/troubleshooting) in the Continue Docs for instructions on obtaining the logs. Copy either the relevant lines or the last 100 lines or so.
Please refer to the [troubleshooting guide](https://docs.continue.dev/troubleshooting) in the Continue Docs for instructions on obtaining the logs. Copy either the relevant lines or the last 100 lines or so.
render: Shell
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -152,4 +152,6 @@ continue_server.build
continue_server.dist

Icon
Icon?
Icon?

.continue
3 changes: 2 additions & 1 deletion .prettierrc
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
{
"tabWidth": 2,
"useTabs": false
"useTabs": false,
"trailingComma": "all"
}
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ Continue is quickly adding features, and we'd love to hear which are the most im

## 📖 Updating / Improving Documentation

Continue is continuously improving, but a feature isn't complete until it is reflected in the documentation! If you see something out-of-date or missing, you can help by clicking "Edit this page" at the bottom of any page on [continue.dev/docs](https://continue.dev/docs).
Continue is continuously improving, but a feature isn't complete until it is reflected in the documentation! If you see something out-of-date or missing, you can help by clicking "Edit this page" at the bottom of any page on [docs.continue.dev](https://docs.continue.dev).

## 🧑‍💻 Contributing Code

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

<div align="center">

**[Continue](https://continue.dev/docs) keeps developers in flow. Our open-source [VS Code](https://marketplace.visualstudio.com/items?itemName=Continue.continue) and [JetBrains](https://plugins.jetbrains.com/plugin/22707-continue-extension) extensions enable you to easily create your own modular AI software development system that you can improve.**
**[Continue](https://docs.continue.dev) keeps developers in flow. Our open-source [VS Code](https://marketplace.visualstudio.com/items?itemName=Continue.continue) and [JetBrains](https://plugins.jetbrains.com/plugin/22707-continue-extension) extensions enable you to easily create your own modular AI software development system that you can improve.**

</div>

Expand All @@ -17,7 +17,7 @@
<a target="_blank" href="https://opensource.org/licenses/Apache-2.0" style="background:none">
<img src="https://img.shields.io/badge/License-Apache_2.0-blue.svg" style="height: 22px;" />
</a>
<a target="_blank" href="https://continue.dev/docs" style="background:none">
<a target="_blank" href="https://docs.continue.dev" style="background:none">
<img src="https://img.shields.io/badge/continue_docs-%23BE1B55" style="height: 22px;" />
</a>
<a target="_blank" href="https://discord.gg/vapESyrFmJ" style="background:none">
Expand Down
5 changes: 4 additions & 1 deletion binary/pkgJson/win32-arm64/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,10 @@
"../../../core/node_modules/sqlite3/**/*",
"../../node_modules/@lancedb/vectordb-win32-x64-msvc/index.node",
"../../out/tree-sitter.wasm",
"../../out/tree-sitter-wasms/*"
"../../out/tree-sitter-wasms/*",
"../../node_modules/win-ca/lib/crypt32-ia32.node",
"../../node_modules/win-ca/lib/crypt32-x64.node",
"../../node_modules/win-ca/lib/roots.exe"
],
"targets": [
"node18-win-arm64"
Expand Down
5 changes: 4 additions & 1 deletion binary/pkgJson/win32-x64/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,10 @@
"../../../core/node_modules/sqlite3/**/*",
"../../node_modules/@lancedb/vectordb-win32-x64-msvc/index.node",
"../../out/tree-sitter.wasm",
"../../out/tree-sitter-wasms/*"
"../../out/tree-sitter-wasms/*",
"../../node_modules/win-ca/lib/crypt32-ia32.node",
"../../node_modules/win-ca/lib/crypt32-x64.node",
"../../node_modules/win-ca/lib/roots.exe"
],
"targets": [
"node18-win-x64"
Expand Down
11 changes: 10 additions & 1 deletion binary/src/core.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { indexDocs } from "core/indexing/docs";
import TransformersJsEmbeddingsProvider from "core/indexing/embeddings/TransformersJsEmbeddingsProvider";
import { CodebaseIndexer, PauseToken } from "core/indexing/indexCodebase";
import { logDevData } from "core/util/devdata";
import { fetchwithRequestOptions } from "core/util/fetchWithOptions";
import historyManager from "core/util/history";
import { Message } from "core/util/messenger";
import { Telemetry } from "core/util/posthog";
Expand Down Expand Up @@ -138,7 +139,11 @@ export class Core {
const config = await this.config();
const items = config.contextProviders
?.find((provider) => provider.description.title === msg.data.title)
?.loadSubmenuItems({ ide: this.ide });
?.loadSubmenuItems({
ide: this.ide,
fetch: (url, init) =>
fetchwithRequestOptions(url, init, config.requestOptions),
});
return items || [];
});
on("context/getContextItems", async (msg) => {
Expand All @@ -160,6 +165,8 @@ export class Core {
ide,
selectedCode: msg.data.selectedCode,
reranker: config.reranker,
fetch: (url, init) =>
fetchwithRequestOptions(url, init, config.requestOptions),
});

Telemetry.capture("useContextProvider", {
Expand Down Expand Up @@ -278,6 +285,8 @@ export class Core {
},
selectedCode,
config,
fetch: (url, init) =>
fetchwithRequestOptions(url, init, config.requestOptions),
})) {
if (content) {
yield { content };
Expand Down
5 changes: 4 additions & 1 deletion binary/src/messenger.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,10 @@ export class IpcMessenger {
constructor() {
const logger = (message: any, ...optionalParams: any[]) => {
const logFilePath = getCoreLogsPath();
const logMessage = `${message} ${optionalParams.join(" ")}\n`;
const timestamp = new Date().toISOString().split(".")[0];
const logMessage = `[${timestamp}] ${message} ${optionalParams.join(
" ",
)}\n`;
fs.appendFileSync(logFilePath, logMessage);
};
console.log = logger;
Expand Down
38 changes: 33 additions & 5 deletions core/autocomplete/completionProvider.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ export interface AutocompleteInput {
recentlyEditedFiles: RangeInFileWithContents[];
recentlyEditedRanges: RangeInFileWithContents[];
clipboardText: string;
// Used for notebook files
manuallyPassFileContents?: string;
// Used for VS Code git commit input box
manuallyPassPrefix?: string;
}

export interface AutocompleteOutcome extends TabAutocompleteOptions {
Expand All @@ -57,13 +61,20 @@ const DOUBLE_NEWLINE = "\n\n";
const WINDOWS_DOUBLE_NEWLINE = "\r\n\r\n";
const SRC_DIRECTORY = "/src/";
// Starcoder2 tends to output artifacts starting with the letter "t"
const STARCODER2_T_ARTIFACTS = ["t.", "\nt"];
const STARCODER2_T_ARTIFACTS = ["t.", "\nt", "<file_sep>"];
const PYTHON_ENCODING = "#- coding: utf-8";
const CODE_BLOCK_END = "```";

const multilineStops = [DOUBLE_NEWLINE, WINDOWS_DOUBLE_NEWLINE];
const commonStops = [SRC_DIRECTORY, PYTHON_ENCODING, CODE_BLOCK_END];

// Errors that can be expected on occasion even during normal functioning should not be shown.
// Not worth disrupting the user to tell them that a single autocomplete request didn't go through
const ERRORS_TO_IGNORE = [
// From Ollama
"unexpected server status",
];

function formatExternalSnippet(
filepath: string,
snippet: string,
Expand Down Expand Up @@ -105,8 +116,11 @@ export async function getTabCompletion(
recentlyEditedFiles,
recentlyEditedRanges,
clipboardText,
manuallyPassFileContents,
manuallyPassPrefix,
} = input;
const fileContents = await ide.readFile(filepath);
const fileContents =
manuallyPassFileContents ?? (await ide.readFile(filepath));
const fileLines = fileContents.split("\n");

// Filter
Expand Down Expand Up @@ -137,7 +151,7 @@ export async function getTabCompletion(
) {
shownGptClaudeWarning = true;
throw new Error(
`Warning: ${llm.model} is not trained for tab-autocomplete, and will result in low-quality suggestions. See the docs to learn more about why: https://continue.dev/docs/walkthroughs/tab-autocomplete#i-want-better-completions-should-i-use-gpt-4`,
`Warning: ${llm.model} is not trained for tab-autocomplete, and will result in low-quality suggestions. See the docs to learn more about why: https://docs.continue.dev/walkthroughs/tab-autocomplete#i-want-better-completions-should-i-use-gpt-4`,
);
}

Expand Down Expand Up @@ -187,6 +201,12 @@ export async function getTabCompletion(
extrasSnippets,
);

// If prefix is manually passed
if (manuallyPassPrefix) {
prefix = manuallyPassPrefix;
suffix = "";
}

// Template prompt
const { template, completionOptions } = options.template
? { template: options.template, completionOptions: {} }
Expand Down Expand Up @@ -281,8 +301,16 @@ export async function getTabCompletion(
lineGenerator = streamWithNewLines(lineGenerator);

const finalGenerator = stopAtSimilarLine(lineGenerator, lineBelowCursor);
for await (const update of finalGenerator) {
completion += update;

try {
for await (const update of finalGenerator) {
completion += update;
}
} catch (e: any) {
if (ERRORS_TO_IGNORE.some((err) => e.includes(err))) {
return undefined;
}
throw e;
}

if (cancelled) {
Expand Down
5 changes: 4 additions & 1 deletion core/autocomplete/languages.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,9 @@ export const Typescript = {

// Python
export const Python = {
stopWords: ["def", "class"],
// """"#" is for .ipynb files, where we add '"""' surrounding markdown blocks.
// This stops the model from trying to complete the start of a new markdown block
stopWords: ["def", "class", '"""#'],
comment: "#",
endOfLine: [],
};
Expand Down Expand Up @@ -211,6 +213,7 @@ export const LANGUAGES: { [extension: string]: AutocompleteLanguageInfo } = {
js: Typescript,
tsx: Typescript,
jsx: Typescript,
ipynb: Python,
py: Python,
pyi: Python,
java: Java,
Expand Down
12 changes: 11 additions & 1 deletion core/autocomplete/lineStream.ts
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,16 @@ export async function* stopAtLines(stream: LineStream): LineStream {
}
}

const LINES_TO_SKIP = ["</START EDITING HERE>"];

export async function* skipLines(stream: LineStream): LineStream {
for await (const line of stream) {
if (!LINES_TO_SKIP.some((skipAt) => line.startsWith(skipAt))) {
yield line;
}
}
}

function shouldRemoveLineBeforeStart(line: string): boolean {
return (
line.trimStart().startsWith("```") ||
Expand Down Expand Up @@ -129,7 +139,7 @@ export async function* filterCodeBlockLines(rawLines: LineStream): LineStream {
return;
}

if (line === "```") {
if (line.startsWith("```")) {
waitingToSeeIfLineIsLast = line;
} else {
yield line;
Expand Down
15 changes: 12 additions & 3 deletions core/autocomplete/templates.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,17 @@ const stableCodeFimTemplate: AutocompleteTemplate = {
};

const codegemmaFimTemplate: AutocompleteTemplate = {
template: "<|fim_prefix|>{{{prefix}}}<|fim_suffix|>{{{suffix}}}<|fim_middle|>",
template:
"<|fim_prefix|>{{{prefix}}}<|fim_suffix|>{{{suffix}}}<|fim_middle|>",
completionOptions: {
stop: ["<|fim_prefix|>", "<|fim_suffix|>", "<|fim_middle|>", "<|file_separator|>", "<end_of_turn>", "<eos>"],
stop: [
"<|fim_prefix|>",
"<|fim_suffix|>",
"<|fim_middle|>",
"<|file_separator|>",
"<end_of_turn>",
"<eos>",
],
},
};

Expand Down Expand Up @@ -106,7 +114,8 @@ export function getTemplateForModel(model: string): AutocompleteTemplate {
lowerCaseModel.includes("star-coder") ||
lowerCaseModel.includes("starchat") ||
lowerCaseModel.includes("octocoder") ||
lowerCaseModel.includes("stable")
lowerCaseModel.includes("stable") ||
lowerCaseModel.includes("codeqwen")
) {
return stableCodeFimTemplate;
}
Expand Down
10 changes: 8 additions & 2 deletions core/commands/slash/edit.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import {
} from "../../autocomplete/lineStream";
import { streamLines } from "../../diff/util";
import { stripImages } from "../../llm/countTokens";
import { dedentAndGetCommonWhitespace } from "../../util";
import { dedentAndGetCommonWhitespace, getMarkdownLanguageTagForFile } from "../../util";
import {
RangeInFileWithContents,
contextItemToRangeInFileWithContents,
Expand Down Expand Up @@ -226,7 +226,7 @@ const EditSlashCommand: SlashCommand = {
}

if (!contextItemToEdit) {
yield "Select (highlight and press `cmd+shift+L` (MacOS) / `ctrl+shift+L` (Windows)) the code that you want to edit first";
yield "Please highlight the code you want to edit, then press `cmd/ctrl+shift+L` to add it to chat";
return;
}

Expand Down Expand Up @@ -456,6 +456,12 @@ const EditSlashCommand: SlashCommand = {
userInput,
filePrefix: filePrefix,
fileSuffix: fileSuffix,

// Some built-in templates use these instead of the above
prefix: filePrefix,
suffix: fileSuffix,

language: getMarkdownLanguageTagForFile(rif.filepath),
systemMessage: llm.systemMessage ?? "",
// "contextItems": (await sdk.getContextItemChatMessages()).map(x => x.content || "").join("\n\n"),
},
Expand Down
2 changes: 1 addition & 1 deletion core/commands/slash/http.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import { removeQuotesAndEscapes } from "../../util";
const HttpSlashCommand: SlashCommand = {
name: "http",
description: "Call an HTTP endpoint to serve response",
run: async function* ({ ide, llm, input, params }) {
run: async function* ({ ide, llm, input, params, fetch }) {
const url = params?.url;
if (!url) {
throw new Error("URL is not defined in params");
Expand Down
Loading

0 comments on commit cf4b557

Please sign in to comment.