-
-
Notifications
You must be signed in to change notification settings - Fork 515
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: add vector database modules (Qdrant, Weaviate, Chroma, pgvector…
…, OpenSearch, Milvus) (#2245) * fix: add nolint to the examples template * chore: bootstrap qdrant module * feat: add container methods for endpoints REST, gRPC and webUI * chore: add a full example using the gRPC client * fix: lint * fix: use inner t * chore: reduce network flakines on GH hosted runners OS-neutral interface to disable TCP/UDP offload to fix flaky networking on GitHub-hosted runners See https://github.com/smorimoto/tune-github-hosted-runner-network/blob/main/action.yml * Revert "chore: reduce network flakines on GH hosted runners" This reverts commit 7ce031e. * chore: try with daemon host * chore: hardcode 127.0.0.1 for testing on GH action * chore: print container logs on error * Revert "chore: hardcode 127.0.0.1 for testing on GH action" This reverts commit b602505. * Revert "chore: try with daemon host" This reverts commit 67a5abf. * chore: wait for the listening ports to be available * feat: new weaviate module * chore: add default values for the weviate module * chore: remove helper test method in qdrant * feat: add method to connect to the REST endpoint of weaviate * chore: add example for creating an weaviate client * docs: document weaviate methods * chore: bootstrap Chroma module * feat: support for passing env vars to modules * chore: add container method to get the base URL * chore: add example on how to connect to Chroma * docs: document weaviate client * chore: include heartbeat in the wait strategies * chore: add example for collections * Revert "feat: support for passing env vars to modules" This reverts commit ac97472. * feat: include pgvector to postgres module * fix: wrong value * feat: bootstrap opensearch module * chore: configure opensearch * chore: implement options and methods for opensearch * chore: bootstrap milvus * chore: improve container lifecycle and defaults for embed etcd * chore: add full example for creating collections * fix: lint * chore: rename to HttpHostAddress
- Loading branch information
1 parent
91f6f59
commit c84e479
Showing
45 changed files
with
3,740 additions
and
8 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| # Chroma | ||
|
|
||
| Not available until the next release of testcontainers-go <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a> | ||
|
|
||
| ## Introduction | ||
|
|
||
| The Testcontainers module for Chroma. | ||
|
|
||
| ## Adding this module to your project dependencies | ||
|
|
||
| Please run the following command to add the Chroma module to your Go dependencies: | ||
|
|
||
| ``` | ||
| go get github.com/testcontainers/testcontainers-go/modules/chroma | ||
| ``` | ||
|
|
||
| ## Usage example | ||
|
|
||
| <!--codeinclude--> | ||
| [Creating a Chroma container](../../modules/chroma/examples_test.go) inside_block:runChromaContainer | ||
| <!--/codeinclude--> | ||
|
|
||
| ## Module reference | ||
|
|
||
| The Chroma module exposes one entrypoint function to create the Chroma container, and this function receives two parameters: | ||
|
|
||
| ```golang | ||
| func RunContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*ChromaContainer, error) | ||
| ``` | ||
|
|
||
| - `context.Context`, the Go context. | ||
| - `testcontainers.ContainerCustomizer`, a variadic argument for passing options. | ||
|
|
||
| ### Container Options | ||
|
|
||
| When starting the Chroma container, you can pass options in a variadic way to configure it. | ||
|
|
||
| #### Image | ||
|
|
||
| If you need to set a different Chroma Docker image, you can use `testcontainers.WithImage` with a valid Docker image | ||
| for Chroma. E.g. `testcontainers.WithImage("chromadb/chroma:0.4.22.dev44")`. | ||
|
|
||
| {% include "../features/common_functional_options.md" %} | ||
|
|
||
| ### Container Methods | ||
|
|
||
| The Chroma container exposes the following methods: | ||
|
|
||
| #### REST Endpoint | ||
|
|
||
| This method returns the REST endpoint of the Chroma container, using the default `8000` port. | ||
|
|
||
| <!--codeinclude--> | ||
| [Get REST endpoint](../../modules/chroma/chroma_test.go) inside_block:restEndpoint | ||
| <!--/codeinclude--> | ||
|
|
||
| ## Examples | ||
|
|
||
| ### Getting a Chroma client | ||
|
|
||
| The following example demonstrates how to create a Chroma client using the Chroma module. | ||
|
|
||
| First of all, you need to import the Chroma module and the Swagger client: | ||
|
|
||
| ```golang | ||
| import ( | ||
| chromago "github.com/amikos-tech/chroma-go" | ||
| chromaopenapi "github.com/amikos-tech/chroma-go/swagger" | ||
| ) | ||
| ``` | ||
|
|
||
| Then, you can create a Chroma client using the Chroma module: | ||
|
|
||
| <!--codeinclude--> | ||
| [Get the client](../../modules/chroma/examples_test.go) inside_block:createClient | ||
| <!--/codeinclude--> | ||
|
|
||
| ### Working with Collections | ||
|
|
||
| <!--codeinclude--> | ||
| [Create Collection](../../modules/chroma/examples_test.go) inside_block:createCollection | ||
| [List Collections](../../modules/chroma/examples_test.go) inside_block:listCollections | ||
| [Delete Collection](../../modules/chroma/examples_test.go) inside_block:deleteCollection | ||
| <!--/codeinclude--> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| # Milvus | ||
|
|
||
| Not available until the next release of testcontainers-go <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a> | ||
|
|
||
| ## Introduction | ||
|
|
||
| The Testcontainers module for Milvus. | ||
|
|
||
| ## Adding this module to your project dependencies | ||
|
|
||
| Please run the following command to add the Milvus module to your Go dependencies: | ||
|
|
||
| ``` | ||
| go get github.com/testcontainers/testcontainers-go/modules/milvus | ||
| ``` | ||
|
|
||
| ## Usage example | ||
|
|
||
| <!--codeinclude--> | ||
| [Creating a Milvus container](../../modules/milvus/examples_test.go) inside_block:runMilvusContainer | ||
| <!--/codeinclude--> | ||
|
|
||
| ## Module reference | ||
|
|
||
| The Milvus module exposes one entrypoint function to create the Milvus container, and this function receives two parameters: | ||
|
|
||
| ```golang | ||
| func RunContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*MilvusContainer, error) | ||
| ``` | ||
|
|
||
| - `context.Context`, the Go context. | ||
| - `testcontainers.ContainerCustomizer`, a variadic argument for passing options. | ||
|
|
||
| ### Container Options | ||
|
|
||
| When starting the Milvus container, you can pass options in a variadic way to configure it. | ||
|
|
||
| #### Image | ||
|
|
||
| If you need to set a different Milvus Docker image, you can use `testcontainers.WithImage` with a valid Docker image | ||
| for Milvus. E.g. `testcontainers.WithImage("milvusdb/milvus:v2.3.9")`. | ||
|
|
||
| {% include "../features/common_functional_options.md" %} | ||
|
|
||
| ### Container Methods | ||
|
|
||
| The Milvus container exposes the following methods: | ||
|
|
||
| #### ConnectionString | ||
|
|
||
| This method returns the connection string to connect to the Milvus container, using the default `19530` port. | ||
|
|
||
| <!--codeinclude--> | ||
| [Get connection string](../../modules/milvus/milvus_test.go) inside_block:connectionString | ||
| <!--/codeinclude--> | ||
|
|
||
| ## Examples | ||
|
|
||
| ### Creating collections | ||
|
|
||
| This example shows the usage of the Milvus module to create and retrieve collections. | ||
|
|
||
| <!--codeinclude--> | ||
| [Create collections](../../modules/milvus/examples_test.go) inside_block:createCollections | ||
| <!--/codeinclude--> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| # OpenSearch | ||
|
|
||
| Not available until the next release of testcontainers-go <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a> | ||
|
|
||
| ## Introduction | ||
|
|
||
| The Testcontainers module for OpenSearch. | ||
|
|
||
| ## Adding this module to your project dependencies | ||
|
|
||
| Please run the following command to add the OpenSearch module to your Go dependencies: | ||
|
|
||
| ``` | ||
| go get github.com/testcontainers/testcontainers-go/modules/opensearch | ||
| ``` | ||
|
|
||
| ## Usage example | ||
|
|
||
| <!--codeinclude--> | ||
| [Creating a OpenSearch container](../../modules/opensearch/examples_test.go) inside_block:runOpenSearchContainer | ||
| <!--/codeinclude--> | ||
|
|
||
| ## Module reference | ||
|
|
||
| The OpenSearch module exposes one entrypoint function to create the OpenSearch container, and this function receives two parameters: | ||
|
|
||
| ```golang | ||
| func RunContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*OpenSearchContainer, error) | ||
| ``` | ||
|
|
||
| - `context.Context`, the Go context. | ||
| - `testcontainers.ContainerCustomizer`, a variadic argument for passing options. | ||
|
|
||
| ### Container Options | ||
|
|
||
| When starting the OpenSearch container, you can pass options in a variadic way to configure it. | ||
|
|
||
| #### Image | ||
|
|
||
| If you need to set a different OpenSearch Docker image, you can use `testcontainers.WithImage` with a valid Docker image | ||
| for OpenSearch. E.g. `testcontainers.WithImage("opensearchproject/opensearch:2.11.1")`. | ||
|
|
||
| {% include "../features/common_functional_options.md" %} | ||
|
|
||
| #### User and password | ||
|
|
||
| If you need to set a different password to request authorization when performing HTTP requests to the container, you can use the `WithUsername` and `WithPassword` options. By default, the username is set to `admin`, and the password is set to `admin`. | ||
|
|
||
| <!--codeinclude--> | ||
| [Custom Credentials](../../modules/opensearch/examples_test.go) inside_block:runOpenSearchContainer | ||
| <!--/codeinclude--> | ||
|
|
||
| ### Container Methods | ||
|
|
||
| The OpenSearch container exposes the following methods: | ||
|
|
||
| #### Address | ||
|
|
||
| The `Address` method returns the location where the OpenSearch container is listening. | ||
| It returns a string with the format `http://<host>:<port>`. | ||
|
|
||
| !!!warning | ||
| TLS is not supported at the moment. | ||
|
|
||
| <!--codeinclude--> | ||
| [Connecting using HTTP](../../modules/opensearch/opensearch_test.go) inside_block:httpConnection | ||
| <!--/codeinclude--> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.