Skip to content

Commit

Permalink
Merge branch 'main' into session-ids
Browse files Browse the repository at this point in the history
* main:
  chore: always generate the examples including a explicit image (testcontainers#1611)
  chore: support linting all modules locally (testcontainers#1609)
  ci(sonarcloud): run only when not root-less (testcontainers#1608)
  remove extra equal in sonar.organization (testcontainers#1607)
  chore(ci): enable code coverage on Sonarcloud (testcontainers#1518)
  docs: use Go testable examples in modules (testcontainers#1603)
  ci(lint): enable errorlint linter (testcontainers#1604)
  ci(lint): enable gocritic linter (testcontainers#1605)
  chore(deps): bump github.com/hashicorp/vault-client-go in /modules/vault (testcontainers#1566)
  ci(security): setup codeql scan  (testcontainers#1606)
  chore(deps): bump github.com/cyphar/filepath-securejoin (testcontainers#1601)
  chore: generate Go examples for new modules (testcontainers#1600)
  chore: generate Go examples for new modules (testcontainers#1600)
  • Loading branch information
mdelapenya committed Sep 12, 2023
2 parents 9655950 + b25ee5a commit 1de0dca
Show file tree
Hide file tree
Showing 94 changed files with 1,513 additions and 734 deletions.
11 changes: 10 additions & 1 deletion .github/workflows/ci-test-go.yml
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ jobs:
# Optional: working directory, useful for monorepos
working-directory: ${{ inputs.project-directory }}
# Optional: golangci-lint command line arguments.
args: --verbose
args: --verbose --out-format=checkstyle:golangci-lint.report.xml
# Optional: if set to true then the all caching functionality will be complete disabled,
# takes precedence over all other caching options.
skip-cache: true
Expand Down Expand Up @@ -107,6 +107,15 @@ jobs:
run: |
go install gotest.tools/gotestsum@latest
make test-unit
- name: Analyze with SonarCloud
if: ${{ github.ref_name == 'main' && github.repository_owner == 'testcontainers' && inputs.run-tests && !inputs.rootless-docker && inputs.project-directory == '.' }}
uses: sonarsource/sonarcloud-github-action@master
with:
projectBaseDir: ${{ inputs.project-directory }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

- name: Run checker
run: |
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ jobs:
go-version: [1.20.x, 1.x]
platform: [ubuntu-latest, macos-latest]
uses: ./.github/workflows/ci-test-go.yml
secrets: inherit
with:
go-version: ${{ matrix.go-version }}
fail-fast: true
Expand All @@ -51,6 +52,7 @@ jobs:
matrix:
go-version: [1.20.x, 1.x]
uses: ./.github/workflows/ci-test-go.yml
secrets: inherit
with:
go-version: ${{ matrix.go-version }}
fail-fast: true
Expand All @@ -72,6 +74,7 @@ jobs:
go-version: [1.20.x, 1.x]
platform: [ubuntu-latest]
uses: ./.github/workflows/ci-test-go.yml
secrets: inherit
with:
go-version: ${{ matrix.go-version }}
fail-fast: true
Expand All @@ -89,6 +92,7 @@ jobs:
go-version: [1.20.x, 1.x]
platform: [ubuntu-latest, macos-latest, windows-latest]
uses: ./.github/workflows/ci-test-go.yml
secrets: inherit
with:
go-version: ${{ matrix.go-version }}
fail-fast: true
Expand All @@ -108,6 +112,7 @@ jobs:
platform: [ubuntu-latest, macos-latest]
module: [artemis, clickhouse, compose, couchbase, elasticsearch, k3s, localstack, mariadb, mongodb, mysql, nats, neo4j, postgres, pulsar, redis, redpanda, vault]
uses: ./.github/workflows/ci-test-go.yml
secrets: inherit
with:
go-version: ${{ matrix.go-version }}
fail-fast: false
Expand All @@ -125,6 +130,7 @@ jobs:
matrix:
module: [bigtable, cockroachdb, consul, datastore, firestore, nginx, pubsub, spanner, toxiproxy]
uses: ./.github/workflows/ci-test-go.yml
secrets: inherit
with:
go-version: "1.20.x"
fail-fast: true
Expand Down
82 changes: 82 additions & 0 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL"

on:
push:
branches: [ "main" ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ "main" ]
schedule:
- cron: '40 6 * * 5'

jobs:
analyze:
name: Analyze
# Runner size impacts CodeQL analysis time. To learn more, please see:
# - https://gh.io/recommended-hardware-resources-for-running-codeql
# - https://gh.io/supported-runners-and-hardware-resources
# - https://gh.io/using-larger-runners
# Consider using larger runners for possible analysis time improvements.
runs-on: ${{ (matrix.language == 'swift' && 'macos-latest') || 'ubuntu-latest' }}
timeout-minutes: ${{ (matrix.language == 'swift' && 120) || 360 }}
permissions:
actions: read
contents: read
security-events: write

strategy:
fail-fast: false
matrix:
language: [ 'go', 'javascript']
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby', 'swift' ]
# Use only 'java' to analyze code written in Java, Kotlin or both
# Use only 'javascript' to analyze code written in JavaScript, TypeScript or both
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support

steps:
- name: Checkout repository
uses: actions/checkout@v3

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.

# For more details on CodeQL's query packs, refer to: https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
# queries: security-extended,security-and-quality


# Autobuild attempts to build any compiled languages (C/C++, C#, Go, Java, or Swift).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v2

# ℹ️ Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun

# If the Autobuild fails above, remove it and uncomment the following three lines.
# modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance.

# - run: |
# echo "Run, Build Application using script"
# ./location_of_script_within_repo/buildscript.sh

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
with:
category: "/language:${{matrix.language}}"
13 changes: 12 additions & 1 deletion .golangci.yml
Original file line number Diff line number Diff line change
@@ -1,15 +1,26 @@
linters:
enable:
- errorlint
- gci
- gocritic
- gofumpt
- misspell

linters-settings:
errorlint:
# Check whether fmt.Errorf uses the %w verb for formatting errors.
# See the https://github.com/polyfloyd/go-errorlint for caveats.
errorf: true
# Permit more than 1 %w verb, valid per Go 1.20 (Requires errorf:true)
errorf-multi: true
# Check for plain type assertions and type switches.
asserts: true
# Check for plain error comparisons.
comparison: true
gci:
sections:
- standard
- default
- prefix(github.com/testcontainers)

run:
timeout: 5m
7 changes: 7 additions & 0 deletions commons-test.mk
Original file line number Diff line number Diff line change
@@ -1,8 +1,14 @@
ROOT_DIR:=$(shell dirname $(realpath $(lastword $(MAKEFILE_LIST))))

.PHONY: dependencies-scan
dependencies-scan:
@echo ">> Scanning dependencies in $(CURDIR)..."
go list -json -m all | docker run --rm -i sonatypecommunity/nancy:latest sleuth --skip-update-check

.PHONY: lint
lint:
golangci-lint run --out-format=github-actions --path-prefix=. --verbose -c $(ROOT_DIR)/.golangci.yml --fix

.PHONY: test-%
test-%:
@echo "Running $* tests..."
Expand All @@ -12,6 +18,7 @@ test-%:
--packages="./..." \
--junitfile TEST-$*.xml \
-- \
-coverprofile=coverage.out \
-timeout=30m

.PHONY: tools
Expand Down
6 changes: 4 additions & 2 deletions docker.go
Original file line number Diff line number Diff line change
Expand Up @@ -792,7 +792,8 @@ func (p *DockerProvider) BuildImage(ctx context.Context, img ImageBuildInfo) (st
err = backoff.Retry(func() error {
resp, err = p.client.ImageBuild(ctx, buildContext, buildOptions)
if err != nil {
if _, ok := err.(errdefs.ErrNotFound); ok {
var enf errdefs.ErrNotFound
if errors.As(err, &enf) {
return backoff.Permanent(err)
}
Logger.Printf("Failed to build image: %s, will retry", err)
Expand Down Expand Up @@ -1155,7 +1156,8 @@ func (p *DockerProvider) attemptToPullImage(ctx context.Context, tag string, pul
err = backoff.Retry(func() error {
pull, err = p.client.ImagePull(ctx, tag, pullOpt)
if err != nil {
if _, ok := err.(errdefs.ErrNotFound); ok {
var enf errdefs.ErrNotFound
if errors.As(err, &enf) {
return backoff.Permanent(err)
}
Logger.Printf("Failed to pull image: %s, will retry", err)
Expand Down
13 changes: 13 additions & 0 deletions docs/features/creating_networks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# How to create a network

Apart from creating containers, `Testcontainers for Go` also allows you to create networks. This is useful when you need to connect multiple containers to the same network.

## Usage example

<!--codeinclude-->
[Creating a network](../../network_test.go) inside_block:createNetwork
<!--/codeinclude-->

<!--codeinclude-->
[Creating a network with IPAM](../../network_test.go) inside_block:withIPAM
<!--/codeinclude-->
16 changes: 11 additions & 5 deletions docs/modules/artemis.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,11 @@ go get github.com/testcontainers/testcontainers-go/modules/artemis
## Usage example

<!--codeinclude-->
[Creating and connecting to an Artemis container](../../modules/artemis/example_test.go) inside_block:ExampleRunContainer
[Creating to an Artemis container](../../modules/artemis/examples_test.go) inside_block:runArtemisContainer
<!--/codeinclude-->

<!--codeinclude-->
[Connecting to an Artemis container](../../modules/artemis/examples_test.go) inside_block:connectToArtemisContainer
<!--/codeinclude-->

## Module reference
Expand Down Expand Up @@ -58,7 +62,7 @@ If you need to change the default admin credentials (i.e. `artemis:artemis`) use
[With credentials](../../modules/artemis/artemis_test.go) inside_block:withCredentials
<!--/codeinclude-->

#### Anonymous Login
#### Enable Anonymous login

If you need to enable anonymous logins (which are disabled by default) use `WithAnonymousLogin`.

Expand All @@ -71,7 +75,9 @@ If you need to enable anonymous logins (which are disabled by default) use `With
If you need to pass custom arguments to the `artemis create` command, use `WithExtraArgs`.
The default is `--http-host 0.0.0.0 --relax-jolokia`.
Setting this value will override the default.
See the documentation on `artemis create` for available options.

!!!info
Please see the documentation on `artemis create` for the available options here: [https://activemq.apache.org/components/artemis/documentation/latest/using-server.html#options](https://activemq.apache.org/components/artemis/documentation/latest/using-server.html#options)

<!--codeinclude-->
[With Extra Arguments](../../modules/artemis/artemis_test.go) inside_block:withExtraArgs
Expand All @@ -96,15 +102,15 @@ The Artemis container exposes the following methods:
User returns the administrator username.

<!--codeinclude-->
[Retrieving the Administrator User](../../modules/artemis/example_test.go) inside_block:containerUser
[Retrieving the Administrator User](../../modules/artemis/examples_test.go) inside_block:containerUser
<!--/codeinclude-->

#### Password

Password returns the administrator password.

<!--codeinclude-->
[Retrieving the Administrator Password](../../modules/artemis/example_test.go) inside_block:containerPassword
[Retrieving the Administrator Password](../../modules/artemis/examples_test.go) inside_block:containerPassword
<!--/codeinclude-->

#### BrokerEndpoint
Expand Down
56 changes: 30 additions & 26 deletions docs/modules/clickhouse.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,7 @@ go get github.com/testcontainers/testcontainers-go/modules/clickhouse
## Usage example

<!--codeinclude-->

[Test for a ClickHouse container](../../modules/clickhouse/clickhouse_test.go)inside_block:customInitialization

[Test for a ClickHouse container](../../modules/clickhouse/examples_test.go) inside_block:runClickHouseContainer
<!--/codeinclude-->

## Module reference
Expand All @@ -33,6 +31,14 @@ func RunContainer(ctx context.Context, opts ...testcontainers.ContainerCustomize
- `context.Context`, the Go context.
- `testcontainers.ContainerCustomizer`, a variadic argument for passing options.

### Container Ports

Here you can find the list with the default ports used by the ClickHouse container.

<!--codeinclude-->
[Container Ports](../../modules/clickhouse/clickhouse.go) inside_block:containerPorts
<!--/codeinclude-->

### Container Options

When starting the ClickHouse container, you can pass options in a variadic way to configure it.
Expand Down Expand Up @@ -65,13 +71,7 @@ Please read the [Create containers: Advanced Settings](../features/creating_cont
#### Set username, password and database name

If you need to set a different database, and its credentials, you can use `WithUsername`, `WithPassword`, `WithDatabase`
options.

<!--codeinclude-->

[Custom Database initialization](../../modules/clickhouse/clickhouse_test.go) inside_block:customInitialization

<!--/codeinclude-->
options. E.g. `WithUsername("user")`, `WithPassword("password")`, `WithDatabase("db")`.

!!!info
The default values for the username is `default`, for password is `clickhouse` and for the default database name is `clickhouse`.
Expand All @@ -84,42 +84,46 @@ it will run any `*.sql` files, run any executable `*.sh` scripts, and source any
initialization before starting the service.

<!--codeinclude-->

[Include init scripts](../../modules/clickhouse/clickhouse_test.go) inside_block:withInitScripts
<!--/codeinclude-->

<!--codeinclude-->
[Init script content](../../modules/clickhouse/testdata/init-db.sh)
<!--/codeinclude-->

#### Custom configuration

If you need to set a custom configuration, the module provides the `WithConfigFile` option to pass the path to a custom configuration file in XML format.

<!--codeinclude-->
[XML config file](../../modules/clickhouse/testdata/config.xml)
<!--/codeinclude-->

[Init script content](../../modules/clickhouse/testdata/init-db.sh)
In the case you want to pass a YAML configuration file, you can use the `WithYamlConfigFile` option.

<!--codeinclude-->
[YAML config file](../../modules/clickhouse/testdata/config.yaml)
<!--/codeinclude-->

### Container Methods

The ClickHouse container exposes the following methods:

#### ConnectionString

This method returns the dsn connection string to connect to the ClickHouse container, using the default configs.
It's possible to pass extra parameters to the connection string, e.g. `dial_timeout=300ms` or `skip_verify=false`, in a variadic way.
#### ConnectionHost

e.g. `clickhouse://default:pass@localhost:9000?dial_timeout=300ms&skip_verify=false`
This method returns the host and port of the ClickHouse container, using the default, native `9000/tcp` port. E.g. `localhost:9000`

<!--codeinclude-->

[Get connection string](../../modules/clickhouse/clickhouse_test.go) inside_block:connectionString

[Get connection host](../../modules/clickhouse/clickhouse_test.go) inside_block:connectionHost
<!--/codeinclude-->

#### ConnectionHost
#### ConnectionString

This method returns the host & port of the ClickHouse container.
This method returns the dsn connection string to connect to the ClickHouse container, using the default, native `9000/tcp` port obtained from the `ConnectionHost` method.
It's possible to pass extra parameters to the connection string, e.g. `dial_timeout=300ms` or `skip_verify=false`, in a variadic way.

e.g. `localhost:9000`
e.g. `clickhouse://default:pass@localhost:9000?dial_timeout=300ms&skip_verify=false`

<!--codeinclude-->

[Get connection host](../../modules/clickhouse/clickhouse_test.go) inside_block:connectionHost

[Get connection string](../../modules/clickhouse/clickhouse_test.go) inside_block:connectionString
<!--/codeinclude-->
Loading

0 comments on commit 1de0dca

Please sign in to comment.