[Bug]: Error when using WithUsername and WithPassword with the WithReplicaSet option

[Chris-Sheridan]

Testcontainers version

v0.33.0

Using the latest Testcontainers version?

Yes

Host OS

MacOS

Host arch

ARM

Go version

1.23.0

Docker version

Client:
 Cloud integration: v1.0.35+desktop.13
 Version:           26.0.0
 API version:       1.45
 Go version:        go1.21.8
 Git commit:        2ae903e
 Built:             Wed Mar 20 15:14:46 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.29.0 (145265)
 Engine:
  Version:          26.0.0
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.8
  Git commit:       8b79278
  Built:            Wed Mar 20 15:18:02 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.28
  GitCommit:        ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Docker info

Client:
 Version:    26.0.0
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.13.1-desktop.1
    Path:     /Users/chriss/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.26.1-desktop.1
    Path:     /Users/chriss/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container. (Docker Inc.)
    Version:  0.0.27
    Path:     /Users/chriss/.docker/cli-plugins/docker-debug
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/chriss/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.23
    Path:     /Users/chriss/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.4
    Path:     /Users/chriss/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.1.0
    Path:     /Users/chriss/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/chriss/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.6.3
    Path:     /Users/chriss/.docker/cli-plugins/docker-scout

Server:
 Containers: 1
  Running: 0
  Paused: 0
  Stopped: 1
 Images: 22
 Server Version: 26.0.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.6.22-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 12
 Total Memory: 7.657GiB
 Name: docker-desktop
 ID: 7b791e9f-a4a6-4582-94dd-1d6571714207
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/chriss/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

What happened?

When running the following code, I consistently receive the error message stated in the log output. I tried different mongo versions and with/without wait strategies and it's the same result.

mongo, err := mongodb.Run(ctx, "mongo:latest", mongodb.WithReplicaSet("rs0"), mongodb.WithUsername("root"), mongodb.WithPassword("example"), testcontainers.WithWaitStrategy(wait.ForLog("Waiting for connections")), )

Relevant log output

failed to start container: start container: started hook: container exec create: Error response from daemon: container b1e332e5aaf38ea01512481ebb257946551b61fc9ad4d56b0d15f1c2cc27eafa is not running
        all exposed ports, [27017/tcp], were not mapped in 5s: port 27017/tcp is not mapped yet

Additional information

No response

[abhipranay]

I faced same issue and I would like to work on this bug.
To replicate the issue i added below test case for mongo:6.

{
  name: "With Replica set, WithUsername, WithPassword and mongo:6",
  img:  "mongo:6",
  opts: []testcontainers.ContainerCustomizer{
    mongodb.WithReplicaSet("rs"),
    mongodb.WithUsername("tester"),
    mongodb.WithPassword("testpass"),
  },
}

The test failed with below error.

=== RUN   TestMongoDB
=== RUN   TestMongoDB/With_Replica_set,_WithUsername,_WithPassword_and_mongo:6
=== PAUSE TestMongoDB/With_Replica_set,_WithUsername,_WithPassword_and_mongo:6
=== CONT  TestMongoDB/With_Replica_set,_WithUsername,_WithPassword_and_mongo:6
    mongodb_test.go:80: 
        	Error Trace:	~/testcontainers-go/modules/mongodb/mongodb_test.go:80
        	Error:      	Received unexpected error:
        	            	generic container: start container: started hook: container exec create: Error response from daemon: Container 37d341411e5c383c4718f8108f99ea63265418208ffe887db14fc881bb55d4c3 is not running
        	            	all exposed ports, [27017/tcp], were not mapped in 5s: port 27017/tcp is not mapped yet
        	            	wait until ready: check target: retries: 1, port: "", last err: container exited with code 2
        	Test:       	TestMongoDB/With_Replica_set,_WithUsername,_WithPassword_and_mongo:6
--- FAIL: TestMongoDB/With_Replica_set,_WithUsername,_WithPassword_and_mongo:6 (89.00s)

[abhipranay]

It seems when root username and password are set in mongodb containers then security.keyfile must be passed.
On debugging and checking the logs of testcontainer i found below error in logs.

Container logs for container: 785fef5b21876ed71a15c1be0a54363bf0bc3b1ccc96ba01f797606ccdd85cac 

about to fork child process, waiting until server is ready for connections.
forked process: 28

}

MongoDB init process complete; ready for start up.
<some other logs: redacted>

BadValue: security.keyFile is required when authorization is enabled with replica sets
try 'mongod --help' for more information

Adding keyfile inside container is not very straight forward. The keyfile requires specific permission and should be read only by mongodb user or group members.
Adding file is not a problem as we can use add it through Files in container request. But adding it with proper ownership is not easy.

@mdelapenya Can you suggest if there is a way to add file to containers with specific ownership. Also it is worth solving this problem. I think another approach is to allow replicaset without username passwords.

[mdelapenya]

@abhipranay thanks for your support in identifying the root cause, I'm happy to review your work when sent. 👏

We do not have a way to define permissions in copied files unless you define a postStart hook running chown. I see that our copyToContainer code is defining tar headers but not defining the one for User and Group. Probably a separare enhancement to be done at the ContainerFile struct, which is the one we use to transfer the files from the host to the container. But unfortunately, checking the public API methods on the container (i.e. CopyToContainer) adding the support for that would come with a breaking change because of a change in the signature of the methods:

CopyToContainer(ctx context.Context, fileContent []byte, containerFilePath string, fileMode int64) error
CopyDirToContainer(ctx context.Context, hostDirPath string, containerParentPath string, fileMode int64) error
CopyFileToContainer(ctx context.Context, hostFilePath string, containerFilePath string, fileMode int64) error

In any case, I'm not against the change offering a deprecation path. Wdyt?

[abhipranay]

@mdelapenya Thanks for the details. I agree as well on deprecation path for mongo module. Given the usecases for testcontainer is in test environments, it doesn't seem a good idea to aim to provide support for all types of configurations as many of them make sense for production environments.

For more advanced use cases where user really care about security configs, testcontainer ContainerRequest.FromDockerfile option can be used or custom image can be used.

I will raise PR for deprecating WithReplicaSet in combination with WithUsername or WithPassword.

[mdelapenya]

@abhipranay my interest in the deprecation was not for the replicaset but for the copy APIs in the core library, as the replicaset is used test production scenarios.

And deprecating them with an alternative method, that we should discuss the name, something like container.CopyTo, or even a public function like testcontainers.CopyTo(ctx, container)

[abhipranay]

@mdelapenya I have another approach to tackle the problem. You may check the PR. Core idea in PR is to use a wrapper entrypoint which creates keyfile inside container with correct ownership and permissions before starting mongo daemon.

To make the changes safer the custom entrypoint is used only when needed.
Can you check the idea.