PAYARA-3988 Dynamic Docker Node Instances #4130

Pandrex247 · 2019-08-06T16:21:08Z

Brings the Autonaming functionality introduced recently to Docker Nodes.

The behaviour around the creation and lifecycle of Docker Instances has changed to be as follows:

If the Docker image is not present on the Docker Node machine, Payara Server will now instruct the Docker REST API to pull the image.
--autoname can be used with the create-instance command as described in the original Dynamic Instances PR to deal with any name conflicts.
- If there is a conflict in the Docker container names, the instance will still fail to be created. This is unlikely to happen during normal operation, and should only really occur if the DAS and the Docker Node somehow get out of sync.
The stop-instance command will now stop the Docker container
The Docker containers are now started, stopped, and deleted based on their ID, rather than their name.
- This allows Payara Server instances created within Docker Containers spawned manually or via scaling to register themselves with the DAS and retain the ability to be managed by the DAS.
If a new container is created and a name conflict is detected with another Payara Server instance, a new instance will be created.
- This should only happen if you create a container manually, or it is somehow duplicated via scaling.
If a new container is created and no instance name is provided to it, a new instance with an auto-generated name will be created.
- Again, this should only happen in cases where you've created a container manually or it has otherwise been scaled up.

Please Note: If creating containers manually or otherwise scaling them, the instances are still tied to a Docker Node, and so the containers will fail to start if the host name does not match what has been configured on the DAS (or if the node doesn't exist).

Examples

Asadmin Examples

This example shows that no new behaviour should be expected on-top of what is already described in #4090.

asadmin -a create-instance --node docky1 insty1     >>>     insty1 created in a container named insty1
asadmin -a create-instance --node docky1 insty1     >>>     insty1-BemusedBass created in a container named insty1-BemusedBass
asadmin -a create-instance --node docky1            >>>     Capricious-Carp created in a container named Capricious-Carp

Manual/Scaling Examples

This example demonstrates what would happen if an instance called insty1 already existed on the DAS.

docker container create --network host --mount type=bind,source="/home/anon/passwordfile.txt",target="/opt/payara/passwords/passwordfile.txt",readonly -e PAYARA_DAS_HOST=payaraDas -e PAYARA_DAS_PORT=4848 -e PAYARA_NODE_NAME=docky1 -e PAYARA_INSTANCE_NAME=insty1 payara/server-node:latest     >>>     Instance named insty1-GarishGoldfish created in a container named elastic_ganguly

This example demonstrates creating new instances and containers on a Docker Node (you must create & configure the Docker Node on the DAS first), the last showing that the PAYARA_INSTANCE_NAME environment variable is now optional:

docker container create --network host --mount type=bind,source="/home/anon/passwordfile.txt",target="/opt/payara/passwords/passwordfile.txt",readonly -e PAYARA_DAS_HOST=payaraDas -e PAYARA_DAS_PORT=4848 -e PAYARA_NODE_NAME=docky1 -e PAYARA_INSTANCE_NAME=insty2 payara/server-node:latest     >>>     Instance named insty2 created in a container named elastic_ganguly

docker container create --network host --mount type=bind,source="/home/anon/passwordfile.txt",target="/opt/payara/passwords/passwordfile.txt",readonly -e PAYARA_DAS_HOST=payaraDas -e PAYARA_DAS_PORT=4848 -e PAYARA_NODE_NAME=docky1 -e PAYARA_INSTANCE_NAME=insty2 payara/server-node:latest     >>>     Instance named insty2-BamboozledBarrcuda created in a container named wonderful_yonath

docker container create --network host --mount type=bind,source="/home/anon/passwordfile.txt",target="/opt/payara/passwords/passwordfile.txt",readonly -e PAYARA_DAS_HOST=payaraDas -e PAYARA_DAS_PORT=4848 -e PAYARA_NODE_NAME=docky1 payara/server-node:latest     >>>     Instance named Magnanimous-Monkfish  created in a container named sleepy_elgamel

rather than name, and make sure autonaming works from console

Add stop container command so that instances can be started, stopped, and started again without an error. Create commands to get and set container ID of instances.

Pandrex247 · 2019-08-07T14:58:08Z

Jenkins test please

rdebusscher

Some small code improvements and doubt about the correct URL for pull image.

rdebusscher · 2019-08-13T12:53:32Z

...ver/docker/src/main/java/fish/payara/docker/instance/admin/CreateDockerContainerCommand.java

+        createContainer(adminCommandContext, actionReport, node, server, dasHost, dasPort);
+    }
+
+    private void pullImage(AdminCommandContext adminCommandContext, ActionReport actionReport, Node node) {


adminCommandContext and actionReport unused parameters. Can thus be removed for private method.

Is it correct that pull Image uses the same Docker URL endpoint as createContainer.

Should it not be the images/create URL?

rdebusscher · 2019-08-13T12:55:19Z

...ver/docker/src/main/java/fish/payara/docker/instance/admin/CreateDockerContainerCommand.java

+    private WebTarget createWebTarget(Node node) {
+        Client client = ClientBuilder.newClient();
+        WebTarget webTarget = null;
+        if (Boolean.valueOf(node.getUseTls())) {


Boolean.parseBoolean does not create first a Boolean which needs to 'boxed'

rdebusscher · 2019-08-13T13:17:11Z

...ver/docker/src/main/java/fish/payara/docker/instance/admin/CreateDockerContainerCommand.java

+
+                    // Set the Docker container ID either to the ID of the container, or to the instance name if the the
+                    // ID can't be obtained
+                    if (dockerContainerId != null && !dockerContainerId.equals("")) {


Maybe use String.isEmpty()? like

!dockerContainerId.trim().isEmpty())

rdebusscher

LGTM

Pandrex247 · 2019-08-13T16:05:55Z

Jenkins test please

jGauravGupta · 2019-08-19T11:13:46Z

...ver/docker/src/main/java/fish/payara/docker/instance/admin/CreateDockerContainerCommand.java

+                JsonObject jsonResponse = response.readEntity(JsonObject.class);
+
+                if (jsonResponse != null) {
+                    String dockerContainerId = jsonResponse.getString("Id");


Id key defined in lower case on StopDockerContainerCommand @RestParam(name="id") annotation.

This is referencing the "Id" field from the Docker Rest API, not our own.
The Docker Rest API has the capital :)
https://docs.docker.com/engine/api/v1.39/#operation/ContainerCreate

AlanRoth

Apart from Guarav's comment and little bit of pedantry from me, LGTM

AlanRoth · 2019-08-19T15:32:33Z

...ver/docker/src/main/java/fish/payara/docker/instance/admin/CreateDockerContainerCommand.java

+    private WebTarget createWebTarget(Node node, String endpoint) {
+        Client client = ClientBuilder.newClient();
+        WebTarget webTarget = null;
+        if (Boolean.parseBoolean(node.getUseTls())) {


Correct me if I'm wrong but could you do something like this to reduce redundency? just an idea I had, feel free to disregard;

boolean isHttps = Boolean.parseBoolean(node.getUseTls()) if(endpoint.startsWith("/")) { webTarget = client.target(isHttps ? "https://" : "http://" + node.getNodeHost() + ":" + node.getDockerPort() + endpoint); } else { webTarget = client.target(isHttps ? "https://" : "http://" + node.getNodeHost() + ":" + node.getDockerPort() + "/" + endpoint); }

I'll give it a test - if I'm going with ternary form I might as well go whole hog with something like the following:

WebTarget webTarget = client.target((Boolean.parseBoolean(node.getUseTls()) ? "https://" : "http://") + node.getNodeHost() + ":" + node.getDockerPort() + (endpoint.startsWith("/") ? endpoint : "/" + endpoint));

That works too, and still somewhat readable :P

…-3988-Dynamic-Docker-Nodes

Pandrex247 · 2019-08-20T10:14:29Z

Jenkins test please

Pandrex247 added 5 commits August 6, 2019 14:40

PAYARA-3988 Pull image if not present, start based on container ID

07bbaab

rather than name, and make sure autonaming works from console

PAYARA-3988 Update Docker image and script to allow dynamic instances.

ccc53ef

Add stop container command so that instances can be started, stopped, and started again without an error. Create commands to get and set container ID of instances.

PAYARA-3988 Add missing copyright

cecc3fe

PAYARA-3988 More missing copyright, and add descriptions of each class

ee06b94

PAYARA-3988 More missing copyright, and clean unused imports

1c5a216

Pandrex247 marked this pull request as ready for review August 7, 2019 14:57

rdebusscher self-requested a review August 13, 2019 12:52

rdebusscher suggested changes Aug 13, 2019

View reviewed changes

PAYARA-3988 Review comments

6e92c68

rdebusscher self-requested a review August 13, 2019 14:59

rdebusscher approved these changes Aug 13, 2019

View reviewed changes

jGauravGupta reviewed Aug 19, 2019

View reviewed changes

AlanRoth approved these changes Aug 19, 2019

View reviewed changes

Merge branch 'master' of https://github.com/payara/Payara into PAYARA…

eba18ac

…-3988-Dynamic-Docker-Nodes

jGauravGupta approved these changes Aug 20, 2019

View reviewed changes

PAYARA-3988 Simplify if block

c8604a3

Pandrex247 merged commit 44ecadc into payara:master Aug 21, 2019

Pandrex247 deleted the PAYARA-3988-Dynamic-Docker-Nodes branch August 21, 2019 09:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PAYARA-3988 Dynamic Docker Node Instances #4130

PAYARA-3988 Dynamic Docker Node Instances #4130

Pandrex247 commented Aug 6, 2019

Pandrex247 commented Aug 7, 2019

rdebusscher left a comment

rdebusscher Aug 13, 2019

rdebusscher Aug 13, 2019

rdebusscher Aug 13, 2019

rdebusscher Aug 13, 2019

rdebusscher left a comment

Pandrex247 commented Aug 13, 2019

jGauravGupta Aug 19, 2019

Pandrex247 Aug 19, 2019

AlanRoth left a comment

AlanRoth Aug 19, 2019 •

edited

Loading

Pandrex247 Aug 19, 2019

AlanRoth Aug 20, 2019

Pandrex247 commented Aug 20, 2019

PAYARA-3988 Dynamic Docker Node Instances #4130

PAYARA-3988 Dynamic Docker Node Instances #4130

Conversation

Pandrex247 commented Aug 6, 2019

Examples

Asadmin Examples

Manual/Scaling Examples

Pandrex247 commented Aug 7, 2019

rdebusscher left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rdebusscher left a comment

Choose a reason for hiding this comment

Pandrex247 commented Aug 13, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AlanRoth left a comment

Choose a reason for hiding this comment

AlanRoth Aug 19, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Pandrex247 commented Aug 20, 2019

AlanRoth Aug 19, 2019 •

edited

Loading