Implement Docker/OCI layer index file support

[plaird]

Spring Boot 2.3 added the ability to supply a layer index file that optimizes the writing of the Spring Boot jar for container use cases. It splits the jar into multiple layers such that the base layers contains the mostly static stuff (dependencies, etc) while the top layer contains the stuff that you change a lot more (the app code). This reduces time to download the image, as the base layers are already cached on your host.

This is important because Spring Boot jars often drag in hundreds of dependency jars. It is not uncommon to have 400MB+ Spring Boot jars. Written as a single layer, a single code line change will require hosts that use the image to pull down a full 400MB again.

Overview: https://spring.io/blog/2020/01/27/creating-docker-images-with-spring-boot-2-3-0-m1
(Unfortunately the code snippets in the article lost their line endings, so the examples are hard to read)

Details:

• https://docs.spring.io/spring-boot/specification/executable-jar/nested-jars.html#appendix.executable-jar.nested-jars.layer-index
• https://docs.spring.io/spring-boot/maven-plugin/packaging.html#packaging.repackage-goal.parameter-details.layers

Exploration:
Explore these features with a simple spring boot app and Maven:

• Generate a spring boot app: initializr Add spring-web and maybe some other dependencies.
• Build the springboot app normally with Maven (no layers) mvn clean install
• Run it locally: java -jar target/myapp.jar
• Create a Dockerfile that launches it in the naive way
• Go back and use the Maven tools to create a layered jar. The doc shows how to do it but I haven't tried it so I am hand waving from here on down...
• Build using Maven.
• Run the app again to show that it still works using java -Djarmode=layertools -jar target/myapp.jar
• Create a Dockerfile that packages up, again using the doc as a guide.
• Prove that you have multiple layers: docker inspect yourimage:latest
• Make a small change to the Spring Boot app code (which is written to the top most layer) and rebuild, only the top layer should have changed. You will see the SHA change in docker inspect.

Implementation:
After we understand the features and options, we can decide how to implement for the Bazel build system. Guessing at the Bazel implementation, I think it will go something like this:

Much like the classpath index file #33, we need to add a springboot() rule attribute (e.g. layer_index_file) so the user can supply the layer index file. I don't think we should do anything magical here (i.e. auto-generating these files). Just allow the user to specify an existing workspace file via the attribute in the rule. This would be optional as the default layout seems sufficient for most use cases. From there, we will have to figure out the fancy work of splitting the written outputs into the layers. From there, we will need to provide examples of rules_oci building the image in layers.

This work may require changing the springboot rule signature in some incompatible way, so assigning to a major release for now.

[chrismgrayftsinc]

One thing that might be nice would be to have a springboot_image rule similar to the rules in bazelbuild/rules_docker (or even building on them). I have been using those rules and they work well -- it just takes a long time to upload a new image even when very little has changed.

[plaird]

Thanks for feedback. We also use rules_docker to package up our springboot apps, and have had good success with it.

I am a little worried about building too much into the rules_spring repo, so I am not sold on springboot_image. But it is worth looking into it for sure. We are finding that external rules that bring in other dependencies (e.g. if rules_spring depended on rules_docker) are a pain to deal with. Bazel Federation just didn't take off unfortunately.

What I have on the books right now is Issue #94 which is to just provide an example of how to build the docker image by combining springboot rule with rules_docker rules. But, once we implement this layer support, there may be a useful reason to also provide a springboot_image rule so we should consider it.

[Marcus-Rosti]

@plaird do you have an example of that just for yourself? is it as easy adding the target and main to a container_image?

[chrismgrayftsinc]

@Marcus-Rosti we use

container_image(
    name = "foo-service-image",
    base = "@java_base//image",
    cmd = [
        "java",
        "-jar",
        "foo-service.jar",
    ],
    files = ["foo-service.jar"],
    ports = [
        "8080",
        "8081",
    ],
)

where @java-base is

container_pull(
    name = "java_base",
    digest = "sha256:0ee3964303ce4c8d8b371457ebeca218c74b240b6740bb9f82b0606e9b56a7f5",
    registry = "index.docker.io",
    repository = "library/openjdk",
)

[Marcus-Rosti]

and @chrismgrayftsinc foo-service.jar is the output of

java_library(
    name = "mrosti_server",
    srcs = ["App.java"],
    deps = [
        "//my/service...",
    ] + springboot_deps,
)

springboot(
    name = "mrosti_server",
    boot_app_class = "com.marcus.rosti.App",
    java_library = "mrosti_server",
)

container_image(
...
    files = ["foo-service.jar"], # output of above?
)

[chrismgrayftsinc]

In this case, it looks like it would be the output of the springboot rule (which would need a different name from your library). Assuming you named the springboot rule "mrosti-service", then "foo-service.jar" would be replaced by "mrosti-service.jar".

[Marcus-Rosti]

@chrismgrayftsinc that worked like a charm! thanks!

[plaird]

Thanks @Marcus-Rosti for the question, and @chrismgrayftsinc for providing the example. Internally we do some additional fancy pants stuff around custom entrypoints, but what Chris provided covers the 95% case.

[chrismgrayftsinc]

@plaird I went ahead and started a small repo that does this. I heavily modified the springboot.bzl file to do it. I will need to clean it up, but for now it appears to work for our use case. Check it out at https://github.com/chrismgrayftsinc/rules_spring_image

[plaird]

@chrismgrayftsinc I am looking forward to look at your solution. This week is crazy, but I am hoping to get to this soon. Thanks!

[plaird]

@chrismgrayftsinc I think you are underselling what you have done - it is more than just layer file support. You also have done a bunch of the needful of migrating off of bash #3 which is the oldest open issue in the project. :) I have a bunch of things in flight right now outside of rules_spring, but I will try to prioritize finding a way to merge your work. I was thinking of using Java based on singlejar for the new rule impl in #3, but if we can get there with Starlark all the better.