-
Notifications
You must be signed in to change notification settings - Fork 20
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/rework dockerfile feedback #292
Changes from all commits
1488dba
dd09d16
001be1f
79f1d60
078fab6
29f2247
457bacb
4bc2cc5
d4f804f
1c2a77e
781ca15
7487744
8c6a6b9
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,17 +1,18 @@ | ||
.vs | ||
.vscode | ||
bin | ||
[_][Bb]uild | ||
Builds/* | ||
Testing/* | ||
.idea | ||
cmake-build-debug | ||
cmake-build-release | ||
build | ||
external/* | ||
!external/CMakeLists.txt | ||
# Ignore everything by default | ||
* | ||
|
||
README.md | ||
CHANGELOG.md | ||
CONTRIBUTING.md | ||
LICENSE | ||
# First-order allow exception for select directories | ||
!/.clang-format | ||
!/.githooks | ||
!/CMakeLists.txt | ||
!/Dockerfile | ||
!/docs | ||
!/extensions | ||
!/extensions.repos | ||
!external/CMakeLists.txt | ||
!/include | ||
!/ros2_standalone | ||
!/setup.py | ||
!/src | ||
!/test | ||
!/tools |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
version: 2 | ||
updates: | ||
- package-ecosystem: "docker" | ||
directory: "/" | ||
schedule: | ||
interval: "daily" | ||
commit-message: | ||
prefix: "🐳 " | ||
- package-ecosystem: "github-actions" | ||
directory: "/" | ||
schedule: | ||
interval: "daily" | ||
commit-message: | ||
prefix: "🛠️ " |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,21 +1,59 @@ | ||
ARG BASE_IMAGE=nvidia/cuda:11.7.1-devel-ubuntu22.04 | ||
ARG BASE_IMAGE=base | ||
# Stage from full image tag name for dependabot detection | ||
FROM nvidia/cuda:11.7.1-devel-ubuntu22.04 as base | ||
|
||
################################################################################ | ||
# MARK: prepper - prep rgl dependencies | ||
################################################################################ | ||
FROM $BASE_IMAGE as prepper | ||
ARG DEBIAN_FRONTEND=noninteractive | ||
|
||
FROM ${BASE_IMAGE} as rgl-core | ||
RUN apt update | ||
RUN apt install -y \ | ||
git \ | ||
cmake \ | ||
python3 | ||
# Edit apt config for caching and update once | ||
RUN mv /etc/apt/apt.conf.d/docker-clean /etc/apt/ && \ | ||
echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' \ | ||
> /etc/apt/apt.conf.d/keep-cache && \ | ||
apt-get update | ||
|
||
# Install bootstrap tools for install scripts | ||
RUN --mount=type=cache,sharing=locked,target=/var/cache/apt \ | ||
apt-get install -y --no-install-recommends \ | ||
cmake \ | ||
git \ | ||
python3 \ | ||
sudo | ||
|
||
# Set working directory using standard opt path | ||
WORKDIR /opt/rgl | ||
|
||
# Copy only dependencies definition files | ||
COPY ./setup.py . | ||
|
||
FROM rgl-core AS build | ||
# install dependencies while caching apt downloads | ||
# RUN --mount=type=cache,sharing=locked,target=/var/cache/apt \ | ||
# ./setup.py --install-deps-only | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This Even better would be splitting up the setup and compilation scripts, such that changes in the compilation script wouldn't ivalidate the build cache of consecutive layers that could install GB of dependencies. Or- a package manifest such as a There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good idea, thank you for the suggestion. |
||
|
||
################################################################################ | ||
# MARK: builder - build rgl binaries | ||
################################################################################ | ||
FROM prepper AS builder | ||
ARG OptiX_INSTALL_DIR=/optix | ||
|
||
WORKDIR /code | ||
COPY . . | ||
# Disable DNS lookups | ||
RUN cat /etc/nsswitch.conf && \ | ||
sed -e 's#hosts:\(.*\)dns\(.*\)#hosts:\1\2#g' -i.bak /etc/nsswitch.conf && \ | ||
cat /etc/nsswitch.conf | ||
|
||
# Copy rest of source tree | ||
COPY . . | ||
RUN --mount=type=bind,from=optix,target=${OptiX_INSTALL_DIR} \ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could we not require an internet connection at build time?
This is a best practice for bolstering deterministic builds, that many other projects try to follow: There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, it is possible to configure a build environment that does not require an internet connection at build time. We can stop using There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Preferably, this arg could call a separate script file, that Docker could then COPY and invoke directly instead of this massive setup.py file, without the need to COPY any other miscellaneous sources that could otherwise change from tasks unrelated to dependency updates or dependency management. |
||
./setup.py | ||
|
||
FROM scratch AS export-binaries | ||
COPY --from=build /code/build/libRobotecGPULidar.so / | ||
# Restore DNS lookups | ||
RUN mv /etc/nsswitch.conf.bak /etc/nsswitch.conf && \ | ||
cat /etc/nsswitch.conf | ||
|
||
################################################################################ | ||
# MARK: exporter - export rgl binaries | ||
################################################################################ | ||
FROM scratch AS exporter | ||
COPY --from=builder /code/build/libRobotecGPULidar.so / |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@msz-rai , I forgot to remove this line. The Dockerfile can of course be ignored, so that changes it itself don't break the build cache after the
COPY . .
directive.