Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run conda clean -a -y after conda install ... in Dockerfile #2

Merged
merged 6 commits into from
Oct 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- Version pinned for Ubuntu, conda and conda packages.
- Run `conda clean -a -y` in Dockerfile reducing image size.
- Added [`usecases/run_mixer.ipynb`](usecases/run_mixer.ipynb) Jupyter notebook for simplifying Slurm job submissions for lists of traits.
- Include ``module purge`` in Slurm job scripts to avoid conflicts with system modules.

### Changed

- Use Ubuntu 20.04 LTS as base image for Docker and Singularity containers.

### Fixed

- Fixed incorrect job dependency bug in [`usecases/run_mixer.ipynb`](usecases/run_mixer.ipynb)
Expand Down
4 changes: 2 additions & 2 deletions singularity/mixer.sif
Git LFS file not shown
33 changes: 23 additions & 10 deletions src/dockerfiles/mixer/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,22 +1,34 @@
FROM 'ubuntu:18.04'
FROM ubuntu:20.04

ENV TZ=Europe
ENV DEBIAN_FRONTEND noninteractive

WORKDIR /tmp
COPY /scripts/apt_get_essential.sh .
RUN chmod +x apt_get_essential.sh
RUN bash apt_get_essential.sh
RUN chmod +x apt_get_essential.sh && \
bash apt_get_essential.sh

WORKDIR /tmp
COPY /scripts/install_miniconda3.sh .
RUN chmod +x install_miniconda3.sh
RUN bash install_miniconda3.sh

RUN conda install python=3.8 --yes
RUN conda install -c conda-forge jupyterlab --yes
RUN conda install numpy scipy pandas statsmodels matplotlib seaborn --yes
RUN conda install -c conda-forge matplotlib-venn numdifftools --yes
RUN chmod +x install_miniconda3.sh && \
bash install_miniconda3.sh

RUN conda install conda=23.9.0 && \
conda config --add channels conda-forge && \
conda config --set channel_priority strict && \
conda install \
jupyterlab=4.0.7 \
matplotlib=3.7.3 \
matplotlib-venn=0.11.9 \
numpy=1.24.4 \
numdifftools=0.9.41 \
pandas=2.0.3 \
python=3.8 \
scipy=1.10.1 \
seaborn=0.13.0 \
statsmodels=0.14.0 \
--yes && \
conda clean -a -y

WORKDIR /tools/plink
COPY /scripts/install_plink.sh /tmp
Expand All @@ -39,5 +51,6 @@ COPY /scripts/install_plink2.sh /tmp
RUN chmod +x /tmp/install_plink2.sh
RUN bash /tmp/install_plink2.sh


WORKDIR /tools

70 changes: 37 additions & 33 deletions src/scripts/apt_get_essential.sh
Original file line number Diff line number Diff line change
@@ -1,36 +1,40 @@
#!/bin/sh
apt-get update && apt-get install -y ca-certificates && update-ca-certificates
apt-get update && apt-get install -y \
ca-certificates=20230311ubuntu0.20.04.1 && \
update-ca-certificates

# (!) Keep the list below sorted (!)
apt-get update && apt-get install -y --no-install-recommends apt-utils \
autoconf \
build-essential \
bzip2 \
cmake \
curl \
dos2unix \
gfortran \
git \
less \
libatlas-base-dev \
libcurl4 \
libcurl4-openssl-dev \
libgomp1 \
libgsl0-dev \
libnss3 \
libpcre2-dev \
libquadmath0 \
libxt-dev \
make \
pandoc \
pandoc-citeproc \
parallel \
perl \
tar \
tofrodos \
unzip \
vim \
wget \
zlib1g-dev \
&& \
rm -rf /var/lib/apt/lists/*

apt-get install -y --no-install-recommends \
apt-utils=2.0.9 \
autoconf=2.69-11.1 \
build-essential=12.8ubuntu1.1 \
bzip2=1.0.8-2 \
cmake=3.16.3-1ubuntu1.20.04.1 \
curl=7.68.0-1ubuntu2.20 \
dos2unix=7.4.0-2 \
gfortran=4:9.3.0-1ubuntu2 \
git=1:2.25.1-1ubuntu3.11 \
less=551-1ubuntu0.1 \
libatlas-base-dev=3.10.3-8ubuntu7 \
libcurl4=7.68.0-1ubuntu2.20 \
libcurl4-openssl-dev=7.68.0-1ubuntu2.20 \
libgomp1=10.5.0-1ubuntu1~20.04 \
libgsl-dev=2.5+dfsg-6build1 \
libnss3=2:3.49.1-1ubuntu1.9 \
libpcre2-dev=10.34-7ubuntu0.1 \
libquadmath0=10.5.0-1ubuntu1~20.04 \
libxt-dev=1:1.1.5-1 \
make=4.2.1-1.2 \
pandoc=2.5-3build2 \
pandoc-citeproc=0.15.0.1-1build4 \
parallel=20161222-1.1 \
perl=5.30.0-9ubuntu0.4 \
tar=1.30+dfsg-7ubuntu0.20.04.3 \
tofrodos=1.7.13+ds-4 \
unzip=6.0-25ubuntu1.1 \
vim=2:8.1.2269-1ubuntu5.18 \
wget=1.20.3-1ubuntu2 \
zlib1g-dev=1:1.2.11.dfsg-2ubuntu1.5

rm -rf /var/lib/apt/lists/*
2 changes: 1 addition & 1 deletion src/scripts/install_miniconda3.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
curl -sSL https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -o /tmp/miniconda3.sh \
curl -sSL https://repo.anaconda.com/miniconda/Miniconda3-py38_23.5.1-0-Linux-x86_64.sh -o /tmp/miniconda3.sh \
&& mkdir /root/.conda \
&& bash /tmp/miniconda3.sh -bfp /usr/local \
&& rm -rf /tmp/miniconda3.sh
Expand Down
1 change: 1 addition & 0 deletions src/scripts/install_mixer.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,6 @@ tar -xzvf boost_1_69_0.tar.gz && cd boost_1_69_0
./b2 --clean && ./b2 --j12 -a
cd /tools
git clone --recurse-submodules -j8 https://github.com/precimed/mixer.git
git reset --hard 2b5ad6b2d9ab5b92d8dd24ec106338af61e27a80
mkdir mixer/src/build && cd mixer/src/build && cmake .. -DBoost_NO_BOOST_CMAKE=ON -DBOOST_ROOT=/tools/boost_1_69_0 && make -j16 bgmg