Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADMNAPI-989] Add read me file and fix docker file warnings #96

Merged
merged 6 commits into from
Apr 1, 2024
Merged

[ADMNAPI-989] Add read me file and fix docker file warnings #96

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
c6f7a67
Add read me file and fix docker file warnings
CSR2017 Mar 27, 2024
1f6c251
Update single tenant/ multi tenant folder details
CSR2017 Mar 28, 2024
756f4fd
Add space
CSR2017 Mar 28, 2024
9a66db0
Re-arranging the layers on docker files for enabling caching
CSR2017 Mar 29, 2024
a7a4656
Update the SDK base layer
stephenfuqua Mar 30, 2024
5bbf212
Merge pull request #97 from Ed-Fi-Alliance-OSS/ADMINAPI-989-sf
CSR2017 Apr 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitattributes
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Linux scripts always need to have lf
*.sh text eol=lf
40 changes: 40 additions & 0 deletions Docker/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Docker files details

## Development environment

1. dev.Dockerfile

The purpose of this file is to facilitate the setup of Admin API docker image in
the development environment, allowing for local testing with latest changes.
It utilizes the assets and dlls from "Docker\Application\EdFi.Ods.AdminApi"
folder.

2. dbadmin.Dockerfile

Purpose of this file to setup the EdFi_Admin database image which includes Admin
API specific tables. It utilizes database artifacts located at
"Docker\Application\EdFi.Ods.AdminApi\Artifacts\PgSql\Structure\Admin".

> [!NOTE]
> The "EdFi.Ods.AdminApi" application folder and "Nuget.config" file will be
> copied over, either manually or through the execution of a script(`build.ps1
> -Command "CopyToDockerContext"`), to the "Application" folder
> within the "Docker" directory.

## Non-development environments

1. api.Dockerfile

File for setting up Admin API docker image with assets and dlls sourced from
"EdFi.Suite3.ODS.AdminApi" nuget package(from
https://pkgs.dev.azure.com/ed-fi-alliance).

2. \Docker\Settings\DB-Admin\pgsql\Dockerfile

This file to setup the EdFi_Admin database image which includes Admin API
specific tables. The database artifacts will be downloaded from
"EdFi.Suite3.ODS.AdminApi" nuget package(from
https://pkgs.dev.azure.com/ed-fi-alliance).

For detailed instructions on setting up docker containers, please refer
[docker.md](../docs/docker.md).
14 changes: 10 additions & 4 deletions Docker/Settings/DB-Admin/pgsql/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# The Ed-Fi Alliance licenses this file to you under the Apache License, Version 2.0.
# See the LICENSE and NOTICES files in the project root for more information.

FROM edfialliance/ods-api-db-admin:7.1
FROM edfialliance/ods-api-db-admin:7.1 as base
LABEL maintainer="Ed-Fi Alliance, LLC and Contributors <techsupport@ed-fi.org>"

ENV POSTGRES_USER=${POSTGRES_USER}
Expand All @@ -12,18 +12,24 @@ ENV POSTGRES_DB=postgres

ARG VERSION=latest

USER root
COPY run-adminapi-migrations.sh /docker-entrypoint-initdb.d/3-run-adminapi-migrations.sh
RUN apk --no-cache add dos2unix=~7.4 unzip=~6.0
USER postgres

FROM base as setup

USER root
RUN apk --no-cache add dos2unix=~7.4 unzip=~6.0 && \
wget -nv -O /tmp/EdFi_AdminApi_Scripts.zip https://pkgs.dev.azure.com/ed-fi-alliance/Ed-Fi-Alliance-OSS/_apis/packaging/feeds/EdFi/nuget/packages/EdFi.Suite3.ODS.AdminApi/versions/${VERSION}/content && \
RUN wget -nv -O /tmp/EdFi_AdminApi_Scripts.zip "https://pkgs.dev.azure.com/ed-fi-alliance/Ed-Fi-Alliance-OSS/_apis/packaging/feeds/EdFi/nuget/packages/EdFi.Suite3.ODS.AdminApi/versions/${VERSION}/content" && \
unzip /tmp/EdFi_AdminApi_Scripts.zip AdminApi/Artifacts/PgSql/Structure/Admin/* -d /tmp/AdminApiScripts/ && \
cp -r /tmp/AdminApiScripts/AdminApi/Artifacts/PgSql/Structure/Admin/. /tmp/AdminApiScripts/PgSql/ && \
rm -f /tmp/EdFi_AdminApi_Scripts.zip && \
rm -r /tmp/AdminApiScripts/AdminApi && \
dos2unix /docker-entrypoint-initdb.d/3-run-adminapi-migrations.sh && \
dos2unix /tmp/AdminApiScripts/PgSql/* && \
chmod -R 777 /tmp/AdminApiScripts/PgSql
USER postgres

EXPOSE 5432
USER postgres

CMD ["docker-entrypoint.sh", "postgres"]
4 changes: 2 additions & 2 deletions Docker/Settings/ssl/generate-certificate.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,5 @@
set -e
set -x

openssl dhparam -out dhparam.pem 4096
openssl req -subj '//CN=localhost' -x509 -newkey rsa:4096 -nodes -keyout server.key -out server.crt -days 365 -addext "subjectAltName = DNS:nginx"
openssl dhparam -out dhparam.pem 2048
openssl req -subj '/CN=localhost' -x509 -newkey rsa:2048 -nodes -keyout server.key -out server.crt -days 365 -addext "subjectAltName = DNS:nginx"
26 changes: 17 additions & 9 deletions Docker/api.Dockerfile
View file
Open in desktop
CSR2017 marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -4,24 +4,29 @@
# See the LICENSE and NOTICES files in the project root for more information.

#tag 6.0-alpine
FROM mcr.microsoft.com/dotnet/aspnet@sha256:201cedd60cb295b2ebea7184561a45c5c0ee337e37300ea0f25cff5a2c762538
LABEL maintainer="Ed-Fi Alliance, LLC and Contributors <techsupport@ed-fi.org>"
ARG VERSION=latest
FROM mcr.microsoft.com/dotnet/aspnet@sha256:201cedd60cb295b2ebea7184561a45c5c0ee337e37300ea0f25cff5a2c762538 AS base
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really thought that I saw the api.nuget.org connectivity problem being resolved a few days ago when I just moved the curl command. Like you, however, this didn't work for me 😞 .

But... I did find an easy solution: just update the base image to the .NET 8 version! Even though .NET 8 upgrade isn't finished, the SDK for .NET 8 will build a project that targets .NET 6.

Changing to the following solved the problem for me. This is slightly newer than the version we have in Admin API 1 - this version must have been released just a few days ago.

FROM mcr.microsoft.com/dotnet/sdk:8.0.203-alpine3.18@sha256:2a8dca3af111071172b1629c12eefaeca0d6c2954887c4489195771c9e90833c as buildBase

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A PR into your branch: #97

ARG DB=pgsql

RUN apk --no-cache add curl=~8 unzip=~6 dos2unix=~7 bash=~5 gettext=~0 jq=~1 icu=~72 && \
if [ "$DB" = "pgsql" ]; then apk --no-cache add postgresql13-client=~13; fi && \
addgroup -S edfi && adduser -S edfi -G edfi

FROM base as build
LABEL maintainer="Ed-Fi Alliance, LLC and Contributors <techsupport@ed-fi.org>"

# Alpine image does not contain Globalization Cultures library so we need to install ICU library to get for LINQ expression to work
# Disable the globaliztion invariant mode (set in base image)
ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false
ARG VERSION=latest

WORKDIR /app

COPY Settings/"${DB}"/appsettings.template.json /app/appsettings.template.json
COPY Settings/"${DB}"/run.sh /app/run.sh
COPY --chmod=600 Settings/"${DB}"/appsettings.template.json /app/appsettings.template.json
COPY --chmod=500 Settings/"${DB}"/run.sh /app/run.sh
COPY Settings/"${DB}"/log4net.config /app/log4net.txt

RUN apk --no-cache add curl=~8 unzip=~6 dos2unix=~7 bash=~5 gettext=~0 jq=~1 icu=~72 && \
if [ "$DB" = "pgsql" ]; then apk --no-cache add postgresql13-client=~13; fi && \
wget -nv -O /app/AdminApi.zip https://pkgs.dev.azure.com/ed-fi-alliance/Ed-Fi-Alliance-OSS/_apis/packaging/feeds/EdFi/nuget/packages/EdFi.Suite3.ODS.AdminApi/versions/${VERSION}/content && \
RUN umask 0077 && \
wget -nv -O /app/AdminApi.zip "https://pkgs.dev.azure.com/ed-fi-alliance/Ed-Fi-Alliance-OSS/_apis/packaging/feeds/EdFi/nuget/packages/EdFi.Suite3.ODS.AdminApi/versions/${VERSION}/content" && \
unzip /app/AdminApi.zip AdminApi/* -d /app/ && \
cp -r /app/AdminApi/. /app/ && \
rm -f /app/AdminApi.zip && \
Expand All @@ -31,8 +36,11 @@ RUN apk --no-cache add curl=~8 unzip=~6 dos2unix=~7 bash=~5 gettext=~0 jq=~1 icu
dos2unix /app/*.sh && \
dos2unix /app/log4net.config && \
chmod 700 /app/*.sh -- ** && \
rm -f /app/*.exe
rm -f /app/*.exe && \
apk del unzip dos2unix curl && \
chown -R edfi /app

EXPOSE 443
USER edfi

ENTRYPOINT [ "/app/run.sh" ]
10 changes: 7 additions & 3 deletions Docker/dbadmin.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,20 @@ ENV POSTGRES_USER=${POSTGRES_USER}
ENV POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
ENV POSTGRES_DB=postgres

USER root
COPY Settings/DB-Admin/pgsql/run-adminapi-migrations.sh /docker-entrypoint-initdb.d/3-run-adminapi-migrations.sh
COPY Application/EdFi.Ods.AdminApi/Artifacts/PgSql/Structure/Admin/ /tmp/AdminApiScripts/PgSql
COPY Settings/dev/adminapi-test-seeddata.sql /tmp/AdminApiScripts/PgSql/adminapi-test-seeddata.sql

RUN apk --no-cache add dos2unix=~7.4 unzip=~6.0
USER postgres

USER root
RUN apk --no-cache add dos2unix=~7.4 unzip=~6.0 && \
dos2unix /docker-entrypoint-initdb.d/3-run-adminapi-migrations.sh && \
RUN dos2unix /docker-entrypoint-initdb.d/3-run-adminapi-migrations.sh && \
dos2unix /tmp/AdminApiScripts/PgSql/* && \
chmod -R 777 /tmp/AdminApiScripts/PgSql/*
USER postgres

EXPOSE 5432
USER postgres

CMD ["docker-entrypoint.sh", "postgres"]
35 changes: 22 additions & 13 deletions Docker/dev.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,45 +3,54 @@
# The Ed-Fi Alliance licenses this file to you under the Apache License, Version 2.0.
# See the LICENSE and NOTICES files in the project root for more information.

# First layer uses a dotnet/sdk image to build the Admin API from source code
# Second layer uses the dotnet/aspnet image to run the built code
# First two layers use a dotnet/sdk image to build the Admin API from source
# code. The next two layers use the dotnet/aspnet image to run the built code.
# The extra layers in the middle support caching of base layers.

FROM mcr.microsoft.com/dotnet/sdk:8.0.203-alpine3.18@sha256:2a8dca3af111071172b1629c12eefaeca0d6c2954887c4489195771c9e90833c as buildBase

#tag sdk:6.0-alpine
FROM mcr.microsoft.com/dotnet/sdk@sha256:c1a73b72c02e7b837e9a93030d545bc4181193e1bab1033364ed2d00986d78ff AS build
WORKDIR /source
FROM buildbase as build

Check failure on line 12 in Docker/dev.Dockerfile

View workflow job for this annotation

GitHub Actions / docker-analysis (development, Docker/dev.Dockerfile, local) 

DL3006 warning: Always tag the version of an image explicitly
RUN apk --no-cache add curl=~8

FROM build AS publish
WORKDIR /source
COPY Application/NuGet.Config EdFi.Ods.AdminApi/
COPY Application/EdFi.Ods.AdminApi EdFi.Ods.AdminApi/

WORKDIR /source/EdFi.Ods.AdminApi
RUN dotnet restore && dotnet build -c Release

FROM build AS publish
RUN dotnet publish -c Release /p:EnvironmentName=Production --no-build -o /app/EdFi.Ods.AdminApi

# TODO: update to .NET 8, will be handled in AdminAPI-983
#tag aspnet:6.0-alpine
FROM mcr.microsoft.com/dotnet/aspnet@sha256:201cedd60cb295b2ebea7184561a45c5c0ee337e37300ea0f25cff5a2c762538
FROM mcr.microsoft.com/dotnet/aspnet@sha256:201cedd60cb295b2ebea7184561a45c5c0ee337e37300ea0f25cff5a2c762538 AS runtimebase

FROM runtimebase AS runtime
RUN apk --no-cache add curl=~8 dos2unix=~7 bash=~5 gettext=~0 icu=~72 && \
addgroup -S edfi && adduser -S edfi -G edfi

FROM runtime AS setup
LABEL maintainer="Ed-Fi Alliance, LLC and Contributors <techsupport@ed-fi.org>"
# Alpine image does not contain Globalization Cultures library so we need to install ICU library to get for LINQ expression to work
# Disable the globaliztion invariant mode (set in base image)
ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false
ENV ASPNETCORE_ENVIRONMENT Production
ENV ASPNETCORE_ENVIRONMENT=Production

WORKDIR /app
COPY --from=publish /app/EdFi.Ods.AdminApi .

COPY Settings/dev/run.sh /app/run.sh
COPY --chmod=500 Settings/dev/run.sh /app/run.sh
COPY Settings/dev/log4net.config /app/log4net.txt

RUN apk --no-cache add curl=~8 dos2unix=~7 bash=~5 gettext=~0 icu=~72 && \
cp /app/log4net.txt /app/log4net.config && \
RUN cp /app/log4net.txt /app/log4net.config && \
dos2unix /app/*.json && \
dos2unix /app/*.sh && \
dos2unix /app/log4net.config && \
chmod 700 /app/*.sh -- **
chmod 500 /app/*.sh -- ** && \
chown -R edfi /app

EXPOSE 443
USER edfi
WORKDIR /app

ENTRYPOINT ["/app/run.sh"]
24 changes: 14 additions & 10 deletions docs/docker.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ graph LR
E --> F[(EdFi_Admin)]
E --> G[(EdFi_Security)]
D --> H[(PGBouncer)]
H --> I[(EdFi_ODS{0})]
H --> I[(EdFi_ODS)]
D --> E

subgraph pb-admin
Expand Down Expand Up @@ -57,36 +57,40 @@ style I fill:#fff
directory:

```bash
cd Application/EdFi.Ods.AdminApi/Docker/ssl
cd Docker/Settings/ssl
bash ./generate-certificate.sh
```

2. Copy and customize the `.env.example` file. Importantly, be sure to change
the encryption key.
the encryption key. In a Bash prompt, generate a random key thusly: `openssl
rand -base64 32`.

```shell
cd ../../Compose/pgsql
cd Docker/Compose/pgsql
cp .env.example .env
code .env
```
> [!NOTE]
> The .env file is a shared resource that can be referenced by both the
> "MultiTenant" and "SingleTenant" compose files.

3. Build local containers (optional step; next step will run the build implicitly)

```shell
docker compose -f compose-build-dev.yml build
docker compose -f SingleTenant/compose-build-dev.yml build
```

4. Start containers

```shell
docker compose -f compose-build-dev.yml up -d
docker compose -f SingleTenant/compose-build-dev.yml up -d
```

5. Inspect containers

```shell
# List processes
docker compose -f compose-build-dev.yml ps
docker compose -f SingleTenant/compose-build-dev.yml ps

# Check status of the AdminAPI
curl -k https://localhost/adminapi
Expand Down Expand Up @@ -160,10 +164,10 @@ Instructions are similar to the Local Development and Pre-Built Binaries setups

Tenants details can be configured on appsettings.dockertemplate.json file.

For local development and testing, use `compose-build-dev-multi-tenant.yml`.
For testing pre-built binaries, use `compose-build-binaries-multi-tenant.yml`.
For local development and testing, use `MultiTenant/compose-build-dev-multi-tenant.yml`.
For testing pre-built binaries, use `MultiTenant/compose-build-binaries-multi-tenant.yml`.

## Admin Api and Ed-Fi ODS / API docker containers

Please refer [DOCKER DEPLOYMENT](https://techdocs.ed-fi.org/display/EDFITOOLS/Docker+Deployment) for
installing and configuring Admin Api along with Ed-Fi ODS / API on Docker containers for testing.
installing and configuring Admin Api along with Ed-Fi ODS / API on Docker containers for testing.
Loading