Kubernetes Setup ( - with Traefik + Longhorn ) Example

[stevenlafl]

Introduction

Here's an example Kubernetes setup.

Note: I do not attempt to encrypt anything other then the master credential store. Therefore SECRET_KEY_BASE and others are not present, as I am not entirely certain what they are actually for in the render.yaml file.

• There is also obviously a way better way to store credentials than opaque.

Paranoia note: I used root in initContainer step to avoid modifying any Dockerfile from the repository as the goal was to get this working in minimal steps. If this project wants to support a secure kubernetes setup, here is a start and it will require more thorough consideration.

• The initContainer step only copies files, generates and loads keys and sets permissions. The actual runtime for the container is under rails like the original production setup. IF credential:edit had an exploit and it was used during reading, or if the PVC were modified by a non-rails user, this may be potentials for compromise.

Instructions

Clone repo, build and push docker image. I found the render-production target works fine. In my testing, I am using commit 7691a03

git clone https://github.com/allyourbot/hostedgpt.git
cd hostedgpt
docker buildx build -t ${IMAGE_TAG} --target render-production --platform=linux/amd64,linux/arm64 .
docker push ${IMAGE_TAG}

In the below:

• Replace "${MASTER_KEY}" with the output of:

  openssl rand -hex 16 | base64
  

• Replace "${HOSTNAME}" with the URI of your service.
• Replace "${IMAGE_TAG}" with your image tag.
• Replace the secrets for POSTGRES and the corresponding connection string.

apiVersion: v1
kind: Namespace
metadata:
  name: hostedgpt
---
apiVersion: v1
kind: Secret
metadata:
  name: master-key-config
  namespace: hostedgpt
type: Opaque
data:
  master.key: ${MASTER_KEY}
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: rails-credentials
  namespace: hostedgpt
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Mi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
  namespace: hostedgpt
spec:
  selector:
    matchLabels:
      app: redis
  replicas: 1
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis
        ports:
        - containerPort: 6379
---
apiVersion: v1
kind: Service
metadata:
  name: redis
  namespace: hostedgpt
spec:
  ports:
  - port: 6379
    targetPort: 6379
  selector:
    app: redis
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-pvc
  namespace: hostedgpt
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
  namespace: hostedgpt
spec:
  selector:
    matchLabels:
      app: postgres
  replicas: 1
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:16
        env:
        - name: POSTGRES_USER
          value: "app"
        - name: POSTGRES_DB
          value: "app_development"
        - name: POSTGRES_PASSWORD
          value: "secret"
        - name: PGDATA
          value: "/var/lib/postgresql/data/pgdata"
        volumeMounts:
        - name: postgres-storage
          mountPath: /var/lib/postgresql/data
        ports:
        - containerPort: 5432
      volumes:
      - name: postgres-storage
        persistentVolumeClaim:
          claimName: postgres-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: postgres
  namespace: hostedgpt
spec:
  ports:
  - port: 5432
    targetPort: 5432
  selector:
    app: postgres
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hostedgpt
  namespace: hostedgpt
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hostedgpt
  template:
    metadata:
      labels:
        app: hostedgpt
    spec:
      initContainers:
      - name: generate-credentials
        securityContext:
          runAsUser: 0  # Running as root
        image: ${IMAGE_TAG}
        command: ["/bin/sh", "-c"]
        args:
        - |
          if [ -f /tmp/credentials.yml.enc ]; then
            cp /tmp/credentials.yml.enc /rails/config/credentials.yml.enc
          fi

          # Edit or create the credentials file
          bin/rails credentials:edit

          # Copy the config directory contents to our credential PVC and set perms for rails.
          cp -R /rails/config/* /tmp/
          chown -R rails:rails /tmp

          # Debug info
          ls -la /rails/config /tmp
        env:
        - name: EDITOR
          value: "true"  # Non-interactive dummy editor command
        - name: RAILS_MASTER_KEY
          valueFrom:
            secretKeyRef:
              name: master-key-config
              key: master.key
        volumeMounts:
        - name: credentials-volume
          mountPath: /tmp
      containers:
      - name: hostedgpt
        image: ${IMAGE_TAG}
        ports:
        - containerPort: 3000
        env:
        - name: REDIS_URL
          value: "redis://redis.hostedgpt.svc.cluster.local:6379/0"
        - name: DATABASE_URL
          value: "postgres://app:secret@postgres.hostedgpt.svc.cluster.local/app_development"
        - name: RAILS_ENV
          value: production
        - name: RAILS_MASTER_KEY
          valueFrom:
            secretKeyRef:
              name: master-key-config
              key: master.key
        volumeMounts:
        - name: credentials-volume
          mountPath: /rails/config
      - name: worker
        image: ${IMAGE_TAG}
        command: ["bin/rails", "solid_queue:start"]
        env:
        - name: REDIS_URL
          value: "redis://redis.hostedgpt.svc.cluster.local:6379/0"
        - name: DATABASE_URL
          value: "postgres://app:secret@postgres.hostedgpt.svc.cluster.local/app_development"
        - name: RAILS_ENV
          value: production
        - name: RAILS_MASTER_KEY
          valueFrom:
            secretKeyRef:
              name: master-key-config
              key: master.key
        volumeMounts:
        - name: credentials-volume
          mountPath: /rails/config
      volumes:
      - name: credentials-volume
        persistentVolumeClaim:
          claimName: rails-credentials
---
apiVersion: v1
kind: Service
metadata:
  name: hostedgpt
  namespace: hostedgpt
spec:
  ports:
  - port: 3000
    targetPort: 3000
  selector:
    app: hostedgpt
---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: hostedgpt-ingress
  namespace: hostedgpt
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`${HOSTNAME}`)
      kind: Rule
      services:
        - name: hostedgpt
          port: 3000
  tls: {}

[krschacht]

Thanks for sharing this! Kubernetes is one of those technologies I hear often, but I’ve never actually dug in so I don’t understand it. With regards to this HostedGPT project, what were you looking to solve that this led to this setup? I’m just trying to figure out where best to slot it in, mentally.

For example, we have support for Render and Fly, but those are obviously very specific hosts and people may want to run a machine in other places. My thinking has been that we should add support for Kamal soon since that’s much more versatile. I am moderately familiar with Kamal. What does Kubernetes help you do that’s different than Kamal? Or is it just an alternate tool to effectively do the same thing: automate deploys on any hardware?

[stevenlafl]

I run a 4-node ARM cluster out of 32GB Turing RK1s and an x86 proxmox cluster running two Docker-in-LXC containers and a pfSense VM for network security/routing among other full VMs. I chose Kubernetes because it:

1. Utilizes Infrastructure as Code
2. Effortlessly manages resources and routing across all my deployed apps.

This is only one part of a large network of IoT and other services I have running, and it's just a home lab setup.

I could convert this to Helm, which essentially simplifies Kubernetes deployments. That way it's kind-of a "one click" solution. I prefer actually seeing and controlling everything upfront, so I personally would not opt for that method, but it is certainly easier for some.

[krschacht]

Got it, you have a lot more context on in this area than I do. But you're basically describing how you've deployed HostedGPT on your home network for personal use? If you're away and aren't on your home network, are you still able to access your instance?

[stevenlafl]

Yes. Also, I -could-, and have, allowed outside access through CloudFront. Though for things that are only for personal use, e.g. not a portfolio website, I have a Wireguard container that tunnels traffic from my phone or laptop to my local network. I use it in cases where I'm on vacation in another country, so I can still get YouTube TV and Netflix without restrictions.