Open in app

Sign in

Write

Sign in

Hosting Django Applications with MySQL, Nginx, and Kubernetes on Digital Ocean Kubernetes

Tejendra Saradhi
9 min read4 days ago

In this blog, we’ll look into how we can containerize a Django Application into Docker and then deploy it using Kubernetes on the Digital Ocean. The steps described in this blog can be used on other hosting solutions.

I’m quite familiar with the Django web framework but to save my time I forked an opensource project in GitHub. My fork’s static file setup is different from the original project so it can work with the Nginx. Additionally, I’ve used the MySQL database.

Overview of the blog:

  • Modify the application to make it work with Nginx and MySQL, especially in a Docker environment.
  • Containerize the application.
  • Push the backend image to the GitHub Container Registry (GHCR).
  • Create Deployment, Persistent Volume Claims (PVC), and Service files for Kubernetes.
  • Modify the code to support the Kubernetes environment.

GitHub - 0xtejas/StudyBud

Contribute to 0xtejas/StudyBud development by creating an account on GitHub.

github.com

Dockerize the Project

We’ll have to check how the static files are configured on our project as we have planned to serve them using the Nginx, we have to store them in a folder. I replaced the following snippet from the settings.py in the Django project.

STATICFILES_DIRS = [
    BASE_DIR / 'static'
]

We use the STATIC_ROOT, folder as the place where all the static files for the project and the Django admin are stored. Django admin’s static files can be collected using the command python manage.py collectstatic.

STATICFILES_DIRS = []
STATIC_ROOT = BASE_DIR / 'static'

Make sure to create two folders, one for the Nginx and another for the MySQL configuration files.

Under the Nginx folder, we will add the nginx.conf and Dockerfile

nginx.conf

server {
    listen 80;
    server_name localhost;

    location / {
        proxy_pass http://backend:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }

    location /static/ {
        alias /usr/src/app/static/;
    }

    location /media/ {
        alias /usr/src/app/media/;
    }

}

Dockerfile

FROM nginx:1.19.0-alpine

RUN rm /etc/nginx/conf.d/default.conf

COPY nginx.conf /etc/nginx/conf.d

WORKDIR /usr/src/app

We pull the Nginx image, remove the default.conf, and add our nginx.conf

We can notice that the proxy_pass uses the container name backend to forward our traffic to the Django application. The other two locations/static/ and /media/ are served by the Nginx server. The files in those folders are shared by Nginx and Backend (Django) containers using the Docker Volumes.

Under the MySQL folder create an .env file with the following contents

MYSQL_ROOT_PASSWORD=tejas
MYSQL_DATABASE=studydb

Note: Feel free to change the values as per your project requirements.

Constructing a docker-compose File for the Project

As per our plan, we require three services:

  • Nginx
  • MySQL
  • Python / Django

Let us define the services in the file.

MySQL Service Definition

  mysql:
    container_name: mysql
    image: mysql
    restart: unless-stopped
    ports:
      - 3306:3306
      - 33060:33060
    volumes:
      - mysql:/var/lib/mysql
    env_file:
      - ./mysql/.env

We require the container name to be mysql as we use the host’s name as mysql in Django’s settings.py.

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'studydb',
        'USER': 'root',
        'PASSWORD': 'tejas',
        'HOST': 'mysql',
        'PORT': '3306',
    }
}

We expose ports 3306 and 33060 so the other container can communicate. The volume mysql mounts at /var/lib/mysql. Finally, the environment file that has the initialization values is imported.

Backend (Django) Service Definition

backend:
    container_name: backend
    image: ghcr.io/0xtejas/studybud:latest
    build: .
    restart: unless-stopped
    expose:
      - 8000
    volumes:
      - static:/usr/src/app/static
      - media:/usr/src/app/media
    command: gunicorn studybud.wsgi:application --bind 0.0.0.0:8000 -w 2 --access-logfile - --error-logfile -
    depends_on:
      - mysql

The image here we use is built with the name ghcr.io/0xtejas/studybud:latest as I’m building the images and pushing them to the GitHub Container Registry. We can use any container registry.

Why do we need a container registry?

Kubernetes pulls the images from the registry and runs the images.

We are exposing port 8000, mounting the required volumes for the Nginx container, and command to start the application using Gunicorn.

Nginx Service Definition

  nginx:
    container_name: nginx
    image: nginx:custom
    build: ./nginx
    restart: unless-stopped
    ports:
      - 8000:80
    volumes:
      - static:/usr/src/app/static
      - media:/usr/src/app/media
    depends_on:
      - backend

We create a custom Nginx and build the Nginx using the Dockerfile, exposing port 80 by binding the local 8000 and mounting the volumes static and media.

Complete docker-compose File

services:
  mysql:
    container_name: mysql
    image: mysql
    restart: unless-stopped
    ports:
      - 3306:3306
      - 33060:33060
    volumes:
      - mysql:/var/lib/mysql
    env_file:
      - ./mysql/.env
  backend:
    container_name: backend
    image: ghcr.io/0xtejas/studybud:latest
    build: .
    restart: unless-stopped
    expose:
      - 8000
    volumes:
      - static:/usr/src/app/static
      - media:/usr/src/app/media
    command: gunicorn studybud.wsgi:application --bind 0.0.0.0:8000 -w 2 --access-logfile - --error-logfile -
    depends_on:
      - mysql
  nginx:
    container_name: nginx
    image: nginx:custom
    build: ./nginx
    restart: unless-stopped
    ports:
      - 8000:80
    volumes:
      - static:/usr/src/app/static
      - media:/usr/src/app/media
    depends_on:
      - backend
volumes:
  mysql:
    external: true
  static:
    external: true
  media:
    external: true

Run the following commands to create the volumes

docker volume create mysql
docker volume create static
docker volume create media

We are doing this as we have mentioned externally as true in the docker-compose. It means that it is managed externally or manually by us and not by the docker-compose.

Generating Personal Access Token (PAT) on GitHub to push, pull, and delete packages

Log into the GitHub > Settings > Developers Settings > Personal access tokens > Tokens (classic).

Log into the GHCR using the docker and push the tagged image.

docker login ghcr.io -u USERNAME --password-stdin
docker push ghcr.io/0xtejas/studybud:latest

Once the image is pushed we should see them under packages in the GitHub profile. By default the package is private hence we require this key for later when we pull the images in the Kubernetes.

We can see there are two tags one for the k8s and the other for the docker itself.

If our application runs correctly in the Docker format, we are ready to jump into Kubernetes.

Kubernetes Cluster Creation

I created the K8 cluster with two nodes and without High Availability (HA).

We will not use an external database as we’d like to have it within the cluster.

You can use my referral code to claim the 200 USD credits for the 60 days.

DigitalOcean | Cloud Infrastructure for Developers

An ocean of simple, scalable cloud solutions.

m.do.co

Once the cluster is created, we can use the instructions shown in the platform to configure our kubectl.

Setting up Project for Kubernetes

Let us start writing some Kubernetes files that are required to deploy our application. Before we do, let us change the tag name in the Docker Compose so it is built with a different tag.

  backend:
    container_name: backend
    image: ghcr.io/0xtejas/studybud:k8-latest
    build: .
    restart: unless-stopped
    expose:
      - 8000
    volumes:
      - static:/usr/src/app/static
      - media:/usr/src/app/media
    command: gunicorn studybud.wsgi:application --bind 0.0.0.0:8000 -w 2 --access-logfile - --error-logfile -
    depends_on:
      - mysql

I have stored all the Kubernetes files in the folder k8s

Deployment Definition File

apiVersion: apps/v1
kind: Deployment
metadata:
  name: studybud
  labels:
    app: studybud
spec:
  replicas: 1
  selector:
    matchLabels:
      app: studybud
  template:
    metadata:
      labels:
        app: studybud
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        resources:
          requests:
            cpu: 200m
            memory: 200Mi
          limits:
            cpu: 400m
            memory: 400Mi
        volumeMounts:
        - name: static
          mountPath: /var/www/html/static
        - name: media
          mountPath: /usr/src/app/media
        - name: nginx-config
          mountPath: /etc/nginx/conf.d
      - name: studybud
        imagePullPolicy: Always
        image: ghcr.io/0xtejas/studybud:k8-latest
        ports:
        - containerPort: 8000
        command: 
          - sh
          - -c
          - |
            ./entrypoint.sh
            cp -r /usr/src/app/static/ /var/www/html/
            ln -s /usr/src/app/static/ /var/www/html/
            gunicorn studybud.wsgi:application --bind 0.0.0.0:8000 -w 2 --access-logfile - --error-logfile -
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
          limits:
            cpu: 200m
            memory: 200Mi
        volumeMounts:
        - name: static
          mountPath: /var/www/html/static
        - name: media
          mountPath: /usr/src/app/media
      - name: mysql
        image: mysql:latest
        ports:
        - containerPort: 3306
        - containerPort: 33060
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-secret
              key: MYSQL_ROOT_PASSWORD
        - name: MYSQL_DATABASE
          valueFrom:
            secretKeyRef:
              name: mysql-db
              key: MYSQL_DATABASE
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
          limits:
            cpu: 1500m
            memory: 512Mi
        volumeMounts:
        - name: mysql-pvc
          mountPath: /var/lib/mysql
      imagePullSecrets:
      - name: ghcr-credentials
      volumes:
      - name: static
        persistentVolumeClaim:
          claimName: static-pvc
      - name: media
        persistentVolumeClaim:
          claimName: media-pvc
      - name: mysql-pvc
        persistentVolumeClaim:
          claimName: mysql-pvc
      - name: nginx-config
        configMap:
          name: nginx-config
          name: nginx-config

To come up with the definition file above, I had to go through several trials and errors because there were things that didn’t work as I had assumed. I’ll explain the assumptions and the reasons why it isn’t like that later on.

For the most part, the definition is similar to the docker-compose file, we have used Configmaps, Secrets, PVCs, and Resources.

Nginx Container Explained

- name: nginx
        image: nginx:latest
        resources:
          requests:
            cpu: 200m
            memory: 200Mi
          limits:
            cpu: 400m
            memory: 400Mi
        volumeMounts:
        - name: static
          mountPath: /var/www/html/static
        - name: media
          mountPath: /usr/src/app/media
        - name: nginx-config
          mountPath: /etc/nginx/conf.d

We are pulling the latest Nginx image, defining resource requests and limits, and the volumeMountswhich are nothing but PVC. If we notice carefully, unlike docker-compose we have set the static path to be /var/www/html/static, why? I’ll explain to you when I explain the studybud (backend)’s definition.

Studybud Container Explained

- name: studybud
  imagePullPolicy: Always
  image: ghcr.io/0xtejas/studybud:k8-latest
  ports:
  - containerPort: 8000
  command: 
    - sh
    - -c
    - |
      ./entrypoint.sh
      cp -r /usr/src/app/static/ /var/www/html/
      ln -s /usr/src/app/static/ /var/www/html/
      gunicorn studybud.wsgi:application --bind 0.0.0.0:8000 -w 2 --access-logfile - --error-logfile -
  resources:
    requests:
      cpu: 100m
      memory: 100Mi
    limits:
      cpu: 200m
      memory: 200Mi
  volumeMounts:
  - name: static
    mountPath: /var/www/html/static
  - name: media
    mountPath: /usr/src/app/media

This is too similar to the docker-compose file, but we copy the files in the static folder to the PVC mounted in the /var/www/html/static I had to do this as creating a PVC on an existing folder will overwrite the folder with the PVC contents, due to which I lost the files that are already in the static folder in the Docker Image.

The next assumption that I had was when a docker image is pulled and executed by Kubernetes, it executes the entrypoint.sh and the Gunincorn to start the application. No, it doesn’t. In docker, it is executed by the docker-compose as it is defined. If we don’t define the command in the deployment file, it won’t execute the application.

MySQL Container Explained

- name: mysql
        image: mysql:latest
        ports:
        - containerPort: 3306
        - containerPort: 33060
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-secret
              key: MYSQL_ROOT_PASSWORD
        - name: MYSQL_DATABASE
          valueFrom:
            secretKeyRef:
              name: mysql-db
              key: MYSQL_DATABASE
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
          limits:
            cpu: 1500m
            memory: 512Mi
        volumeMounts:
        - name: mysql-pvc
          mountPath: /var/lib/mysql

We’ll store the secrets in the Kubernetes secrets (The value is not encrypted but encoded, it is encoded with base64).

Execute the following commands to create the secrets for MySQL

kubectl create secret generic mysql-secret --from-literal=MYSQL_ROOT_PASSWORD=$(echo -n 'dGVqYXM=' | base64 --decode)

kubectl create secret generic mysql-db --from-literal=MYSQL_DATABASE=$(echo -n 'c3R1ZHlkYg==' | base64 --decode)

dGVqYXM= and c3R1ZHlkYg== are base64 encoded values.

Volumes in Deployment File

imagePullSecrets:
      - name: ghcr-credentials
      volumes:
      - name: static
        persistentVolumeClaim:
          claimName: static-pvc
      - name: media
        persistentVolumeClaim:
          claimName: media-pvc
      - name: mysql-pvc
        persistentVolumeClaim:
          claimName: mysql-pvc
      - name: nginx-config
        configMap:
          name: nginx-config

You can notice that we use a secret to pull images from the registry. To create the secret run the following command.

kubectl create secret docker-registry ghcr-credentials --docker-server=ghcr.io --docker-username=<GH_USERNAME> --docker-password=<CLASSIC_TOKEN> --docker-email=<YOUR_EMAIL>

We also have to create ConfigMap for the Nginx.conf

kubectl create cm nginx-config --from-file=nginx.conf

Persistent Volume Claims Definition

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: static-pvc
spec:
  accessModes:
    - ReadWriteOnce 
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: media-pvc
spec:
  accessModes:
    - ReadWriteOnce 
  resources:
    requests:
      storage: 1Gi

Requesting storage of 1Gi with ReadWriteOnce access on the PVC. This will create a Digital Ocean Block storage.

We now have to define services so that containers can communicate with one another. Also, by using a service we can expose our web application.

MySQL Service Definition

apiVersion: v1
kind: Service
metadata:
  name: mysql
spec:
  selector:
    app: studybud
  ports:
    - name: mysql
      protocol: TCP
      port: 3306
      targetPort: 3306
    - name: mysql-ssl
      protocol: TCP
      port: 33060
      targetPort: 33060
  type: ClusterIP

It is a ClusterIP type, and it exposes ports 3306 and 33060 ports.

StudyBud Service Definition

apiVersion: v1
kind: Service
metadata:
  name: studybud-svc
spec:
  selector:
    app: studybud
  ports:
    - protocol: TCP
      name: studybud-server
      port: 8000
      targetPort: 8000
  type: ClusterIP

I’m exposing the port 8080 of the Django application.

Nginx Service Definition — LoadBalancer

apiVersion: v1
kind: Service
metadata:
  name: nginx-svc
spec:
  selector:
    app: studybud
  ports:
    - protocol: TCP
      name: nginx-server
      port: 80
  type: LoadBalancer

It finally requests the Public IP address and exposes our application on the port.

Once I had deployed it manually, I tried using ArgoCD and it worked without any issues. The following diagram is generated using the ArgoCD.

Kuberenetes
Django

--

--

Tejendra Saradhi

Written by Tejendra Saradhi

18 Followers

Music Producer | Cyber Security Enthusiast

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams