Docker configuration for Django+Vuejs+Gunicorn+Nginx lead to 502 Bad Gateway
Context
I'm developing an application that will have to be deployed once per customer (it's not a multi-tenant app that all my customers would use).
Ultimately, it will be deployed on AWS with services like RDS (DB), S3 (storage) and ECS (container service).
You can see how I imagined it with the image below if it can help to answer me:
My application is working well locally without Docker.
But I want to use a dockerized version of it locally before trying to deploy it on AWS, because I want to be able to run it locally with Docker before trying to do it on AWS (...).
Since all my customers would have their own instance (and customer1 could have a different version than customer2 at some time), I thought about a single container with everything needed, as you can see on the image : the Django application, the Vue built files, gunicorn and nginx.
Question 1 : Is it a good idea to do it like this ? Or should I use a docker-compose thing with multiple services (backend (django) & nginx). Would it lead to multiple containers ?
Problem
To do so, I did the following configuration :
Dockerfile:
FROM node:lts-alpine as build-frontend-stage
WORKDIR /frontend
COPY ./frontend/package*.json /frontend/
RUN npm install
COPY ./frontend .
RUN npm run build
FROM python:3.8.10-slim as build-backend-stage
RUN apt-get update
RUN apt-get install --yes --no-install-recommends \
g++ \
libpq-dev \
nginx
WORKDIR /backend
RUN pip install --upgrade pip
COPY ./backend/requirements.txt /backend
RUN pip install -r requirements.txt
COPY ./backend .
COPY --from=build-frontend-stage /frontend/dist/static /backend/static
COPY --from=build-frontend-stage /frontend/dist/index.html /backend/static
COPY ./docker-entrypoint.sh /backend
RUN ["chmod", "+x", "/backend/docker-entrypoint.sh"]
RUN rm -rf /etc/nginx/sites-available/default
RUN rm -rf /etc/nginx/sites-enabled/default
COPY ./nginx/nginx.config /etc/nginx/sites-available/
RUN ln -s /etc/nginx/sites-available/nginx.config /etc/nginx/sites-enabled
ENTRYPOINT ["bash", "/backend/docker-entrypoint.sh"]
This leads to a container with following structure :
ls -al
drwxr-xr-x 1 root root 4096 Oct 21 14:21 .
drwxr-xr-x 1 root root 4096 Oct 22 08:56 ..
drwxrwxr-x 1 root root 4096 Oct 22 08:56 backend
-rwxrwxr-x 1 root root 552 Oct 21 14:20 docker-entrypoint.sh
-rwxrwxr-x 1 root root 638 Oct 5 12:37 manage.py
-rw-rw-r-- 1 root root 115 Oct 5 10:05 requirements.txt
drwxr-xr-x 1 root root 4096 Oct 21 14:09 static
where the static folder has :
ls -al static
drwxr-xr-x 1 root root 4096 Oct 21 14:09 .
drwxr-xr-x 1 root root 4096 Oct 21 14:21 ..
drwxr-xr-x 2 root root 4096 Oct 18 14:47 css
drwxr-xr-x 2 root root 4096 Oct 18 14:47 fonts
-rw-r--r-- 1 root root 773 Oct 18 14:47 index.html
drwxr-xr-x 2 root root 4096 Oct 18 14:47 js
I have a nginx config like this :
upstream app_server {
server unix:gunicorn.sock fail_timeout=0;
}
server {
listen 80;
root /backend/static;
index index.html;
include /etc/nginx/mime.types;
location / {
proxy_pass http://app_server;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
}
}
Question 2 : I use a gunicorn.sock thing. When I do not, and use something like 0.0.0.0:8000 or whatever, I usually can't even reach nginx, it reaches gunicorn directly, don't know why...
My Docker-compose is like so :
version: "3.8"
services:
web:
network_mode: host
env_file:
- local.env
build: .
ports:
- "8000:8000"
my local.env file has env variables such as :
DB_HOST=127.0.0.1
DB_NAME=db_name
DB_PASSWORD=db_password
DB_PORT=5432
DB_USER=db_user
SECRET_KEY=some_django_secret_key
and finally my entrypoint is :
#!/bin/bash
DJANGO_SETTINGS_MODULE=backend.settings.production
DJANGO_WSGI_MODULE=backend.wsgi
SOCK_FILE=gunicorn.sock
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
echo Migrating...
python manage.py migrate --no-input
service nginx start
service nginx status
echo Sarting gunicorn...
gunicorn ${DJANGO_WSGI_MODULE}:application \
--workers 3 \
--bind=unix:$SOCK_FILE \
--log-file=-
To start it all, I do :
docker-compose -f ./docker-compose-dev.yml up
which builds everything correctly and gunicorn starts :
web_1 | [2021-10-22 09:06:50 +0000] [53] [INFO] Starting gunicorn 20.0.4
web_1 | [2021-10-22 09:06:50 +0000] [53] [INFO] Listening at: unix:gunicorn.sock (53)
web_1 | [2021-10-22 09:06:50 +0000] [53] [INFO] Using worker: sync
web_1 | [2021-10-22 09:06:50 +0000] [55] [INFO] Booting worker with pid: 55
web_1 | [2021-10-22 09:06:50 +0000] [56] [INFO] Booting worker with pid: 56
web_1 | [2021-10-22 09:06:50 +0000] [57] [INFO] Booting worker with pid: 57
But, I can't get anything except a 502 Bad Gateway from nginx when trying to reach 127.0.0.1 in my browser:
and I can't figure out why. I guess it must be a confusion of some sort from me..
Question 3 : Can you help me figure out what is the error in my configuration or giving me some guidance on how to configure such a deployment (for local here, not AWS), or some guidance on how to find the source of the error ?
In advance thank you for reading. I'm really bad at those devOps things and any help will be appreciated.
You do not want to run everything into a single container. A container should be doing just one thing. You have a couple of options. Run one container per module (nginx, app, etc) and group them into a single ECS task. The task is a deployment and scaling unit in ECS that you can control with an ECS service. So you could have 1 ECS service per customer and start with 1 task backing it. If you need more "power" ECS will scale your single task to 2 tasks (doubling all your containers). And so on.
The second approach (architecturally better but perhaps more expensive) is to have a task per each container and so a total of 3 independent tasks wrapped in 3 independent services. Each service then can scale on its own (e.g. if you need more capacity for NGINX only the NGINX task will be scaled in the NGINX ECS service. If you adopt this approach you will need to have 3 services (plus the db) per customer.
This is a demo application I am using for these experiments that kind of mimics your app. My docker-compose file for this application looks like this:
version: "3.0"
services:
yelb-ui:
image: mreferre/yelb-ui:0.7
depends_on:
- yelb-appserver
ports:
- 80:80
networks:
- yelb-network
yelb-appserver:
image: mreferre/yelb-appserver:0.5
depends_on:
- redis-server
- yelb-db
networks:
- yelb-network
redis-server:
image: redis:4.0.2
networks:
- yelb-network
# uncomment the following lines if you want to persist redis data across deployments
#volumes:
# - redisdata:/data
yelb-db:
image: mreferre/yelb-db:0.5
networks:
- yelb-network
# uncomment the following lines if you want to persist postgres data across deployments
#volumes:
# - postgresqldata:/var/lib/postgresql/data
networks:
yelb-network:
driver: bridge # a user defined bridge is required; the default bridge network doesn't support name resolution
That's how your application will run locally. If you want to run it on ECS you can either use a tool like Copilot or, if you want to stick using docker compose you could leverage the new docker compose integration with ECS and docker compose up your app into ECS directly.
Both of these solutions will implement the second pattern I have described (1 container per task). If you want the "all containers in a task" pattern you should craft a native CloudFormation template.