Able to run Dockerized Django app with supervisor locally, but timing out when deployed to Azure app services
So I have a Dockerized Django app running with uWSGI and NGINX, managed by supervisord (so both of them are run in the same container). It works fine and dandy when running it locally, but when I deploy it to a Azure web-app for containers, it fails to respond on the http warmup-request.
I've tried doing different kinds of adjustments regarding the ports on Azure/in the app, but nothing seems to be working.
The set-up I've been trying is:
- NGINX exposing the app in port 80, adding configuration in the Azure portal with PORT=80
- NGINX exposing the app in port 8000, adding configuration in the Azure portal with PORT=80 and WEBSITES_PORT=8000
- In both cases exposing both 80 and 8000 in the Dockerfile
- Increase the timeout-limit for the app to 1800 seconds
But I'm still receiving the same error, and I really feeling I don't know where to go from here. Can anyone help me figure this one out?
I'll post the logs and the relevant files below. As you can see in the logs, supervisor seems to be starting up, but not giving the response that Azure seems to need to start up the server.
Repository structure
myapp/
┣ .devcontainer/
┃ ┗ devcontainer.json
┣ docker/
┃ ┣ projectile/
┃ ┃ ┣ deploy/
┃ ┃ ┃ ┣ nginx-app.conf
┃ ┃ ┃ ┣ supervisor-app.conf
┃ ┃ ┃ ┣ uwsgi.ini
┃ ┃ ┃ ┗ uwsgi_params
┃ ┃ ┣ imagefile/
┃ ┃ ┃ ┣ migrations/
┃ ┃ ┃ ┃ ┗ __init__.py
┃ ┃ ┃ ┣ templates/
┃ ┃ ┃ ┃ ┗ index.html
┃ ┃ ┃ ┣ __init__.py
┃ ┃ ┃ ┣ admin.py
┃ ┃ ┃ ┣ apps.py
┃ ┃ ┃ ┣ exceptions.py
┃ ┃ ┃ ┣ models.py
┃ ┃ ┃ ┣ serializers.py
┃ ┃ ┃ ┣ tests.py
┃ ┃ ┃ ┣ urls.py
┃ ┃ ┃ ┗ views.py
┃ ┃ ┣ projectile/
┃ ┃ ┃ ┣ .env
┃ ┃ ┃ ┣ .env.example
┃ ┃ ┃ ┣ __init__.py
┃ ┃ ┃ ┣ asgi.py
┃ ┃ ┃ ┣ settings.py
┃ ┃ ┃ ┣ urls.py
┃ ┃ ┃ ┗ wsgi.py
┃ ┃ ┣ Dockerfile
┃ ┃ ┣ __init__.py
┃ ┃ ┣ app.sock
┃ ┃ ┣ db.sqlite3
┃ ┃ ┣ init_api.sh
┃ ┃ ┣ manage.py
┃ ┃ ┣ requirements.txt
┃ ┣ __init__.py
┃ ┗ docker-compose.yml
┣ .gitignore
┗ README.md
Dockerfile
FROM python:3.9.6
# Environment varialbes
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED=1
# Install Poppler (requirement from Pdf2Image)
RUN apt-get update
RUN apt-get install poppler-utils -y
# install required packages
RUN apt-get install -y supervisor
# update packages after adding nginx repository
RUN apt-get update
# install latest stable nginx
RUN apt-get install -y nginx
# install uwsgi now because it takes a little while
RUN pip install uwsgi
# install our code
ADD . /home/docker/code/
# setup all the configfiles
RUN echo "daemon off;" >> /etc/nginx/nginx.conf
RUN rm /etc/nginx/sites-enabled/default
RUN ln -s /home/docker/code/deploy/nginx-app.conf /etc/nginx/sites-enabled/
RUN ln -s /home/docker/code/deploy/supervisor-app.conf /etc/supervisor/conf.d/
# RUN pip install
# ------- ACTIVATE FOR EXTERNAL WITH GITHUB ACTIONS -------
#RUN pip install -r /home/docker/code/docker/projectile/requirements.txt
# --------- ACTIVATE FOR LOCAL TESTING ------------------
RUN pip install -r /home/docker/code/requirements.txt
EXPOSE 8000 80
CMD ["supervisord", "-n"]
nginx-app.conf
# mysite_nginx.conf
# the upstream component nginx needs to connect to
upstream django {
server unix:/home/docker/code/app.sock; # for a file socket
# server 127.0.0.1:8001; # for a web port socket (we'll use this first)
}
# configuration of the server
server {
# the port your site will be served on, default_server indicates that this server block
# is the block to use if no blocks match the server_name
listen 80 default_server;
# the domain name it will serve for
server_name xxx.azurewebsites.net www.xxx.azurewebsites.net localhost; # substitute your machine's IP address or FQDN
charset utf-8;
# max upload size
client_max_body_size 75M; # adjust to taste
# Django media
location /media {
alias /home/docker/persistent/media; # your Django project's media files - amend as required
}
location /static {
alias /home/docker/volatile/static; # your Django project's static files - amend as required
}
# Finally, send all non-media requests to the Django server.
location / {
uwsgi_pass django;
include /home/docker/code/deploy/uwsgi_params; # the uwsgi_params file you installed
}
}
# RUN echo "PWD is: $PWD"
# RUN echo $(ls)
# RUN echo $(ls home/)
supervisor-app.conf
[program:app-uwsgi]
command = /usr/local/bin/uwsgi --ini /home/docker/code/deploy/uwsgi.ini
[program:nginx-app]
command = /usr/sbin/nginx
uwsgi.ini
[uwsgi]
# this config will be loaded if nothing specific is specified
# load base config from below
ini = :base
# %d is the dir this configuration file is in
socket = /home/docker/code/app.sock
master = true
processes = 4
[base]
# chdir to the folder of this config file, plus app/website
chdir = /home/docker/code/
# load the module from wsgi.py, it is a python path from
# the directory above.
module=projectile.wsgi:application
# allow anyone to connect to the socket. This is very permissive
chmod-socket=666
Logs
Connecting...
2021-12-08T11:52:13 Welcome, you are now connected to log-streaming service.
Starting Log Tail -n 10 of existing logs ----
/home/LogFiles/__lastCheckTime.txt (https://myapplicationnginx2.scm.azurewebsites.net/api/vfs/LogFiles/__lastCheckTime.txt)12/08/2021 11:51:45
/home/LogFiles/kudu/trace/5e51430037b6-e0c14976-83b4-49ef-af6e-4ac48125345b.txt (https://myapplicationnginx2.scm.azurewebsites.net/api/vfs/LogFiles/kudu/trace/5e51430037b6-e0c14976-83b4-49ef-af6e-4ac48125345b.txt)
2021-12-07T16:55:15 Startup Request, url: /api/logs/docker, method: GET, type: request, pid: 67,1,6, ScmType: None
/home/LogFiles/kudu/trace/6daf861dee06-d50e1458-407c-4753-85c6-f26a86ba15f0.txt (https://myapplicationnginx2.scm.azurewebsites.net/api/vfs/LogFiles/kudu/trace/6daf861dee06-d50e1458-407c-4753-85c6-f26a86ba15f0.txt)
2021-12-08T08:48:31 Startup Request, url: /api/logstream/, method: GET, type: request, pid: 64,1,15, ScmType: None
/home/LogFiles/kudu/trace/7703cd1d0b17-cb46d37f-3be3-4fca-8033-fda23be96a73.txt (https://myapplicationnginx2.scm.azurewebsites.net/api/vfs/LogFiles/kudu/trace/7703cd1d0b17-cb46d37f-3be3-4fca-8033-fda23be96a73.txt)
2021-12-07T16:56:12 Startup Request, url: /api/vfs/site/wwwroot/?_=1638895916577, method: GET, type: request, pid: 67,1,14, ScmType: None
/home/LogFiles/kudu/trace/82d803e305dd-766fccef-b243-4109-ae49-7f9f592d6f4a.txt (https://myapplicationnginx2.scm.azurewebsites.net/api/vfs/LogFiles/kudu/trace/82d803e305dd-766fccef-b243-4109-ae49-7f9f592d6f4a.txt)
2021-12-07T16:54:47 Startup Request, url: /api/logs/docker, method: GET, type: request, pid: 67,1,4, ScmType: None
/home/LogFiles/kudu/trace/868bce7e3d08-45082c86-ddac-4bf7-a0a4-2d0cd2b7f128.txt (https://myapplicationnginx2.scm.azurewebsites.net/api/vfs/LogFiles/kudu/trace/868bce7e3d08-45082c86-ddac-4bf7-a0a4-2d0cd2b7f128.txt)
2021-12-07T17:37:45 Startup Request, url: /api/logstream/, method: GET, type: request, pid: 68,1,4, ScmType: None
/home/LogFiles/kudu/trace/8bf3f1ea4636-924d7fbc-3f5a-4e52-be4b-32a38eb29328.txt (https://myapplicationnginx2.scm.azurewebsites.net/api/vfs/LogFiles/kudu/trace/8bf3f1ea4636-924d7fbc-3f5a-4e52-be4b-32a38eb29328.txt)
2021-12-07T16:53:37 Startup Request, url: /api/vfs/site/wwwroot/?_=1638895916573, method: GET, type: request, pid: 70,1,4, ScmType: None
/home/LogFiles/kudu/trace/b175e56a17f5-a25d3e0a-42fb-4145-9f5e-4c6f3ccd6838.txt (https://myapplicationnginx2.scm.azurewebsites.net/api/vfs/LogFiles/kudu/trace/b175e56a17f5-a25d3e0a-42fb-4145-9f5e-4c6f3ccd6838.txt)
2021-12-08T11:52:00 Startup Request, url: /api/vfs/site/wwwroot/?_=1638955595033, method: GET, type: request, pid: 67,1,6, ScmType: None
/home/LogFiles/kudu/trace/e250e4c752b3-9896e30b-76b3-41cc-9928-510329a6546e.txt (https://myapplicationnginx2.scm.azurewebsites.net/api/vfs/LogFiles/kudu/trace/e250e4c752b3-9896e30b-76b3-41cc-9928-510329a6546e.txt)
2021-12-08T08:22:02 Startup Request, url: /api/vfs/LogFiles/2021_12_08_pl0sdlwk0006D6_default_docker.log, method: GET, type: request, pid: 68,1,14, ScmType: None
/home/LogFiles/2021_12_07_10-30-0-16_default_docker.log (https://myapplicationnginx2.scm.azurewebsites.net/api/vfs/LogFiles/2021_12_07_10-30-0-16_default_docker.log)
2021-12-07T17:00:29.809197561Z 2021-12-07 17:00:29,808 INFO supervisord started with pid 1
2021-12-07T17:26:12.274224620Z /usr/lib/python3/dist-packages/supervisor/options.py:474: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
2021-12-07T17:26:12.274375619Z self.warnings.warn(
2021-12-07T17:26:12.295254461Z 2021-12-07 17:26:12,294 CRIT Supervisor is running as root. Privileges were not dropped because no user is specified in the config file. If you intend to run as root, you can set user=root in the config file to avoid this message.
2021-12-07T17:26:12.295482359Z 2021-12-07 17:26:12,295 INFO Included extra file "/etc/supervisor/conf.d/supervisor-app.conf" during parsing
2021-12-07T17:26:12.311450038Z 2021-12-07 17:26:12,310 INFO RPC interface 'supervisor' initialized
2021-12-07T17:26:12.311613537Z 2021-12-07 17:26:12,311 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2021-12-07T17:26:12.312829428Z 2021-12-07 17:26:12,311 INFO supervisord started with pid 1
/home/LogFiles/2021_12_07_10-30-0-16_docker.log (https://myapplicationnginx2.scm.azurewebsites.net/api/vfs/LogFiles/2021_12_07_10-30-0-16_docker.log)
2021-12-07T17:28:26.028Z INFO - Waiting for response to warmup request for container myapplicationnginx2_0_211c5c93. Elapsed time = 136.9046693 sec
2021-12-07T17:28:41.139Z INFO - Waiting for response to warmup request for container myapplicationnginx2_0_211c5c93. Elapsed time = 152.0163041 sec
2021-12-07T17:28:56.330Z INFO - Waiting for response to warmup request for container myapplicationnginx2_0_211c5c93. Elapsed time = 167.207411 sec
2021-12-07T17:29:11.447Z INFO - Waiting for response to warmup request for container myapplicationnginx2_0_211c5c93. Elapsed time = 182.3245343 sec
2021-12-07T17:29:26.596Z INFO - Waiting for response to warmup request for container myapplicationnginx2_0_211c5c93. Elapsed time = 197.4735662 sec
2021-12-07T17:29:41.794Z INFO - Waiting for response to warmup request for container myapplicationnginx2_0_211c5c93. Elapsed time = 212.6707766 sec
2021-12-07T17:29:56.906Z INFO - Waiting for response to warmup request for container myapplicationnginx2_0_211c5c93. Elapsed time = 227.7834968 sec
2021-12-07T17:29:59.948Z ERROR - Container myapplicationnginx2_0_211c5c93 for site myapplicationnginx2 did not start within expected time limit. Elapsed time = 230.8253839 sec
2021-12-07T17:29:59.953Z ERROR - Container myapplicationnginx2_0_211c5c93 didn't respond to HTTP pings on port: 80, failing site start. See container logs for debugging.
2021-12-07T17:29:59.988Z INFO - Stopping site myapplicationnginx2 because it failed during startup.
/home/LogFiles/2021_12_07_pl0sdlwk0006D6_default_docker.log (https://myapplicationnginx2.scm.azurewebsites.net/api/vfs/LogFiles/2021_12_07_pl0sdlwk0006D6_default_docker.log)
2021-12-07T23:43:14.102947014Z 2021-12-07 23:43:14,102 INFO supervisord started with pid 1
2021-12-07T23:53:17.020714393Z /usr/lib/python3/dist-packages/supervisor/options.py:474: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
2021-12-07T23:53:17.020751795Z self.warnings.warn(
2021-12-07T23:53:17.023282812Z 2021-12-07 23:53:17,023 CRIT Supervisor is running as root. Privileges were not dropped because no user is specified in the config file. If you intend to run as root, you can set user=root in the config file to avoid this message.
2021-12-07T23:53:17.024082849Z 2021-12-07 23:53:17,023 INFO Included extra file "/etc/supervisor/conf.d/supervisor-app.conf" during parsing
2021-12-07T23:53:17.027976029Z 2021-12-07 23:53:17,027 INFO RPC interface 'supervisor' initialized
2021-12-07T23:53:17.028236541Z 2021-12-07 23:53:17,028 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2021-12-07T23:53:17.028634459Z 2021-12-07 23:53:17,028 INFO supervisord started with pid 1
/home/LogFiles/2021_12_07_pl0sdlwk0006D6_docker.log (https://myapplicationnginx2.scm.azurewebsites.net/api/vfs/LogFiles/2021_12_07_pl0sdlwk0006D6_docker.log)
2021-12-07T23:55:21.911Z INFO - Waiting for response to warmup request for container myapplicationnginx2_0_735c8819. Elapsed time = 124.7120412 sec
2021-12-07T23:55:36.986Z INFO - Waiting for response to warmup request for container myapplicationnginx2_0_735c8819. Elapsed time = 139.7866961 sec
2021-12-07T23:55:52.066Z INFO - Waiting for response to warmup request for container myapplicationnginx2_0_735c8819. Elapsed time = 154.8667009 sec
2021-12-07T23:56:07.149Z INFO - Waiting for response to warmup request for container myapplicationnginx2_0_735c8819. Elapsed time = 169.9498992 sec
2021-12-07T23:56:22.231Z INFO - Waiting for response to warmup request for container myapplicationnginx2_0_735c8819. Elapsed time = 185.0323056 sec
2021-12-07T23:56:37.304Z INFO - Waiting for response to warmup request for container myapplicationnginx2_0_735c8819. Elapsed time = 200.1047548 sec
2021-12-07T23:56:52.380Z INFO - Waiting for response to warmup request for container myapplicationnginx2_0_735c8819. Elapsed time = 215.1809152 sec
2021-12-07T23:57:07.308Z ERROR - Container myapplicationnginx2_0_735c8819 for site myapplicationnginx2 did not start within expected time limit. Elapsed time = 230.1090441 sec
2021-12-07T23:57:07.310Z ERROR - Container myapplicationnginx2_0_735c8819 didn't respond to HTTP pings on port: 8000, failing site start. See container logs for debugging.
2021-12-07T23:57:07.315Z INFO - Stopping site myapplicationnginx2 because it failed during startup.
/home/LogFiles/2021_12_08_pl0sdlwk0006D6_default_docker.log (https://myapplicationnginx2.scm.azurewebsites.net/api/vfs/LogFiles/2021_12_08_pl0sdlwk0006D6_default_docker.log)
2021-12-08T11:31:48.378748054Z 2021-12-08 11:31:48,378 INFO supervisord started with pid 1
2021-12-08T11:41:50.890532671Z /usr/lib/python3/dist-packages/supervisor/options.py:474: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
2021-12-08T11:41:50.890571572Z self.warnings.warn(
2021-12-08T11:41:50.893473802Z 2021-12-08 11:41:50,893 CRIT Supervisor is running as root. Privileges were not dropped because no user is specified in the config file. If you intend to run as root, you can set user=root in the config file to avoid this message.
2021-12-08T11:41:50.893865319Z 2021-12-08 11:41:50,893 INFO Included extra file "/etc/supervisor/conf.d/supervisor-app.conf" during parsing
2021-12-08T11:41:50.898646333Z 2021-12-08 11:41:50,898 INFO RPC interface 'supervisor' initialized
2021-12-08T11:41:50.898935246Z 2021-12-08 11:41:50,898 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2021-12-08T11:41:50.899576774Z 2021-12-08 11:41:50,899 INFO supervisord started with pid 1
/home/LogFiles/2021_12_08_pl0sdlwk0006D6_docker.log (https://myapplicationnginx2.scm.azurewebsites.net/api/vfs/LogFiles/2021_12_08_pl0sdlwk0006D6_docker.log)
2021-12-08T11:45:42.640Z INFO - Stopping site myapplicationnginx2 because it failed during startup.
2021-12-08T11:51:45.582Z INFO - Pulling image: myapplicationcontainerregistry.azurecr.io/myapplication_super:latest
2021-12-08T11:51:46.116Z INFO - latest Pulling from myapplication_super
2021-12-08T11:51:46.116Z INFO - Digest: sha256:83b42aa6390d7ec45b10aaedbeb1de55378d22a666986f66d6edfd786dcad076
2021-12-08T11:51:46.117Z INFO - Status: Image is up to date for myapplicationcontainerregistry.azurecr.io/myapplication_super:latest
2021-12-08T11:51:46.120Z INFO - Pull Image successful, Time taken: 0 Minutes and 0 Seconds
2021-12-08T11:51:46.125Z INFO - Starting container for site
2021-12-08T11:51:46.126Z INFO - docker run -d -p 80:80 --name myapplicationnginx2_0_dd7a4f47 -e PORT=80 -e WEBSITES_ENABLE_APP_SERVICE_STORAGE=false -e WEBSITE_SITE_NAME=myapplicationnginx2 -e WEBSITE_AUTH_ENABLED=False -e WEBSITE_ROLE_INSTANCE_ID=0 -e WEBSITE_HOSTNAME=myapplicationnginx2.azurewebsites.net -e WEBSITE_INSTANCE_ID=xxx -e HTTP_LOGGING_ENABLED=1 myapplicationcontainerregistry.azurecr.io/myapplication_super:latest
2021-12-08T11:51:50.233Z INFO - Initiating warmup request to container myapplicationnginx2_0_dd7a4f47 for site myapplicationnginx2
/home/LogFiles/webssh/.log (https://myapplicationnginx2.scm.azurewebsites.net/api/vfs/LogFiles/webssh/.log)
Ending Log Tail of existing logs ---
Starting Live Log Stream ---
2021-12-08T11:52:20.594Z INFO - Waiting for response to warmup request for container myapplicationnginx2_0_dd7a4f47. Elapsed time = 30.3606932 sec