Django IO bound performance drop with gunicorn + gevent
I have been working on an IO bound (only simple queries to database for getting info or updating it) application written in Django. As application is mostly db and redis queries, I decided to use gunicorn with async gevent worker class, but the weird part is that eventhough I'm running gunicorn with gevent (also monkey patched db for that purpose), I'm not seeing any performance gain by it, in fact, both requests/s and response time has dropped.
Steps taken
To do so, I have done the following:
- Installed greenlet, gevent & psycogreen as instructed in official docs.
- Patched postgres with
psycogreen.gevent.patch_psycopg(tried wsgi.py, settings.py and post_fork in gunicorn) but to no avail. - Tried running gevent's
monkey.patch_allmanually, also no use.
This is my gunicorn.config.py:
import gunicorn
gunicorn.SERVER_SOFTWARE = ""
gunicorn.SERVER = ""
bind = "0.0.0.0:8000"
worker_class = 'gevent'
worker_connections = 1000 # Tested with different values
keepalive = 2 # Tested with different values
workers = 10 # Tested with different values
loglevel = 'info'
access_log_format = 'sport %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"'
def post_fork(server, worker): # also tried pre_fork and on_starting
from psycogreen.gevent import patch_psycopg
patch_psycopg()
Result
But the app only got slower, also note that I am using django_prometheus.db.backends.postgresql as db backend, django-redis (which uses gevent friendly redis-py) for cache and no notable other third party library.
Test method
To check and compare performance, I ran app with several different configs and ran a load test each time for one api endpoint (the api simply queries database for a list of objects and serializes it). As for specs, I'm running app in my Intel© Core™ i7-9750HF CPU @ 2.60GHz × 6 laptop with 16GB memory. For load test, I am using "hey" like this:
hey -n 1000 -c 100 -t 0 <api>
And this is the result for sync and async tests:
sync class result
Requests/sec: 101.4230
99% in 1.6029 secs
gevent class result
Requests/sec: 87.9190
99% in 5.2386 secs
I would appreciate if someone has any idea what could be wrong with my config or application.