Django Test Environment Runs Twice and Fails When Using if not IS_PRODUCTION and INJECT_FAKE_DATA
Description:
I'm encountering a strange issue in my Django test environment. When running my test suite, the environment appears to initialize twice, causing a specific test to pass the first time and fail on the second run. The problem seems to be related to the following conditional statement in my settings.py file:
if not IS_PRODUCTION and INJECT_FAKE_DATA:
# Code for injecting fake data
Environment Details:
- Django version: 5.1.1
- Python version: 3.12.1
- Test Framework: pytest with pytest-django
- Database: MongoDB for test
What Happens:
- When I run my tests, the environment is reloaded twice (as evidenced by logs).
- On the first initialization, everything works as expected, and the test passes.
- On the second run, the same test fails due to the environment not behaving as intended.
INFO:C:\path\to\settings.py changed, reloading.
INFO:INJECT_FAKE_DATA is True.
INFO:Fake data mode activated.
INFO:Watching for file changes with StatReloader
...
Performing system checks...
System check identified no issues (0 silenced).
...
First run: test passes.
Second run: test fails with inconsistent state.
and
PS C:\Users\Chevr\OneDrive\Bureau\eOnsight\eOnsightApi\eonsight-api> pytest app_auth
============================================================= test session starts ==============================================================
platform win32 -- Python 3.12.1, pytest-8.3.3, pluggy-1.5.0
django: version: 5.1.1, settings: eOnsight_api.settings (from ini)
rootdir: C:\Users\Chevr\OneDrive\Bureau\eOnsight\eOnsightApi\eonsight-api
configfile: pytest.ini
plugins: Faker-30.0.0, django-4.9.0, factoryboy-2.7.0, mongo-3.1.0
collected 2 items
app_auth/tests/test_auth_viewset.py::test_login PASSED [ 50%]
app_auth/tests/test_auth_viewset.py::test_login ERROR [ 50%]
app_auth/tests/test_auth_viewset.py::test_logout PASSED [100%]
app_auth/tests/test_auth_viewset.py::test_logout ERROR [100%]
==================================================================== ERRORS ====================================================================
_______________________________________________________ ERROR at teardown of test_login ________________________________________________________
code in setting.py:
# eOnsight_api/settings.py
import os
import sys
from pathlib import Path
from datetime import timedelta
from decouple import config
from mongoengine import connect
import mimetypes
from .config.os_specific_paths import OSPaths
from dotenv import load_dotenv
load_dotenv()
# Get the paths to the GDAL and GEOS libraries based on the current OS
GDAL_LIBRARY_PATH, GEOS_LIBRARY_PATH = OSPaths.get_gdal_geos_paths()
# Logging configuration
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'formatter': 'simple',
},
},
'formatters': {
'simple': {
'format': '%(levelname)s:%(message)s',
},
},
'root': {
'handlers': ['console'],
'level': 'INFO',
},
'loggers': {
'django.db.backends': {
'level': 'WARNING',
'handlers': ['console'],
'propagate': False,
},
},
}
BASE_DIR = Path(__file__).resolve().parent.parent
DEBUG = os.getenv('DEBUG', 'False').lower() == 'true'
SECRET_KEY = os.getenv('SECRET_KEY', 'Default_Key')
# Allowed hosts
ALLOWED_HOSTS = os.getenv('ALLOWED_HOSTS', '127.0.0.1').split(',')
# Production
IS_PRODUCTION = os.environ.get('IS_PRODUCTION', 'True').lower() == 'true'
# Fake data injection
INJECT_FAKE_DATA = os.getenv('INJECT_FAKE_DATA', 'False').lower() == 'true'
# Base INSTALLED_APPS
INSTALLED_APPS_BASE = [
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.messages',
'django.contrib.staticfiles',
'eOnsight_api.apps.EonsightApiConfig',
'corsheaders',
'drf_spectacular',
'rest_framework',
'rest_framework_simplejwt',
'app_infrastructures',
'app_users',
'app_auth'
]
DATABASE_ROUTERS = ['eOnsight_api.config.routers.AuthRouter']
MIDDLEWARE = [
'django_extensions',
]
# Database-dependent apps
DATABASE_DEPENDENT_APPS = [
'django.contrib.admin',
'django.contrib.sessions',
'django.contrib.gis',
]
# Middleware base
MIDDLEWARE_BASE = [
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'corsheaders.middleware.CorsMiddleware',
]
# Configuration DRF Spectacular pour le schéma OpenAPI
REST_FRAMEWORK = {
'DEFAULT_SCHEMA_CLASS': 'drf_spectacular.openapi.AutoSchema',
'DEFAULT_AUTHENTICATION_CLASSES': (
'app_users.authentication.MongoJWTAuthentication',
),
}
SIMPLE_JWT = {
'USER_ID_FIELD': 'id',
'USER_ID_CLAIM': 'user_id',
'ROTATE_REFRESH_TOKENS': True,
'BLACKLIST_AFTER_ROTATION': True,
'AUTH_HEADER_TYPES': ('Bearer',),
'ACCESS_TOKEN_LIFETIME': timedelta(minutes=60 * 2),
'REFRESH_TOKEN_LIFETIME': timedelta(minutes=60 * 4),
}
if 'pytest' in sys.argv:
MONGODB_URI = config('MONGO_TEST_DB_URI')
MONGODB_NAME = config('MONGO_TEST_DB_NAME')
else:
MONGODB_URI = config('MONGO_DB_URI')
MONGODB_NAME = config('MONGO_DB_NAME')
# Connect to MongoDB
try:
connect(
db=MONGODB_NAME,
host=MONGODB_URI,
uuidRepresentation='standard'
)
print("Connected to MongoDB")
except Exception as e:
print(f"Error connecting to MongoDB: {e}")
if not IS_PRODUCTION and INJECT_FAKE_DATA: # Fake data mode
# Use dummy database backend
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.dummy',
}
}
INSTALLED_APPS = INSTALLED_APPS_BASE
# Disable migrations
class DisableMigrations:
def __contains__(self, item):
return True
def __getitem__(self, item):
return None
MIGRATION_MODULES = DisableMigrations()
# Adjust middleware
MIDDLEWARE = MIDDLEWARE_BASE
# Configurer le backend de sessions basé sur les fichiers
SESSION_ENGINE = 'django.contrib.sessions.backends.file'
SESSION_FILE_PATH = os.path.join(BASE_DIR, 'sessions_fakedata')
os.makedirs(SESSION_FILE_PATH, exist_ok=True)
else: # Production mode
# Database configuration
DATABASES = {
'default': {
'ENGINE': 'django.contrib.gis.db.backends.postgis',
'NAME': os.getenv('DB_NAME', 'db_clear'),
'USER': os.getenv('DB_USER', 'user_clear'),
'PASSWORD': os.getenv('DB_PASSWORD', 'password_clear'),
'HOST': os.getenv('DB_HOST', 'localhost'),
'PORT': os.getenv('DB_PORT', '5432'),
'ATOMIC_REQUESTS': True,
}
}
INSTALLED_APPS = INSTALLED_APPS_BASE + DATABASE_DEPENDENT_APPS
# Middleware including database-dependent middleware
MIDDLEWARE = MIDDLEWARE_BASE + [
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
]
# URL configuration
ROOT_URLCONF = 'eOnsight_api.urls'
# Templates configuration
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
# WSGI application
WSGI_APPLICATION = 'eOnsight_api.wsgi.application'
# Password validation
AUTH_PASSWORD_VALIDATORS = [
{
'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
},
]
# Static files
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
mimetypes.add_type("text/css", ".css", True)
# Default primary key field type
DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'
# REST Framework configuration
REST_FRAMEWORK = {
'DEFAULT_SCHEMA_CLASS': 'drf_spectacular.openapi.AutoSchema',
}
# drf_spectacular settings
SPECTACULAR_SETTINGS = {
'TITLE': 'eOnsight API',
'DESCRIPTION': 'Documentation de l\'API pour eOnsight',
'VERSION': '1.0.0',
'SERVE_INCLUDE_SCHEMA': False,
'SECURITY': [{
'apiKeyAuth': {
'type': 'apiKey',
'in': 'header',
'name': 'Authorization'
}
}],
}
# CORS configuration
CORS_ALLOWED_ORIGINS = [
"http://localhost:4200",
]
one off my test app_auth:
import pytest
from django.urls import reverse
from rest_framework.test import APIClient
from rest_framework import status
from app_users.models import User
from app_users.service import UserService
@pytest.fixture
def api_client():
return APIClient()
@pytest.fixture
def create_user(mongo_db):
"""
Fixture pour créer des utilisateurs dans la base MongoDB.
"""
def _create_user(email=None, password="password123", role="user"):
if not email:
email = f"user{User.objects.count()}@example.com"
return UserService.create_user(email=email, password=password, role=role)
return _create_user
@pytest.mark.django_db
def test_login(api_client, create_user):
"""
Test for user login and JWT token retrieval.
"""
user = create_user(email='testuser@example.com', password='password123')
payload = {
'email': user.email,
'password': 'password123'
}
response = api_client.post(reverse('token_obtain_pair'), payload, format='json')
assert response.status_code == status.HTTP_200_OK
assert 'access' in response.data
assert 'refresh' in response.data
@pytest.mark.django_db
def test_logout(api_client, create_user):
"""
Test for user logout using both access and refresh tokens.
"""
user = create_user(email='testuser@example.com', password='password123')
login_payload = {
'email': user.email,
'password': 'password123'
}
login_response = api_client.post(reverse('token_obtain_pair'), login_payload, format='json')
assert login_response.status_code == status.HTTP_200_OK
access_token = login_response.data['access']
refresh_token = login_response.data['refresh']
api_client.credentials(HTTP_AUTHORIZATION=f'Bearer {access_token}')
print(f'user access token: {access_token}')
print(f'user refresh token: {refresh_token}')
logout_payload = {
'refresh_token': refresh_token
}
logout_response = api_client.post(reverse('logout'), logout_payload, format='json')
assert logout_response.status_code == status.HTTP_205_RESET_CONTENT
assert logout_response.data['success'] == 'Tokens have been invalidated successfully'
Steps to Reproduce:
- Set the environment variables IS_PRODUCTION=False and INJECT_FAKE_DATA=True.
- Run the test suite using pytest with pytest-django.
- Observe that the test passes on the first run and fails on the second due to inconsistent state.
What I've Tried:
- Using @override_settings in tests to ensure IS_PRODUCTION and INJECT_FAKE_DATA are correctly set.
- Forcing IS_PRODUCTION and INJECT_FAKE_DATA to be explicit booleans in settings.py.
- Disabling the file watcher (--noreload) during test execution.
- Logging the values of IS_PRODUCTION and INJECT_FAKE_DATA to ensure they are correctly set.
- The bug disappears when comparing IS_PRODUCTION and INJECT_FAKE_DATA as strings, like this but this os not a consistent solution:
if IS_PRODUCTION == 'false' and INJECT_FAKE_DATA == 'true':
# Code here
Expected Behavior: The environment should only initialize once per test run, and the condition in settings.py should not cause the environment to reload.
Actual Behavior: The environment reloads twice, and the second initialization causes inconsistent behavior, leading to test failures.
Question: What could cause Django's test environment to initialize twice, and how can I ensure the conditional block in settings.py doesn’t interfere with the test execution? How can I properly manage this configuration for tests?