How to make django faster when there are large datasets?
I am making e-commerce website using django restframework.
However, I realize that the number of reviews are going to be larger as time goes..
(reviews have bunch of images and texts)
let say 1000 reviews per items, It will make whole website slow when I go on to the
product detail page or product list page..
as I use react/redux, I query and get whole 1000 reviews and
store in redux state, I think it takes a long time..
how to prevent django getting slower when
related datasets are bigger and bigger?
So as this is quite a generic question, and there isnt a small snippet of code you can add that makes fetching thousands of reviews at once a good idea I'll answer how to make things fast, in general.
Profile and disover why its slow.
For this you want to use something like django-debug-toolbar and check how much time your view is taking on its different actions, for some views its the database, for some its JSON serialization or external service calls. But step 1 is always to get an understanding why its slow.
Pick the low hanging fruits
Some things are easy to miss and easy to fix, things like make sure you are always using prefetch_related and select_related whenever you are using Django's relation / foreignkeys. Add database indices for faster retrievals. Use aggregations and values / values_list to only select what you need.
Only fetch what you need.
With pagination and filtering you should limit the data sent to the browser / client to the things it needs. In your example - rework the UI to only show the top 10 reviews and let the reader click a button to view more.
Cache views that are slow to render but rarely changed.
Caching is a double edged sword - it makes things fast but also more complex, its a thing you should learn as there is only so much you can optimize in code and some views are just naturally slower. There are extensions that makes this job easier.