Discussion About Architecture: ROS2-Django-Webinterface [closed]
I am currently building a cobot/robot arm control interface and workflow planning tool. I am currently contemplating all my architecture choices so far, because I feel like the architecture is not scaling well and could fail under large loads. So I need to ask the community...
General Architecture Overview
The whole architecture is what I would call microservice-based (correct me if I am wrong). I have multiple standalone components (standalone means containerized in this case and also with its own area of functionality. E.g. camera controller, robot controller, gripper controller etc.). All of these components use ROS2 for communication and are implemented in their respective language, mostly C++ and Python. Then there is the Core Application which is a backend for connecting all information and managing the components. This is kind of the brain. This brain also has a UI. For the Core/UI stack I chose Python Django + ReactJS. I chose this stack because: I am fluent in Python and React, fast Prototyping for the first Prototype, Direct ROS2 Integration in Python, Django has a good ORM and supports async operations so that I can connect ROS2 (as a bridge for the UI kind of) and store data through the ORM in the PostgreSQL DB. I am utilizing Django Channels ASGI for Websocket communcation and controls both ways. E.g.:
- ROS2 subscribed topic Log Message from a component comes in (the subscriber runs in a separate container in order to not block the main thread with its wait based executor. Publishing, actions and services are currently executed by spinning the node once or until complete)
- I save the log message in the DB
- Django signal is triggered
- Django Receiver handles signal and dispatches Websocket message send I use Redis as the message broker btw.
The Problem
The larger the app gets, the harder it is to maintain with the current architecture and I am not sure if this architecture will be sufficient under higher loads. I am planning to implement a ROS2 node which subscribes to a camera stream. You can imagine how much data this is. This can definitely not be sent through Redis and I cannot just create a new container every time I need a subscriber... And even then I still have the problem that I need to send the data through my container to my backend to my UI...
I have never designed such a complex architecture (and had to implement it), and I dont know how to go on from here. It is also the first time I used ROS2, so I am not totally familiar with advanced ROS topics like async features...
Does anybody have knowlegde in that area or some tips? Thanks!