Distributed AI Micromouse

System architecture

The project is organized as five ROS2 packages: shared messages, training, dashboard, robot control, and simulation. A robot — simulated or physical — publishes over ROS2 topics to a training server, which produces models and metrics that the dashboard reads.

Splitting messages into their own package is the load-bearing decision. Every other package depends on the contract rather than on each other, which is what allows the simulated robot and a physical one to be interchangeable from the training server's point of view.

The three flows run at different rates for good reason: state is frequent because it drives control, experience is less frequent because it drives learning, and telemetry is slow because nobody needs battery voltage ten times a second. Quality-of-service profiles are configuration rather than code, which is what a lossy radio link to a microcontroller eventually requires.

The interface, declared

The interface between parts is written down as message, service and action definitions rather than left as whatever the publisher happened to send. Six message types, four services and one long-running action.

Three of those four services exist for a single purpose: moving a model between a trainer and a device. One uploads weights, one fetches the current global model, and one fetches it in chunks. That third call is the whole federated design in miniature — a microcontroller cannot receive a model the way a workstation can, so the contract admits it up front.

Q-Learning vs Deep Q-Learning

Both training paths are implemented. The tabular trainer discretizes state, explores with an epsilon-greedy schedule, and persists a table small enough to live in the device's own flash. The deep trainer approximates the same values with a network, stabilized by a replay buffer and a target network, trained in PyTorch on a workstation.

They trade compute against generalization, which is precisely what the experiments are meant to measure. A table is fast, cheap and specific to the maze it learned; a network is expensive to train, needs floating-point inference, and stands a chance of transferring to a maze it has never seen.

Which one suits a constrained device better is a question this project has not answered yet. No winner is declared here, and no comparative figure is reported, because the comparison has not been run and recorded.

Getting it onto the device

Deploying a policy to a microcontroller is its own problem, and the repository treats it as one rather than as a final step. A model trained in PyTorch is converted to TensorFlow Lite, versioned by a model manager, and fetched by the device — in pieces, through the chunked service, because it cannot hold the transfer in memory otherwise.

The tabular path needs none of that. A Q-table is small enough to write straight to flash, which is the practical half of the comparison: not only which algorithm learns better, but which one can actually be deployed and updated on the hardware in question, and what each update costs in flash wear.

A firmware sketch is committed and a converted model sits in the repository, so the path exists end to end in source. The hardware experiments themselves have not been run.

What would be measured

The instrumentation is in place and the measures are chosen. Hardware telemetry covers processor load, free memory, battery voltage, inference time and flash write cycles; training telemetry covers loss, episode reward, exploration decay and buffer size; and three maze layouts of increasing difficulty are defined for testing transfer.

These are targets, not findings. They describe what the experiments are designed to capture, and no value for any of them is reported on this page.

What the tooling can and cannot prove

The analysis tooling is genuinely built: a Q-table visualizer producing heatmaps, coverage maps, per-action panels and policy views; a comparative benchmark tool producing learning curves, resource comparisons and performance tables; and a dashboard showing all of it live. Everything renders at publication resolution.

It also has a mode that generates all of that from synthetic data, alongside a synthetic data generator in the simulation package. That exists for good reasons — you cannot develop a plotting pipeline while waiting on hardware — but it means a finished-looking comparison from this repository is not evidence of anything until it says which data produced it.

That is why no figure from this project appears on this page, and why any that is published later should carry its provenance on its face.

The federated direction

The federated work is a direction with a contract, not a completed result. The service definitions for uploading weights and distributing a global model exist; the intended arrangement is several simulated robots alongside one physical one, with local adaptation on each and aggregation at the centre.

The measure that would make it research rather than plumbing is communication efficiency — what the coordination costs relative to what it buys. Nothing below has been run, and every row is marked accordingly.

The environment is the result

For a project whose output is meant to be research, the environment is not setup detail — it is part of the finding. A comparison of memory use and inference time means very little if a reader cannot determine which toolchain and which versions produced it.

Two things need settling before any number is published. The setup instructions name two different ROS2 releases in different places, and several of the guides the README links to are not in the repository. Neither is difficult to fix, and both would undermine a published figure if left as they are.

What holds, and what does not

The repository is public, so the left column below can be checked rather than trusted: the packages, the nodes, the declared contract, both training paths, the conversion utility and the committed converted model are all there to read.

The right column is the part that keeps this a project rather than a paper. No comparative run has been recorded, no hardware measurement taken, no federated experiment performed, and no maze held out to test transfer. The gap between a system that could produce results and results is exactly the work that remains.

Current status

Phases 1 and 2 are complete: the architecture, both training paths, the simulation, the dashboard and the analysis tooling all exist and run.

Phase 3 — hardware integration, the comparative experiments, the federated work and the analysis that would follow — is ahead. It is listed here as outstanding rather than folded into a summary that would read as finished.