Hi! In the second year at FILS we were assigned to develop and build a project at Microprocessor Architecture course. To just build something using a Rust-based framework and a microcontroller would be too easy. That's why we were also tasked to document the whole development process.

Now, I am a highly autistic individual, and that's why I took personally the whole "document the process" part. And this is why you are reading a blog post on a ghost selfhosted website that lies behind a three level architecture to just expose my server from my dorm room to the internet. But that's beyond the scope of this post, right?

Week 1-3: Let's talk about Radu

Radu is project and project is Radu. Throughout this blog I will be referring to my project by its name - Radu.

Radu started initially as a smart camera that would detect the object in front of it and then describe it using a LLM, do sentiment analysis on the description and build a song based on that sentiment analysis.

The issue here is that Radu was supposed to run on a microcontroller: an ESP32S2 which had a whopping 2mb of SRAM, 4mb Flash and 240MhZ Single Core CPU. Running a LLM on such a device would be technically impossible. And yet some dude made it work on a slighter more advanced version of my microcontroller: an ESP32S3. Notice the difference between ESP32S2 and ESP32S3? Although it's just one digit, the difference is in three times improved performance for matrix operations. And that means three times more tokens/second generated by the S3 compared to my little S2. Nonetheless, using ESP-IDF-HAL with std support and the lovely template that enables Rust and C to live under the same hood I got some text:

I (81) boot: Loaded app from partition at offset 0x10000
I (83) boot: Disabling RNG early entropy source...
I (513) main_task: Started on CPU0
I (513) main_task: Calling app_main()
I (513) LLM: Initializing SPIFFS
I (563) LLM: Partition size: total: 1920401, used: 1072021
I (573) LLM: LLM Path is /data/stories260K.bin
I (573) LLM: Vocab size if 512
I (603) LLM: File size: 1056540 bytes
I (603) LLM: Free ram available: 2298304
I (1113) LLM: Successfully read LLM into memory
I (1113) LLM: Free ram available: 1241980
I (1113) LLM: Successfully read checkpoint
There was a tree. The animals were very happy. The animals liked to play with the animals. They liked to play together.
One day, the animals saw a big, red ball. The animals were scared. They wanted to play with the ball. They did not know what to do.
The animals said, "I want to play with your ball. We can find it."
The animals were sad. They did not want to play with the ball. They wanted to play together. They went to the animals and showed the animals. They were happy. They played together and had fun.
The animals were happy. They played with the animals and had fun. They were happy.
achieved tok/s: 2.276379
I (113503) main_task: Returned from app_main()

It generates something. But the text makes little to no sense, as expected from a 260k params model. Some dude from my year told me that I would be better off using a bunch of ifs instead of a LLM this small, and I think he was right.

Not only this output was incoherent but also was not describing the input object at all (the input prompt was "there was a tree") . There wasn't much I could do here anymore. Yet I proved the teacher wrong, that you can indeed run a LLM on an ESP32.

Week 3-5: A reborn Radu, somewhat.

This is the new Radu. And this time it will no longer be a project using a large language model, but it will measure the speed of light (blazingly fast).

Just don't you ask why in the god damn world would you want to do that, let's just focus on the fun part: how. In 1850 Fizeau measured the speed of light using an gear system, located 8km from Paris where was a mirror. He used a light beam that passed through that aperture and returned back. If he could spin the gear fast enough, the light would not be able to return back through the same aperture and the speed of light could be mathematically computed (more about that in the theoretical setup part)

If we would like to measure the speed of light the same way Fizeau did, we'd have to go on a trip outside of Bucharest during PMFair, which is not really conventient. Luckily for us, both optics and mechanics have evolved since 1850's and we can compress the whole setup in only 3m. The key improvements here are:

• High Reflectivity Mirrors for an increased optical path
• High precision laser cutting for a smaller aperture
• Electric motors for high speeds

Theoretical setup

(will complete later)

Architecture