One of the things I love using LLMs for is helping me understand systems better. In the Strands Decider post I wrote about how I often ask ask me questions to check I understand this. The other technique I’m enjoying right now, and am delighted to see folks around me using too, is building animations.
Animations can help us build an intuition for how and why systems work, or how and why they fail.
Here’s one example: did you know that a Raft cluster can lose writes if even one of the cluster members has a storage layer that lies about fsync()?
When I built Stability Sim, I wanted to help engineers understand metastable failures better. That worked pretty well on an abstract level (e.g. retries and queues make metastability really easy to trigger), but I still hear doubts that these behaviors happen in real systems. After all, those real systems are production-proven, and wouldn’t we have shaken out the behaviors by now if they existed?
