Since Richard Sutton’s influential essay in 2019, the concept of the bitter lesson has guided much of modern progress in AI. This concept espouses general methods that leverage computation and learn with scale, as free of human-engineered priors as possible. Our interpretation is that, fundamentally, intelligence is about learning from raw data: autonomously finding patterns, building abstractions, and developing meaning from messy low-level inputs, ideally end-to-end and as data-efficiently as possible. As AI advances into complex domains such as science, robotics, and biology – where data is scarce and often difficult for humans to understand, let alone engineer for – internalizing this philosophy to develop more general learning algorithms is more important than ever.
Scaling Hierarchical Networks
A newer generation of H-Nets learns directly from raw bytes, and matches an open-source 3B Transformer with 10x less compute and data.

Key takeaways
The article discusses hierarchical networks (H-Nets) as a step toward general AI learning from raw data, improving on traditional language models.
- H-Nets aim to learn from raw data without hand-crafted encoding steps.
- The approach is seen as foundational for future AI progress beyond current language models.
- Developing byte-level models is considered a key test for general learning methods.
Summarised automatically by AI from the original article by Hacker News. AI can make mistakes, so check the original for details.
Story details
- Published
- By
- guybedo
- Format
- Article
- Original
- cartesia.ai ↗




