For an LLM pretraining company (or academic organization), the speed and quality of new model development heavily depend on the stability and efficiency of the infrastructure. An unstable infrastructure can cause significant project delays—sometimes by several months. We value a strong engineering culture over a purely academic one, as some researchers tend to overlook infrastructure, resulting in work that lacks continuity and long-term impact.

The entropy of infrastructure determines LLM productivity. That’s a good point of one of my colleages.