Llama 4 Maverick

Ingredient list

1. #1YaRN RoPE Context Extension

   Invented by Nous Research · 2023-08 · Velocity 1.6y

   “Llama 4 Maverick supports 1M context. Meta's previous long-context models (Llama 3.1) used YaRN.”

   architecturemedium
2. #2Rotary Position Embedding (RoPE)

   Invented by Zhuiyi Technology · 2021-04 · Velocity 4y

   “Llama family models consistently use RoPE. Llama 4 is a direct successor.”

   architecturehigh
3. #3Transformer Self-Attention

   Invented by Google · 2017-06 · Velocity 8y

   “Llama 4 is a Transformer-based LLM, the core architecture is self-attention.”

   architecturehigh
4. #4Mixture of Experts (Sparse MoE for LLMs)

   Invented by Google · 2017-01 · Velocity 8y

   “Meta's flagship open-weight multimodal MoE with 17B active / 400B total params, 128 experts”

   architecturehigh