Gemini 3.1

Ingredient list

1. #1YaRN RoPE Context Extension

   Invented by Nous Research · 2023-08 · Velocity 2y

   “The 10M token context window suggests use of advanced RoPE extension techniques like YaRN.”

   architecturemedium
2. #2Chain-of-Thought Prompting

   Invented by Google · 2022-01 · Velocity 4y

   “Gemini models are known to be prompted with chain-of-thought reasoning for complex tasks.”

   reasoningmedium
3. #3Instruction Tuning (FLAN)

   Invented by Google · 2021-09 · Velocity 4y

   “Gemini models are instruction-tuned, building on Google's FLAN tradition.”

   trainingmedium
4. #4Rotary Position Embedding (RoPE)

   Invented by Zhuiyi Technology · 2021-04 · Velocity 5y

   “Gemini models use Rotary Position Embeddings (RoPE) for position encoding.”

   architecturemedium
5. #5Transformer Self-Attention

   Invented by Google · 2017-06 · Velocity 9y

   “Gemini is a Transformer-based model, using self-attention as its core architecture.”

   architecturehigh
6. #6Mixture of Experts (Sparse MoE for LLMs)

   Invented by Google · 2017-01 · Velocity 9y

   “Gemini 3.1 is distinguished by its 'Mixture of Experts' (MoE) architecture.”

   architecturehigh