Claude 3.5 Sonnet

Ingredient list

1. #1FlashAttention

   Invented by Stanford · 2022-05 · Velocity 4y

   “Anthropic's research mentions using FlashAttention for efficient training of their transformer models.”

   inferencemedium
2. #2Zero-Shot Chain-of-Thought

   Invented by University of Tokyo · 2022-05 · Velocity 4y

   “Claude exhibits zero-shot chain-of-thought reasoning when prompted appropriately.”

   reasoninghigh
3. #3Chain-of-Thought Prompting

   Invented by Google · 2022-01 · Velocity 4y

   “Claude 3.5 Sonnet demonstrates chain-of-thought reasoning in its outputs when solving complex problems.”

   reasoninghigh
4. #4Instruction Tuning (FLAN)

   Invented by Google · 2021-09 · Velocity 4y

   “Claude models are instruction-tuned on diverse tasks to follow instructions and generalize to new tasks.”

   traininghigh
5. #5Rotary Position Embedding (RoPE)

   Invented by Zhuiyi Technology · 2021-04 · Velocity 5y

   “Anthropic's Claude models use rotary position embeddings (RoPE) for position encoding.”

   architecturehigh
6. #6Transformer Self-Attention

   Invented by Google · 2017-06 · Velocity 9y

   “Claude 3.5 Sonnet is built on transformer architecture with self-attention mechanisms.”

   architecturehigh