Claude 3

Ingredient list

1. #1Grouped-Query Attention (GQA)

   Invented by Google · 2023-05 · Velocity 3y

   “Claude 3 models use GQA to improve inference efficiency, as stated in the system card.”

   architecturehigh
2. #2Constitutional AI

   Invented by Anthropic · 2022-12 · Velocity 3y

   “Claude models are trained using Constitutional AI, Anthropic's proprietary alignment method.”

   alignmenthigh
3. #3Zero-Shot Chain-of-Thought

   Invented by University of Tokyo · 2022-05 · Velocity 4y

   “Claude 3 demonstrates zero-shot chain-of-thought reasoning without explicit few-shot examples.”

   reasoninghigh
4. #4Self-Consistency

   Invented by Google · 2022-03 · Velocity 4y

   “Claude 3 can generate multiple reasoning paths and select the best answer, demonstrating self-consistency.”

   reasoninghigh
5. #5Chain-of-Thought Prompting

   Invented by Google · 2022-01 · Velocity 4y

   “Claude 3 exhibits strong chain-of-thought reasoning capabilities in its technical report.”

   reasoninghigh
6. #6Instruction Tuning (FLAN)

   Invented by Google · 2021-09 · Velocity 4y

   “Claude 3 is instruction-tuned to follow diverse instructions, a core capability mentioned in documentation.”

   traininghigh
7. #7Rotary Position Embedding (RoPE)

   Invented by Zhuiyi Technology · 2021-04 · Velocity 5y

   “Claude 3 uses Rotary Position Embeddings (RoPE) for positional encoding, per technical details.”

   architecturehigh
8. #8Transformer Self-Attention

   Invented by Google · 2017-06 · Velocity 9y

   “Claude 3 is built on a Transformer architecture with self-attention, as stated in its technical report.”

   architecturehigh