GPT-5.4-Cyber

Ingredient list

1. #1FlashAttention

   Invented by Stanford · 2022-05 · Velocity 4y

   “OpenAI's models are known to use optimized attention kernels for training and inference efficiency.”

   inferencemedium
2. #2Zero-Shot Chain-of-Thought

   Invented by University of Tokyo · 2022-05 · Velocity 4y

   “GPT-4 exhibits zero-shot CoT reasoning when prompted with 'think step by step'.”

   reasoninghigh
3. #3Self-Consistency

   Invented by Google · 2022-03 · Velocity 4y

   “GPT-4 can be used with self-consistency by sampling multiple reasoning paths.”

   reasoninghigh
4. #4Chain-of-Thought Prompting

   Invented by Google · 2022-01 · Velocity 4y

   “GPT-4 can generate step-by-step reasoning when prompted appropriately, a core CoT capability.”

   reasoninghigh
5. #5Instruction Tuning (FLAN)

   Invented by Google · 2021-09 · Velocity 5y

   “GPT models are instruction-tuned to follow diverse tasks, aligning with the FLAN paradigm.”

   traininghigh
6. #6Rotary Position Embedding (RoPE)

   Invented by Zhuiyi Technology · 2021-04 · Velocity 5y

   “GPT models use Rotary Position Embeddings (RoPE) for positional encoding.”

   architecturemedium
7. #7Transformer Self-Attention

   Invented by Google · 2017-06 · Velocity 9y

   “GPT models are based on the Transformer architecture with self-attention.”

   architecturehigh
8. #8Mixture of Experts (Sparse MoE for LLMs)

   Invented by Google · 2017-01 · Velocity 9y

   “GPT-4 is reported to use a Mixture of Experts architecture.”

   architecturehigh