Research paper introducing mixture-of-depths attention mechanism to mitigate feature dilution in deep LLMs

Research paper published introducing FlashAttention-4 algorithm and kernel pipelining co-design for asymmetric hardware scaling