INT8 Weight Quantization for LLMs

Row-wise and vector-wise INT8 quantization with outlier detection that enables zero-degradation 8-bit inference of LLMs.

Two-hop subgraph: this entity, every entity it directly relates to, and every entity those neighbors relate to. Drag a node, scroll to zoom, click to inspect — or click any neighbor and re-center the atlas there.