DeepSeek-V4.1 Flash is a multimodal mixture‑of‑experts model with 552 billion parameters that natively supports multimodal input and contexts of up to one million tokens, according to the technical report titled DeepSeek‑V4.1‑Flash: Pushing the Limits of KV Cache Compression. The paper states that the model adopts a Causal Encoder‑Decoder (CED) architecture in which the decoder’s global KV cache is obtained by projecting the encoder’s final hidden states. This design means that, during the prefill stage, only the first 20 of the model’s 40 layers are activated, yielding 8 billion activated parameters, while decoding uses all 40 layers for 16 billion activated parameters. KV cache compression is achieved through several complementary techniques. First, head‑count reduction similar to grouped‑query attention (GQA) lowers the number of distinct key/value heads. Second, block‑based compression akin to CSA (Compressed Sparse Attention) is applied. Third, the paper introduces CSA2, which adds cross‑layer KVCache reuse so that multiple layers share the same global cache. Finally, numerical‑precision optimizations store the cache in FP4 format. Under the condition of preserving high‑quality task completion, the report notes that KVCache is further compressed by a factor of four. Consequently, compared with DeepSeek‑V4‑Flash, DeepSeek‑V4.1‑Flash requires only about one‑quarter the runtime KVCache storage and one‑eighth the persistent KVCache storage at the same sequence length. The storage growth of the global main KV and its indexer is reported to be roughly 890 bytes per token when FP4 quantization is used. Architecturally, the channel dimension uses a 512‑dimensional latent vector that shares the key/value representation across attention heads. In the sequence dimension, the encoder merges every two adjacent positions into a single cache entry via channel‑wise learned weights, whereas the decoder retains per‑position entries. In the layer dimension, the network keeps only three copies of the encoder cache and one copy of the decoder cache, reflecting the cross‑layer sharing enabled by CSA2. The sliding window attention component operates with a window size of 128 tokens. Overall, the paper concludes that DeepSeek‑V4.1‑Flash delivers superior performance to its predecessor while drastically reducing the memory and bandwidth demands of long‑context agent workloads. (All factual statements are drawn directly from the provided source text.)
