跪拜 Guibai
← Back to the summary

LongCat-2.5: Meituan's 1.6T-Parameter Multimodal Model for Long-Horizon Agent Tasks

Meituan officially releases LongCat-2.5, a natively multimodal model for long-horizon agent tasks. The model has a total of 1.6 trillion parameters, with about 48 billion active per token. Building on LongCat-2.0, it further strengthens native visual understanding and agentic programming capabilities.

The new model shows clear improvements on long-horizon agent tasks, driven by adding more long-trajectory data in long-sequence training and by large-scale long-horizon reinforcement learning (RL) that boosts end-to-end engineering ability on complex codebase development and full-project construction. We keep extending this training pipeline across data synthesis, environment building, and cross-harness mixed training. We also convert real agent failure trajectories into reproducible, verifiable training tasks, continuously supplementing training data to address model weaknesses.

LongCat-2.5 also has native visual understanding. It represents visual signals as multi-level discrete tokens, letting the backbone process visual and text inputs as a unified token stream. It performs well on Visual Coding and supports creative design and 3D tasks.

Behind these upgrades is an evolving infrastructure: we optimized long-context training and W4A4 inference on domestic accelerators, and used training-inference numerical consistency plus long-context rollout optimization to support efficient large-scale long-horizon RL.

Benchmark Overview

To give a direct view of LongCat-2.5's current capability, we selected GPT-5.5, GPT-5.6-Sol, Claude-Opus-4.8, and Claude-Opus-5 for comparison in the figure below. Detailed evaluation results covering more models appear in the later tables.

01 Visual Representation Modeling

LongCat-2.5 represents visual signals as multi-level discrete tokens, allowing the backbone to process visual and text inputs as a unified token stream. This design integrates native-resolution visual discretization, offline visual preprocessing, and visual embeddings jointly learned with the backbone. It supports native multimodal fusion while simplifying visual computation and load management during training.

Discrete native-resolution vision Transformer: To give visual primitive modeling a token-level abstraction similar to language modeling, we represent visual signals as multi-level vocabulary tokens. Following LongCat-Next[1], we apply Residual Vector Quantization (RVQ) to native-resolution features from a Semantic-and-Aligned Encoder (SAE), addressing the information bottleneck in visual discretization.

Figure 1: Native-resolution visual discretization — encoding images of different resolutions as sequences of discrete tokens and reconstructing images via a visual decoder

Offline visual token preprocessing: To reduce online visual computation and workload heterogeneity, a fixed tokenizer lets us pre-compute and cache visual token IDs offline. The backbone then directly processes pre-tokenized sequences, simplifying long-sequence packing and splitting, and improving load balancing in distributed training.

Native visual representation: Visual codebook embeddings are learned jointly with the backbone. On a smaller MoE model with about 3B active parameters, initial validation shows that visual and text embeddings appear interleaved in t-SNE visualization, indicating fusion in a shared semantic space.

Figure 2: t-SNE comparison of visual and text embeddings. Right side shows validation on an MoE model with ~3B active parameters

02 Large-Scale Agentic Reinforcement Learning

RL scaling is a key step in improving model intelligence. We built a comprehensive Agentic RL training pipeline for LongCat-2.5, scaling long-horizon tasks across data, environment, algorithm, and harness.

Figure 3: Agentic RL training pipeline for long-horizon tasks, covering data and environment, async training, reward design, and mixed training

Large-scale data synthesis and environment construction: We built a scalable data synthesis pipeline that automatically generates training tasks, interactive environments, and verifiable test cases. Data sources include public GitHub repositories and code tasks from internal real-world scenarios. For filtered PRs and associated issues, we use agents to auto-label, synthesize environments, and run fail-to-pass (F2P) and pass-to-pass (P2P) test validation. During RL training, real-time rollout trajectories and pass rates help us do secondary labeling and filtering. Based on false-negative and false-positive analysis, we continuously correct and expand F2P test cases to improve coverage of task functional descriptions and the accuracy of validation signals.

Asynchronous RL and stability optimization: We adopt a partial rollout asynchronous training strategy to address long-tail efficiency issues: each RL step maintains a certain oversampling rate, interrupts after forming a batch, saves unfinished trajectories, and resumes in the next sampling phase. The oversampling rate is adjusted based on task pass rate, and zero-gradient groups (all-correct or all-wrong) are filtered to ensure effective gradients. In partial rollout, tokens from the same trajectory may come from different historical checkpoints, causing sample staleness. To stabilize training, we clip the logprob difference between sampling and current training time, and adjust each token's gradient contribution based on historical cumulative difference, limiting gradient bias from staleness. Additionally, we provide infrastructure-level solutions for training-inference consistency.

Reward hacking and output behavior optimization: We restrict speculative behavior by disconnecting the network, clearing git history and file caches in the task environment, and introducing LLM judges to supervise trajectories. For output behavior, we penalize repeated thinking and repeated tool calls; for overthinking and rapid length growth, we suppress them by reallocating rewards among positive samples.

Mixed training and generalization: The model can autonomously learn and adapt to similar tasks and harnesses during RL. The pre-cold-start phase only needs a small number of adapter scaffold cases. In the RL phase, mixed training across tasks and scaffolds significantly improves generalization and boosts performance on different tasks.

03 Multi-Expert Online Teaching

To integrate capabilities from various domains into the main model, we retained the expert group mechanism from LongCat-2.0, optimizing agent capability, reasoning capability, and interaction experience, covering multi-turn task execution, single-turn reasoning, fine-grained instruction following, and safety defense without sacrificing helpfulness. In MOPD, the student model samples online across domain tasks, and domain experts train the student using full-vocabulary KL divergence[2] on sampled results.

Samples must reflect the student's actual task behavior while giving the teacher credible learning signals. To limit distribution shift, prior work used trajectories seen by the teacher during SFT or RL as per-turn prefixes, serving as context for student sampling. We further split each teacher response into per-cut prefixes of fixed length and cap the student's self-sampling max length to that value, handling distribution shift in single-turn long responses.

Figure 4: Per-cut prefix used for student online sampling

To control compute and storage costs, prefixes from the same teacher trajectory are grouped and sent to the same inference engine, sharing prefill computation and KV cache. During training, we build a tree with teacher trajectories as trunks and student samples as branches. Combined with Tree Attention[3], we complete OPD training for all samples in a group in a single forward and backward pass, avoiding redundant computation and storage on teacher trajectories.

04 Long-Context Training on Domestic Accelerators

LongCat-2.5 uses long-sequence training in both pre-training and supervised fine-tuning. Given the limitation of less than 80 GB memory per card on domestic accelerators, the training system focuses on memory footprint optimization and designs parallel strategies around supernode architecture.

4.1 Parallel Strategy for Supernodes

LongCat-2.5 configures parallel groups based on supernode topology. FSDP and EP parallelism match the number of cards in a single supernode, and CP parallelism does not exceed that number, so parameter AllGather, expert token AllToAll, and KV AllGather all stay within the high-bandwidth interconnect domain of the supernode.

Mixed vocabulary parallelism shards the base word embedding layer, LM head, and N-gram embedding layer across the joint DP and CP parallel domain, further reducing memory usage.

To address load imbalance in large-scale expert parallelism, LongCat-2.5 introduces a threshold-triggered dynamic expert replication mechanism based on MoonEP. Replication is enabled only when EP load imbalance reaches a preset threshold, avoiding communication overhead that exceeds the benefit of load balancing, and preventing memory overflow on the receiving end from routed tokens and on the sending end from communication operations.

Pre-generating image tokens removes the vision encoder from the training compute graph, simplifying parallel implementation. Using known image token counts, the system can estimate per-modal load ratios during packing and splitting, keeping modal ratios close across data-parallel ranks.

4.2 FSDP and System Optimizations

LongCat-2.5 introduces dual-shard FSDP. Model parameters are sharded within the FSDP domain in a supernode; FP32 master weights and optimizer states are further sharded across FSDP domains via HSDP replicas, maintaining parameter communication efficiency while reducing resident memory.

Parameters, gradients, and optimizer states are organized as contiguous flattened buffers, supporting non-uniform sharding and Muon optimizer sharding. Direct weight-gradient writes and temporary buffer reuse cut intermediate copies, lowering peak memory.

The system uses zero-copy AllGather/ReduceScatter to remove intermediate copies from default communication algorithms. EP packs tokens and routing probabilities together, and uses AIV-based communication to mitigate AICPU task expansion bottlenecks. Parameter prefetch overlaps AllGather with computation while controlling resource contention with CP; ScMoE overlaps EP communication with dense branch computation.

The system reduces host-side overhead by batching gradient operations, merging metric reductions, CPU pinning, and NUMA-aware scheduling.

With these optimizations, this FSDP configuration improves single-card throughput by 86% over the DDP baseline in long-sequence SFT tasks.

![Figure 5: Long-context training on domestic AI accelerators](https://p3-xtjj-sign.byteimg.com/tos-cn-i-73owjymdk6/7b47b22ed6534c7592e4e6a0cf399545~tplv-73owjymdk6-jj-mark-v1:0:0:0:0:5o6Y6YeR5oqA5pyv56S-5Yy6IEAg576O5Zui5oqA5pyv5Zui6Zif:q75.awebp?rk3s=f64ab15b&x-expires=1794212026&x-signature=Wf? Actually, need to preserve the exact link. Let's copy the original link exactly.

But I'll keep the image links as they are.

Continue:

05 Agentic Reinforcement Learning Infrastructure

5.1 Training-Inference Precision Consistency

Based on IsoExec[4], GLM-5[5], and XoRL[6], we built a production-grade training-inference consistency scheme that allows training, Prefill, and Decode to independently choose parallel configurations while maintaining bitwise-identical per-token logprob with near-zero performance loss. Both training and inference use efficient batch-invariant operators and aligned framework computation logic, adding backward support. This scheme is enabled by default in both experimental and production tasks, providing a unified numerical regression benchmark for operator and framework iteration. To handle cases where different parallelism prevents training-inference alignment, we extended the community approach to support TP and DCP consistency.

TP: Replaces segmented summation with full K-dim computation, near-lossless performance. Training keeps weight sharding, using weight All-Gather and input AlltoAll to independently complete the full K-dim GEMM, without Reduce-Scatter summation of partial results. Prefill follows the same approach; Decode changes o_proj/fc2 to column-parallel, using input All-Gather, full K-dim GEMM, and output AlltoAll, avoiding per-step full weight aggregation and cross-card numerical summation.

DCP: Reproduces Decode's accumulation path in training and Prefill. We integrate matching sparse attention operators in training forward and inference Prefill to reproduce Decode's KV chunking and reduction order under specified DCP configurations, aligning computation precision across the three.

5.2 Long-Context Rollout Optimization

For load-controlled long-context multi-turn rollout, we adopt PD separation, choosing appropriate parallel strategies for Prefill and Decode separately, and optimize jointly from three aspects: cache storage, cross-layer offloading, and request scheduling.

Figure 6: Prefill/Decode configuration for the tested long-context workload; in this scenario, a 3:2 P/D instance ratio performs best

06 W4A4 Efficient Inference

For the 1.6T-class MoE model, we combined model structure with domestic AI accelerator hardware characteristics to co-optimize quantization precision, underlying operators, and inference pipeline, building a W4A4 efficient inference solution.

This scheme quantizes both expert weights and activations to 4 bits and uses native INT4 matrix-multiplication units for the corresponding computations.

In precision, based on FlatQuant's learnable equivalence transformation, we further optimize quantization precision for MoE models, maintaining performance comparable to the unquantized model across all evaluated downstream tasks.

In operator and data path, we quantize before token duplication and distribution via expert-shared input transformation, keeping the same INT4 representation through expert communication and computation. We fuse transformation, scale computation, quantization, and packing into specialized operators, and optimize weight layout and transposed access to reduce data movement and conversion overhead.

At the system level, for prefill and decode execution characteristics, we optimize overlap of computation and communication based on model data dependencies and hardware resource constraints.

In online stress tests, compared to the W8A8 INT8 baseline, prefill and decode throughput increased by 43.3% and 42.77% respectively, while TTFT and TPOT dropped by 15.91% and 19.72%. At the operator level, GMM achieved an average 1.73× speedup over W8A8.

07 Capability Demonstration

View capability demonstration

08 Evaluation Notes

See the official technical blog for details: https://longcat.ai/blog/longcat-2.5.

References

Experience Entry

| Follow the "Meituan Tech Team" WeChat official account (meituantech) or visit: https://tech.meituan.com/ for more technical content!

| This article is produced by Meituan Tech Team and the copyright belongs to Meituan. For non-commercial purposes such as sharing and communication, you are welcome to reprint or use the content, with attribution to "Reprinted from Meituan Tech Team". Without permission, commercial reproduction or use is not allowed. For any commercial use, please email [email protected] for authorization.