NavMCP scaffolds a Vision-Language Model (VLM) and a Navigation Foundation Model (NFM) into a long-horizon physical-world agent through intent, observation, and memory channels.
01 · The premise
“The central challenge is not simply to build a stronger reasoner or navigator, but to organize their complementary capabilities into persistent physical-world agency.”
VLMs can infer missing information and adapt high-level plans, but grounding every decision into navigation is brittle. NFMs execute semantic goals robustly, yet typically operate as bounded episodes without persistent task-level reasoning.
NavMCP closes this episodic interface gap with an evidence-centric scaffold that turns isolated navigation rollouts into persistent embodied interaction—without retraining either foundation model.
02 · The scaffold
Each channel resolves one failure mode at the agent–executor boundary.
From evidence needs to navigation calls
The agent converts a missing-evidence hypothesis into a semantic call with a mode, natural-language sub-goal, budget, and constraints. The NFM remains responsible for the route.
target: “small black sofa”budget: 120 stepsFrom trajectories to grounded evidence
Keyframes across the complete rollout become a source-grounded journey artifact: observed rooms and objects, route context, uncertainty, and cues for the next search.
evidence: frame_18 → hallway sofauncertainty: mediumFrom tool returns to task state
Positive evidence, negative evidence, checked regions, unresolved goals, and answer support persist in the next-turn context—preventing repeated searches and premature answers.
checked: kitchen, loungenext: search bedroom corners
The VLM turns the question and current view into an explicit evidence need.
NavMCP delegates a grounded sub-goal while the NFM handles local execution.
Trajectory keyframes preserve what was seen, where it appeared, and what remains uncertain.
Accumulated positive and negative evidence supports the next search or a grounded final response.
03 · Evidence across benchmarks
Results cover single-question search, multi-target reasoning, and free-form embodied QA.
+7.5 points over FAST-EQA
+3.9 points over FAST-EQA
+10.57 LLM Score
+4.71 Epath
Matched HM-EQA comparison
Episodic interface gap
Experiment 01 · Embodied question answering
The agent moves beyond the initial view, selects semantic navigation goals, and gathers the visual evidence needed to answer.
04 · Beyond benchmark episodes
The same evidence-centric interface scales from controlled EQA to extended real-world execution and open-ended assistant scenarios.
Experiment 02 · Long-horizon deployment
A Unitree Go2 follows a long semantic route toward a coffee shop using a lightweight map while the agent monitors progress and revises its search.
Experiment 03 · Future-facing scenarios
Beyond a benchmark question, users can delegate open-ended goals while the agent coordinates reasoning, evidence seeking, movement, and a grounded response.
Express a goal in language rather than specifying waypoints or control sequences.
Combine global reasoning with a navigation foundation model that handles local motion.
Preserve journey evidence so the final response reflects what the robot actually observed.
05 · Read & cite
Read the full paper for the agentic scaffolding framework, evaluation protocols, controlled ablations, and additional qualitative cases.
Open paper ↗@article{lei2026scaffolding,
title = {Scaffolding Foundation Models into Physical-World
Agents Pushes the Frontier of Long-Horizon Navigation},
author = {Lei, Zixing and Zhou, Gengze and Chen, Xiong-Hui
and Zhang, Jiazhao and Huang, Yiyang and Yin, Hang
and Yuan, Haoqi and Wu, Qi and Li, Weixin
and Chen, Siheng},
year = {2026}
}