Abstract
We consider the problem of sequential task continuation, where one robot acts in a workspace that has been affected by a previous robot. Current approaches in multi-robot cooperation rely heavily on co-training or co-optimization, assuming that the robots have access to collective interaction data during the policy training or planning process. However, as increasingly varied robotic embodiments and policies are deployed and operate in the same real-world environment, learning policies for cooperation becomes infeasible due to the combinatorial explosion of possible robot combinations that need to be accounted for. To address this challenge, this paper considers the problem of ad hoc sequential cooperation between robots that have never before operated together in a shared environment, and proposes PAVE (Policy Adaptation via Value Exchange) as a solution. We demonstrate that robots can explicitly communicate their environmental preferences through sharing their value functions, enabling coordination without the need for prior joint training. To ground this exchange and prevent overconfidence in out-of-distribution states, we further introduce a k-nearest neighbors (KNN)-based uncertainty filter. Our method yields an increase of up to 47.7% in task success rates across simulated and real-world experiments compared with purely selfish agents, demonstrating that value function transmission between agents is useful as a basis for ad hoc coordination.
Experiments
The first agent pushes the table aside to clean beneath it; a follower must then traverse the room. Naive (left) the leader is unaware of the follower's needs and leaves the table blocking the path, preventing traversal. PAVE(right): the follower shares its value function over environment states, and the leader adapts by moving the out of the way for the follower, clearing the path, and enabling traversal.
The first agent picks up and object and places it in one of three transfer areas. The second agent must then pick up the object and place it in the goal area. Naive (left): the leader is unaware of the follower's needs and places the object in a transfer area that is impossible for the follower to access, preventing task completion. PAVE(right): the follower shares its value function over environment states as well as its uncertainty, and the leader adapts by placing the object in a reachable transfer area that is also high-value for the follower, enabling task completion.
The first agents controls the movement of a mobile base, navigating to the vercinity of an object of interest. The second agents control the robotics arm mounted on the mobile base, and must pick up the object and place it in the goal area. The arms rotation is restricted, such that i can only pick up objects in front of it. Naive (left): the leader is unaware of the follower's needs and navigates to a location, but orients the mobile base such that the follower cannot access the object, preventing task completion. PAVE(right): the follower shares its value function over environment states as well as its uncertainty, and the leader adapts by orienting the mobile base such that the object is within the follower's reach, enabling task completion.
The first agent has to clear the lower half of the arena, by pushing it out of the center and into contact with any wall; a follower must then traverse the room. Naive (left): the leader is unaware of the follower's needs and pushes the obstacle into the gap blocking the path, preventing traversal. PAVE(right): the leader adapts by moving the out of the way for the follower, clearing the path, and enabling traversal.
BibTeX
@inproceedings{gomezandreu2026leave,
title = {Leave It Better Than Found: Value-Guided Cooperation in Sequential Tasks},
author = {Gomez Andreu, Mario and Amir, Michael and Marino, Antonio and An, Tianxu and Hutter, Marco and Prorok, Amanda},
booktitle = {Conference on Robot Learning (CoRL)},
year = {2026},
note = {Spotlight}
}