Unsupervised Partner Design Enables Robust Ad-Hoc Teamwork

Constantin Ruhdorfer,   Matteo Bortoletto,   Victor Oei,   Anna Penzkofer,   Andreas Bulling

ICML 2026, Seoul, South Korea

Unsupervised Partner Design (UPD) is a new framework for ad-hoc teamwork that draws on ideas from unsupervised environment design (UED). UPD focuses on scalability and integratation with other UED. Our practical instantiation of this framework builds on a partner generator derived inspired and derived from E3T and on the UED algorithm sampling-for-learnability.

NOTE: Our UPD is called DPD (dual partner design) in this repository which was our intial name for our algorithm. A future version of this repository will likely rename DPD to UPD but for the moment the code is provided as is.

Repository Structure

src/
--| agents/
--| --| overcooked/                  # Hardcoded policies from ROTATE
--| --| actors.py                    # Neural networks
--| --| agent_interface.py           # Agent interface class

--| envs/
--| --| overcooked/                  # Augemented overcooked from Jaxmarl and ROTATE
--| --| ogc/                         # The Overcooked Generalisation Challange

--| jaxzsc/                          # Algorithms
--| --| best_response/               # Algothrims for BR training against a population
--| --| brdiv/                       # BRDiv algorithm for eval partners
--| --| dpd/                         # Our unsupervised partner desing method
--| --| --| dpd_ippo_*_w_bias_rnn.py # Our main contribution.
--| --| e3t/                         # E3T baseline
--| --| evaluation/                  # Evaluation scripts
--| --| fcp/                         # FCP baseline
--| --| mep/                         # MEP baseline
--| --| sp/                          # Self-play baseline

Our algorithm can be found in src/jaxzsc/dpd/dpd_ippo_overcooked_w_bias_rnn.py.

We generally implement algorithms as single file implementations as much as possible. As such there is considerable repetition between files.

Reproducing results

For the ROTATE results in the appendix please note that we used the official code to train ROTATE and wrote a second evaluation script to pair these ROTATE agents with our own partners (src/jaxzsc/evaluation/eval_overcooked_rotate_checkpoints_aht.py). A consequence of this is that ROTATE is a bit more cumbersome to evaluate since we load their checkpoint structure into our evaluation scripts. Specifically, to eval ROTATE checkpoints, look for the heldout_ego.yaml in evaluation/config and modify the path: variable to run the correct checkpoint you desire and set layout in the corresponding evaluation script.

References

We built upon several open-source code bases. Please cite them when making use of our work.

Our AHT code base adopts from three projects. Jha2025Cross provides the basis for FCP, E3T and by extension therefore UPD and MEP (which references Zhao2023Maximum). Wang2025ROTATE provides the basis for our evaluation partners, esp. src/agents.

@inproceedings{Jha2025Cross
  author       = {Kunal Jha and
                  Wilka Carvalho and
                  Yancheng Liang and
                  Simon Shaolei Du and
                  Max Kleiman{-}Weiner and
                  Natasha Jaques},
  editor       = {Aarti Singh and
                  Maryam Fazel and
                  Daniel Hsu and
                  Simon Lacoste{-}Julien and
                  Felix Berkenkamp and
                  Tegan Maharaj and
                  Kiri Wagstaff and
                  Jerry Zhu},
  title        = {Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination},
  booktitle    = {Forty-second International Conference on Machine Learning, {ICML}
                  2025, Vancouver, BC, Canada, July 13-19, 2025},
  series       = {Proceedings of Machine Learning Research},
  publisher    = {{PMLR} / OpenReview.net},
  year         = {2025},
  url          = {https://proceedings.mlr.press/v267/jha25b.html},
}

@article{Wang2025ROTATE,
  author       = {Caroline Wang and
                  Arrasy Rahman and
                  Jiaxun Cui and
                  Yoonchang Sung and
                  Peter Stone},
  title        = {{ROTATE:} Regret-driven Open-ended Training for Ad Hoc Teamwork},
  journal      = {CoRR},
  volume       = {abs/2505.23686},
  year         = {2025},
  url          = {https://doi.org/10.48550/arXiv.2505.23686},
  doi          = {10.48550/ARXIV.2505.23686},
  eprinttype   = {arXiv},
  eprint       = {2505.23686},
}

@inproceedings{Zhao2023Maximum,
  author       = {Rui Zhao and
                  Jinming Song and
                  Yufeng Yuan and
                  Haifeng Hu and
                  Yang Gao and
                  Yi Wu and
                  Zhongqian Sun and
                  Wei Yang},
  editor       = {Brian Williams and
                  Yiling Chen and
                  Jennifer Neville},
  title        = {Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordination},
  booktitle    = {Thirty-Seventh {AAAI} Conference on Artificial Intelligence, {AAAI}
                  2023, Thirty-Fifth Conference on Innovative Applications of Artificial
                  Intelligence, {IAAI} 2023, Thirteenth Symposium on Educational Advances
                  in Artificial Intelligence, {EAAI} 2023, Washington, DC, USA, February
                  7-14, 2023},
  pages        = {6145--6153},
  publisher    = {{AAAI} Press},
  year         = {2023},
  url          = {https://doi.org/10.1609/aaai.v37i5.25758},
  doi          = {10.1609/AAAI.V37I5.25758},
}

We extend SFL from

@inproceedings{Rutherford2024No,
  author       = {Alexander Rutherford and
                  Michael Beukman and
                  Timon Willi and
                  Bruno Lacerda and
                  Nick Hawes and
                  Jakob N. Foerster},
  editor       = {Amir Globersons and
                  Lester Mackey and
                  Danielle Belgrave and
                  Angela Fan and
                  Ulrich Paquet and
                  Jakub M. Tomczak and
                  Cheng Zhang},
  title        = {No Regrets: Investigating and Improving Regret Approximations for
                  Curriculum Discovery},
  booktitle    = {Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024},
  year         = {2024},
  url          = {http://papers.nips.cc/paper\_files/paper/2024/hash/1d0ed12c3fda52f2c241a0cebcf739a6-Abstract-Conference.html},
}

Other useful code bases we used or referenced:

@misc{ruhdorfer2025overcookedgeneralisationchallenge,
      title={The Overcooked Generalisation Challenge},
      author={Constantin Ruhdorfer and Matteo Bortoletto and Anna Penzkofer and Andreas Bulling},
      year={2025},
      eprint={2406.17949},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2406.17949},
}

@article{coward2024JaxUED,
  title={JaxUED: A simple and useable UED library in Jax},
  author={Samuel Coward and Michael Beukman and Jakob Foerster},
  journal={arXiv preprint},
  year={2024},
}

@inproceedings{
    flair2024jaxmarl,
    title={JaxMARL: Multi-Agent RL Environments and Algorithms in JAX},
    author={Alexander Rutherford and Benjamin Ellis and Matteo Gallici and Jonathan Cook and Andrei Lupu and Gar{\dh}ar Ingvarsson and Timon Willi and Ravi Hammond and Akbir Khan and Christian Schroeder de Witt and Alexandra Souly and Saptarashmi Bandyopadhyay and Mikayel Samvelyan and Minqi Jiang and Robert Tjarko Lange and Shimon Whiteson and Bruno Lacerda and Nick Hawes and Tim Rockt{\"a}schel and Chris Lu and Jakob Nicolaus Foerster},
    booktitle={The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
    year={2024},
}