ProToM: Promoting Prosocial Behaviour via Theory of Mind-Informed Feedback

Matteo Bortoletto,   Yichao Zhou,   Lance Ying,   Tianmin Shu,   Andreas Bulling

[Paper] [Website]

Introduction

We aim to use AI to encourage prosocial interactions among human agents pursuing independent goals. We introduce ProToM, a facilitator that promotes prosocial actions by providing targeted, context-sensitive feedback to individual agents.

We test ProToM using two environments: Overcooked and Multi-Agent Doors, Keys, and Gems (mDKG).

Overcooked

Installation

conda create --name <ENV_NAME> python=3.9
conda activate <ENV_NAME>
conda install pip 
cd gym-cooking
pip install -r requirements.txt
cd .. 
pip install . 

Usage

nice -n 5 python main.py --level $level --diff_goals --faci_per_agent_particles=5 --agent_particles=5 --partial-obs --mp --model1 single --model2 single --facilitator prof --record --po_render --logging 

The arguments are in arguments.py.

Code Structure

The main code is in gym_cooking/. Here are the main related files for this project:

• main.py: The main entry point for running the environment with agents.

Environment

• envs/overcooked_environment.py: The Overcooked environment implementation.

Agents

• agents/single_agent.py: Contains the SingleAgent class that implements the single agent model.
• agents/facilitator.py: Contains the Facilitator class that defines our model.

Utilities

• utils/partial_obs.py: Contains the PartialObsWrapper class that wraps the environment to provide partial observation for the agents. To be modified.
• utils/belief_particle.py: Contains the BeliefParticle class that represents a belief particle for both agents and the facilitator to deal with partial observation. To be modified.
• utils/feedback.py: Defines the feedback format and corresponding planner.

Data

• data/levels/: Contains the predefined level files, including the initial layout and the goal recipes for the agents.

Acknowledgments

The code is adapted from the original version of gym_cooking.

mDKG

### Installation To set up the environment for this project, first open the Julia REPL in this directory, then run the following commands in via the REPL's Pkg mode:

activate .
add https://github.com/probcomp/InversePlanning.jl.git#release-0.0.1 https://github.com/probcomp/GenGPT3.jl.git
instantiate

Usage

julia julia doors-keys-gems/evaluate_single.jl --problem_id $PROBLEM_ID --facilitator $FACILITATOR --llm_model $LLM --ablation $ABLATION --num_human_players 0

For seeing prints, add --verbose. To save logs, add --logging. The arguments are defined in arguments.jl. Our facilitator is prof.

Helper functions are in src/.

Acknowledgments

The code is based on the CLIPS.jl repository.

Citation

This is a temporary reference that will be updated after the proceedings are published.

@article{bortoletto2025protom,
  title={ProToM: Promoting Prosocial Behaviour via Theory of Mind-Informed Feedback},
  author={Bortoletto, Matteo and Zhou, Yichao and Ying, Lance and Shu, Tianmin and Bulling, Andreas},
  journal={arXiv preprint arXiv:XXXX.XXXXX},
  year={2025}
}