Limits of Theory of Mind Modelling in Dialogue-Based Collaborative Plan Acquisition

Matteo Bortoletto,   Constantin Ruhdorfer,   Adnen Abdessaied,   Lei Shi,   Andreas Bulling

ACL 2024, Bangkok, Thailand
[Paper]

Citation

@inproceedings{bortoletto24_acl,
  author = {Bortoletto, Matteo and Ruhdorfer, Constantin and Abdessaied, Adnen and Shi, Lei and Bulling, Andreas},
  title = {Limits of Theory of Mind Modelling in Dialogue-Based Collaborative Plan Acquisition},
  booktitle = {Proc. 62nd Annual Meeting of the Association for Computational Linguistics (ACL)},
  year = {2024},
  pages = {1--16},
  doi = {}
}

Dataset

Original Dataset

Extended Dataset

Code overview

The code is based on the original implementation.

ToM tasks

Baselines:

• training code: baselines_with_dialogue_moves.py
• bash script: baselines_with_dialogue_moves.sh
• model: src/models/model_with_dialogue_moves.py

Graph models:

• training code: baselines_with_dialogue_moves_graphs.py
• bash script: baselines_with_dialogue_moves_graphs.sh
• model: src/models/model_with_dialogue_moves_graphs.py

To extract ToM features, run intermediate_representations.py

CPA tasks

Baselines:

• training code: plan_predictor.py
• bash script: run_plan_predictor.sh
• model: src/models/plan_model.py

Baselines with ToM ground-truth as input:

• trainining code: plan_predictor_oracle.py
• bash script: run_plan_predictor_oracle.sh
• model: src/models/plan_model_oracle.py

Graph models:

• training code: plan_predictor_graphs.py
• bash script: run_plan_predictor_graphs.sh
• model: src/models/plan_model_graphs.py

Graph models with ToM ground-truth as input:

• training code: plan_predictor_graphs_oracle.py
• bash script: run_plan_predictor_graphs_oracle.sh
• model: src/models/plan_model_graphs_oracle.py

Graph models with full candidate sampling (i.e. considering all possible candidate edges):

• training code: plan_predictor_graphs_allcand.py
• bash script: run_plan_predictor_graphs_allcand.sh
• model: src/models/plan_model_graphs.py (same as the constrained selection)

Train graph model with int0 features as input: run_plan_predictor_graphs_int0.sh

Analysis

• correlation analysis: compare_tom_cpa.ipynb
• diagnostic probing: logistic_regression_tom_feats.py. To extract CPA features, use plan_predictor_graphs_test.py