SummAct: Uncovering User Intentions Through Interactive Behaviour Summarisation
**[Guanhua Zhang][4], [Mohamed Ahmed][3], [Zhiming Hu][5], [Andreas Bulling][6]**
**ACM CHI 2025**, Yokohama, Japan
**[[Project][2]]** **[[Paper][7]]**
----------------
# Directory Structure
```
SummAct
│ README.md
│ environment.yml
│
└───preprocess
│ convert_dataset.py
│ create_steps.py
│
└───hf_bmt
│ hf_2_bmtrain.py
│ hf_2_bmtrain.sh
│ bmt_hf.py
│
└───train
│ train.py
│ train.sh
│
└───inference
│ inference.py
│ inference.sh
```
# Setup
We recommend setting up a virtual environment using Anaconda.
1. Create a conda environment and install dependencies
```shell
conda env create --name summact --file=environment.yml
conda activate summact
```
2. Since `model_center==1.0.3` is needed but is not yet available on PYPI, build from [source](https://github.com/OpenBMB/ModelCenter)
```
$ git clone https://github.com/OpenBMB/ModelCenter.git
$ cd ModelCenter
$ pip install -r requirements.txt
$ python3 setup.py install
```
3. Clone our repository to download our code and a pretrained model
```shell
git clone this_repo.git
```
# Preprocessing
1. Convert actions from symbolic formats to natural language by running `preprocess/convert_dataset.py`. Adapt it to your local dataset paths.
2. Prompting the pretrained LLM with examples to generate sub-intentions using `preprocess/create_steps.py`. Adapt it to your local prompt txt path.
# Fine-tuning
1. After downloading the model from Hugging Face, convert it into `model_center` weights using the script in `hf_bmt/hf_2_bmtrain.sh`. Adapt it to your local paths of the downloaded model and the wanted output.
2. Run `train/train.sh`, which will call `train/train.py` to fine-tune the model for interactive behaviour summarisation. Make sure your computer has GPUs.
# Inference
Run `inference/inference.sh`, which will call `inference/inference.py` to convert the fine-tuned model back to HF format, and then calculate metrics to evaluate the summarisation quality.
# Citation
If you find our code useful or use it in your own projects, please cite our paper:
```
@inproceedings{zhang25_chi,
title = {SummAct: Uncovering User Intentions Through Interactive Behaviour Summarisation},
author = {Zhang, Guanhua and Ahmed, Mohamed and Hu, Zhiming and Bulling, Andreas},
year = {2025},
pages = {1--17},
booktitle = {Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)},
doi = {10.1145/3706598.3713190}
}
```
# Acknowledgements
Our work relied on the codebase of [Mind2Web][1], [ScreenAgent][8] and [Tell Me More!][9]. Thanks to the authors for sharing their code.
[1]: https://osu-nlp-group.github.io/Mind2Web/
[2]: https://collaborative-ai.org/publications/zhang25_chi/
[3]: https://www.linkedin.com/in/mohamed-adel-naguib/
[4]: https://scholar.google.com/citations?user=NqkK0GwAAAAJ&hl=en
[5]: https://scholar.google.com/citations?hl=en&user=OLB_xBEAAAAJ
[6]: https://www.collaborative-ai.org/people/bulling/
[7]: https://collaborative-ai.org/publications/zhang25_chi.pdf
[8]: https://github.com/niuzaisheng/ScreenAgent
[9]: https://github.com/OpenBMB/Tell_Me_More