add all query masks

This commit is contained in:
Anna Penzkofer 2024-04-29 17:38:37 +02:00
parent e3efa4fc52
commit 0c6e0d42b4
2 changed files with 15 additions and 3 deletions

View file

@ -39,18 +39,30 @@ trainval_balanced_programs.json
> GQA dictionaries: `gqa_all_attributes.json` and `gqa_all_vocab_classes` are also adapted from [https://github.com/wenhuchen/Meta-Module-Network](https://github.com/wenhuchen/Meta-Module-Network) > GQA dictionaries: `gqa_all_attributes.json` and `gqa_all_vocab_classes` are also adapted from [https://github.com/wenhuchen/Meta-Module-Network](https://github.com/wenhuchen/Meta-Module-Network)
## Generate Query Masks ## Generate Query Masks
- generates full_relations_df.pkl if not already present > All 37 generated query masks are available in [relations.zip](relations.zip) as numpy files
- generates query masks for all relations with more than 1000 samples
```python
# loading a query mask with numpy
import numpy as np
rel = 'to_the_right_of'
path = f'relations/{rel}.npy'
mask = np.load(path)
mask = mask > 0.05 # binary mask
```
If you want to run the generation process, run:
```shell ```shell
python generate_query_masks.py python generate_query_masks.py
``` ```
- generates full_relations_df.pkl if not already present
- generates query masks for all relations with more than 1000 samples
## Pipeline ## Pipeline
Execute Pipeline for all samples in GQA: train_balanced (with `TEST=False`) or validation_balanced (with `TEST=True`) Execute Pipeline for all samples in GQA: train_balanced (with `TEST=False`) or validation_balanced (with `TEST=True`)
```shell ```shell
python run_programs.py python run_programs.py
``` ```
For visualizing samples with full all pipeline steps see [VSA4VQA_examples.ipynb](VSA4VQA_examples.ipynb) <br> For visualizing samples with all pipeline steps see [VSA4VQA_examples.ipynb](VSA4VQA_examples.ipynb) <br>
## Citation ## Citation
Please consider citing this paper if you use VSA4VQA or parts of this publication in your research: Please consider citing this paper if you use VSA4VQA or parts of this publication in your research:

BIN
relations.zip Normal file

Binary file not shown.