Adding an experiment¶

Add your experiment into https://github.com/baumanno/psqe-2024/tree/publish/qexp/experiments.

You probably want to base any new experiments off of experiment2.py, where all documents in the corpus are indexed by the underlying search engine.

Based on the infrastructure created in Adding an expansion source, probably the following specs should be taken to include a new experiment:

New experiment_X.py¶

1. line 18: build the new registry for your expansion source
2. line 22: build the new Searcher, or modify the current constructor call to accomodate for the new registry

The construction of Search (l. 17), SearchSpecs (l. 20), as well as writing out the results can remain the same.

Export new experiment¶

Add your experiment to the module's init.py:

from .experiment_X import main as experimentX

Register new experiment¶

Add a subparser for the new experiment to the main entrypoint:

    # all args are passed to `experimentX.main` and read from there
    parser_exp_X = subparsers.add_parser(
        "experimentX", help="Run experiment with a different expansion source"
    )

    # this line may not be needed, depending on the structure of the new registry
    # parser_exp_X.add_argument("--model-path", type=str)
    parser_exp_X.add_argument("--results-path", type=str)
    parser_exp_X.set_defaults(func=experiments.experimentX)

You can then run your experiment with:

$ python main.py \
    experimentX \
    # additional args \
    --results-path ./results/experimentX.csv