Active Evaluation Software for Selection of Ground Truth Labels
Description
This software repository contains a python package Aegis (Active Evaluator Germane Interactive Selector) package that allows us to evaluate machine learning systems's performance (according to a metric such as accuracy) by adaptively sampling trials to label from an unlabeled test set to minimize the number of labels needed. This includes sample (public) data as well as a simulation script that tests different label-selecting strategies on already labelled test sets. This software is configured so that users can add their own data and system outputs to test evaluation.
Resources
| Name |
Format |
Description |
Link |
|
0 |
|
https://doi.org/10.18434/M32227 |
Tags
- ar
- active-evaluation
- machine-learning