Ask a librarian

The material consists of TF-IDF data matrices for machine learning, compiled from the "Ask a Librarian" question/answer corpus. The corpus is in Finnish. The data matrices are particularly suitable for so-called class-rich data arrays. Extreme Multi-label Text Classification (XMTC) machine learning model training.

The original corpus contains 3150 short documents in Finnish from the Ask a Librarian service. Each document is a question asked by the public and answered by a librarian.

The corpus was picked from a collection of over 25,000 question/answer pairs, limiting the document to at least 4 subjects.

The corpus is divided into the following directories:

all: contains all documents (N = 3150)
train: includes questions asked before 2016 (N = 2625) for training purposes
maui-train: random sample set (N = 200) from the train directory for training the Maui model
validate: contains questions asked in 2016 (N = 213) for validation (e.g. selecting hyperparameters for the classifier)
test: includes the questions raised in 2017 (N = 312) for the final evaluation

The original corpus is available at https://github.com/NatLibFi/Annif-corpora/tree/master/fulltext/librarian

The actual Ask a Librarian service can be found at https://www.kirjastot.fi/ask. Libraries.fi editorial staff is responsible for the development and maintenance of the service.