• Cleaned_Dataset.csv – The combined CSV files of all scraped documents from DABI, e-LiS, o-bib and Springer.
• Data_Cleaning.ipynb – The Jupyter Notebook with python code for the analysis and cleaning of the original dataset.
• ger_train.csv – The German training set as CSV file.
• ger_validation.csv – The German validation set as CSV file.
• en_test.csv – The English test set as CSV file.
• en_train.csv – The English training set as CSV file.
• en_validation.csv – The English validation set as CSV file.
• splitting.py – The python code for splitting a dataset into train, test and validation set.
• DataSetTrans_de.csv – The final German dataset as a CSV file.
• DataSetTrans_en.csv – The final English dataset as a CSV file.
• translation.py – The python code for translating the cleaned dataset.

The Blog-1K corpus is a redistributable authorship identification testbed for contemporary English prose. It has 1,000 candidate authors, 16K+ posts, and a pre-defined data split (train/dev/test proportional to ca. 8:1:1). It is a subset of the Blog Authorship Corpus from Kaggle. The MD5 for Blog-1K is '0a9e38740af9f921b6316b7f400acf06'.

1. Preprocessing

We first filter out texts shorter than 1,000 characters. Then we select one thousand authors whose writings meet the following criteria: - accumulatively at least 10,000 characters, - accumulatively at most 49,410 characters, - accumulatively at least 16 posts, - accumulatively at most 40 posts, and - each text has at least 50 function words found in the Koppel512 list (to filter out non-English prose).

Blog-1K has three columns: 'id', 'text', and 'split', where 'id' corresponds to its parent corpus.

1. Statistics

Its creation and statistics can be found in the Jupyter Notebook.

    Split
    # Authors
    # Posts
    # Characters
    Avg. Characters Per Author (Std.)
    Avg. Characters Per Post (Std.)


    Train
    1,000
    16,132
    30,092,057
    30,092 (5,884)
    1,865 (1,007)


    Validation
    935
    2,017
    3,755,362
    4,016 (2,269)
    1,862 (999)


    Test
    924
    2,017
    3,732,448
    4,039 (2,188)
    1,850 (936)

1. Usage

import pandas as pd

df = pd.read_csv('blog1000.csv.gz', compression='infer')

read in training data

train_text, train_label = zip(*df.loc[df.split=='train'][['text', 'id']].itertuples(index=False))

1. License All the materials is licensed under the ISC License.

2. Contact Please contact its maintainer for questions.