95 datasets found
  1. g

    Datasets for evaluation of keyword extraction in Russian

    • github.com
    Updated May 27, 2018
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    Mikhail Nefedov (2018). Datasets for evaluation of keyword extraction in Russian [Dataset]. https://github.com/mannefedov/ru_kw_eval_datasets
    Explore at:
    Dataset updated
    May 27, 2018
    Authors
    Mikhail Nefedov
    Description

    Datasets for evaluation of keyword extraction in Russian

  2. Russian ASR Open STT (public phone calls 1 and 2)

    • kaggle.com
    zip
    Updated Apr 7, 2025
    + more versions
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    alex cumder (2025). Russian ASR Open STT (public phone calls 1 and 2) [Dataset]. https://www.kaggle.com/datasets/alexcumder/audiosets
    Explore at:
    zip(14556669524 bytes)Available download formats
    Dataset updated
    Apr 7, 2025
    Authors
    alex cumder
    License

    Attribution-NonCommercial 4.0 (CC BY-NC 4.0)https://creativecommons.org/licenses/by-nc/4.0/
    License information was derived automatically

    Description

    Source https://github.com/snakers4/open_stt Include asr_public_phone_calls_1 and asr_public_phone_calls_2 in one directory. Directory asr_public_phone_calls_1/0/ сontains audio and corresponding transcripts. File dataset_target consist two columns "filename" and "text" from directory asr_public_phone_calls_1/0/.

    All files are normalized for easier / faster runtime augmentations and processing as follows: 1)Converted to mono, if necessary; 2)Converted to 16 kHz sampling rate, if necessary; 3)Stored as 16-bit integers;

  3. d

    Ukraine and Russia Conflict Tweet IDs Release v1.3

    • dataone.org
    Updated Nov 8, 2023
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    Chen, Emily; Ferrara, Emilio (2023). Ukraine and Russia Conflict Tweet IDs Release v1.3 [Dataset]. http://doi.org/10.7910/DVN/XZSYQO
    Explore at:
    Dataset updated
    Nov 8, 2023
    Dataset provided by
    Harvard Dataverse
    Authors
    Chen, Emily; Ferrara, Emilio
    Area covered
    Russia
    Description

    The repository contains an ongoing collection of tweets IDs associated with the current conflict in Ukraine and Russia, which we commenced collecting on Februrary 22, 2022. To comply with Twitter’s Terms of Service, we are only publicly releasing the Tweet IDs of the collected Tweets. The data is released for non-commercial research use. Note that the compressed files must be first uncompressed in order to use included scripts. This dataset is release v1.3 and is not actively maintained -- the actively maintained dataset can be found here: https://github.com/echen102/ukraine-russia. This release contains Tweet IDs collected from 2/22/22 - 1/08/23. Please refer to the README for more details regarding data, data organization and data usage agreement. This dataset is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License . By using this dataset, you agree to abide by the stipulations in the license, remain in compliance with Twitter’s Terms of Service, and cite the following manuscript: Emily Chen and Emilio Ferrara. 2022. Tweets in Time of Conflict: A Public Dataset Tracking the Twitter Discourse on the War Between Ukraine and Russia. arXiv:cs.SI/2203.07488

  4. Gazeta Summaries

    • kaggle.com
    zip
    Updated Sep 5, 2021
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    Ilya Gusev (2021). Gazeta Summaries [Dataset]. https://www.kaggle.com/phoenix120/gazeta-summaries
    Explore at:
    zip(193749591 bytes)Available download formats
    Dataset updated
    Sep 5, 2021
    Authors
    Ilya Gusev
    Description

    Context

    This is the first Russian news summarization dataset. A paper about this dataset: https://arxiv.org/pdf/2006.11063.pdf Additional files and notebooks: https://github.com/IlyaGusev/gazeta/ Previous datasets for headline generation: https://github.com/RossiyaSegodnya/ria_news_dataset https://www.kaggle.com/yutkin/corpus-of-russian-news-articles-from-lenta

    Content

    This is the second version of the dataset. The data structure is pretty straightforward. Every line of a file is a JSON object with 5 fields: URL, title, text, summary, and date. The dataset consists of 74126 examples. The first 60964 examples by date are in the training dataset, the proceeding 6369 examples are in the validation dataset, and the remaining 6793 pairs are in the test dataset.

    Legal issues

    Legal basis for distribution of the dataset: https://www.gazeta.ru/credits.shtml, paragraph 2.1.2. All rights belong to "www.gazeta.ru". This dataset can be removed at the request of the copyright holder. Usage of this dataset is possible only for personal purposes on a non-commercial basis.

  5. Russia - Ukraine War Tweets

    • kaggle.com
    zip
    Updated Nov 29, 2022
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    The Devastator (2022). Russia - Ukraine War Tweets [Dataset]. https://www.kaggle.com/datasets/thedevastator/invasion-of-ukraine-tweets-and-user-features
    Explore at:
    zip(19340125 bytes)Available download formats
    Dataset updated
    Nov 29, 2022
    Authors
    The Devastator
    License

    https://creativecommons.org/publicdomain/zero/1.0/https://creativecommons.org/publicdomain/zero/1.0/

    Area covered
    Ukraine, Russia
    Description

    Russia - Ukraine War Tweets

    Tweets about ongoing Russia - Ukraine war

    By [source]

    About this dataset

    This dataset consists of tweets relating to the Russian invasion of Ukraine that were scraped for this study. Only tweets of which user features were available are included in the dataset. The tweets and corresponding user features can be rehydrated using the Twitter API. However, it could be that some tweets or users might be deleted or put on private and are therefore no longer available. Moreover, user and tweet features might change over time

    More Datasets

    For more datasets, click here.

    Featured Notebooks

    • 🚨 Your notebook can be here! 🚨!

    How to use the dataset

    The dataset consists of tweets relating to the Russian invasion of Ukraine that were scraped for this study. Only tweets of which user features were available are included in the dataset. The tweets and corresponding user features can be rehydrated using the Twitter API. However, it could be that some tweets or users might be deleted or put on private and are therefore no longer available. Moreover, user and tweet features might change over time This dataset can be used to study the change in sentiment, and topics over time as the war continues

    Research Ideas

    • Find out which tweets are most popular among people interested in the Russian invasion of Ukraine
    • Identify which user attributes are associated with tweets about the Russian invasion of Ukraine
    • Study the change in sentiment and public opinion on the war as events unfold.

    Acknowledgements

    If you use this dataset in your research, please credit the original authors. Data Source

    License

    License: CC0 1.0 Universal (CC0 1.0) - Public Domain Dedication No Copyright - You can copy, modify, distribute and perform the work, even for commercial purposes, all without asking permission. See Other Information.

    Columns

    File: after_invasion_tweetids.csv | Column name | Description | |:--------------|:-----------------------| | id | The tweet id. (String) |

    File: before_invasion_tweetids.csv | Column name | Description | |:--------------|:-----------------------| | id | The tweet id. (String) |

    Acknowledgements

    If you use this dataset in your research, please credit the original authors. If you use this dataset in your research, please credit .

  6. a

    GitVerse Code Dataset

    • academictorrents.com
    bittorrent
    Updated Jan 9, 2026
    + more versions
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    nyuuzyou (2026). GitVerse Code Dataset [Dataset]. https://academictorrents.com/details/8d4375542da8412f19d7d867413e3c29eeb6f4a0
    Explore at:
    bittorrent(2113649354)Available download formats
    Dataset updated
    Jan 9, 2026
    Dataset authored and provided by
    nyuuzyou
    License

    https://academictorrents.com/nolicensespecifiedhttps://academictorrents.com/nolicensespecified

    Description

    GitVerse Code Dataset ## Dataset Description This dataset was compiled from code repositories hosted on GitVerse, a Russian code hosting platform and an alternative to GitHub in the Russian developer community. GitVerse is used by Russian developers, enterprises, and open-source projects, making this dataset particularly valuable for training code models with Russian language understanding and Russian coding conventions. ### Dataset Summary | Statistic | Value | |—————-|———-| | Total Files | 2,802,994 | | Total Repositories | 9,014 | | Total Size | 2 GB (compressed Parquet) | | Programming Languages | 416 | | File Format | Parquet (single file) | ### Key Features - Russian code corpus: Contains code from over 9,000 repositories, many featuring Russian comments, documentation, and variable names - Diverse language coverage: Spans 416 programming languages identified by [github-linguist](

  7. h

    RuREBus

    • huggingface.co
    Updated Aug 31, 2022
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    Ilya Malakhov (2022). RuREBus [Dataset]. https://huggingface.co/datasets/iluvvatar/RuREBus
    Explore at:
    Dataset updated
    Aug 31, 2022
    Authors
    Ilya Malakhov
    Description

    RuREBus dataset

      Dataset Description
    

    RuREBus dataset (https://github.com/dialogue-evaluation/RuREBus) is a Russian dataset for named entity recognition and relation extraction.

      Dataset Structure
    

    There are two subsets of the dataset. Using load_dataset('MalakhovIlya/RuREBus') you can download annotated data (DatasetDict) for named entity recognition task and relation extraction tasks. This subset consists of two splits: "train" and "test". Using… See the full description on the dataset page: https://huggingface.co/datasets/iluvvatar/RuREBus.

  8. RusTitW: Russian Language Visual Text Recognition

    • kaggle.com
    zip
    Updated Jun 9, 2024
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    Nikita (2024). RusTitW: Russian Language Visual Text Recognition [Dataset]. https://www.kaggle.com/datasets/hardtype/rustitw-russian-language-visual-text-recognition
    Explore at:
    zip(135305919719 bytes)Available download formats
    Dataset updated
    Jun 9, 2024
    Authors
    Nikita
    License

    Apache License, v2.0https://www.apache.org/licenses/LICENSE-2.0
    License information was derived automatically

    Description

    RusTitW: Russian Language Text Dataset for Visual Text in-the-Wild Recognition

    Authors: Igor Markov, Sergey Nesteruk, Andrey Kuznetsov, Denis Dimitrov

    arXiv: https://arxiv.org/abs/2303.16531

    GitHub: github.com/markovivl/SynthText

    📄Abstract

    Information surrounds people in modern life. Text is a very efficient type of information that people use for communication for centuries. However, automated text-in-the-wild recognition remains a challenging problem. The major limitation for a DL system is the lack of training data. For the competitive performance, training set must contain many samples that replicate the real-world cases. While there are many high-quality datasets for English text recognition; there are no available datasets for Russian language. In this paper, we present a large-scale human-labeled dataset for Russian text recognition in-the-wild. We also publish a synthetic dataset and code to reproduce the generation process.

    ⚙️About the data

    • Data is divided into train and test which are also splitted into real and synthetic (synth) examples.
    • For usability each folder contains info.csv file which has the same format for all splits of data.
    • Original labels and information are also preserved and can be found either in info_raw.csv or json_*_*.json files.
    • You can find duplicate images in dataset, which are not filtered from the original data. For example, some of the images are the same but have different resolution.
    • Some images from the train sample can be found in test, which is also from original.

    📍Label format

    [[{'left': 0.10259433962264151,
      'top': 0,
      'width': 0.4056603773584906,
      'height': 0.9303675048355899,
      'label': 'ALL you NEED
    is 20 SECONDS
    of Insane',
      'shape': 'rectangle'},
     {'left': 0.5141509433962265,
      'top': 0.009671179883945842,
      'width': 0.48584905660377353,
      'height': 0.5222437137330754,
      'label': 'COURAGE
    AND I PROMISE YOU
    something GREAT',
      'shape': 'rectangle'},
     {'left': 0.5165094339622641,
      'top': 0.5357833655705996,
      'width': 0.46344339622641517,
      'height': 0.31334622823984526,
      'label': 'will come of it
    Benjmin Mee',
      'shape': 'rectangle'}]]
    

    where: * left - x-axis relative left position of bbox (x_min) * top - y-axis relative top position of bbox (y_min) * width - x-axis relative width of bbox * height - y-axis relative height of bbox * label - text inside bounding box * shape - always 'rectangle'

    💻Display image and bbox:

    import pandas as pd
    import cv2
    import matplotlib.pyplot as plt
    
    
    TRAIN_PATH = 'train/real/'
    train = pd.read_csv(TRAIN_PATH + 'info.csv')
    
    idx = train.sample(1).iloc[0].name
    im = cv2.imread(TRAIN_PATH + train.iloc[idx]['image_path'])
    
    fig, ax = plt.subplots()
    
    # Display the image
    ax.imshow(im)
    
    # Create a Rectangle patch
    bboxes = json.loads(
      train.iloc[idx]['box_and_label']
    )[0]
    
    for bbox in bboxes:
      x = bbox['left']  * train.iloc[idx]['width']
      y = bbox['top']   * train.iloc[idx]['height']
      w = bbox['width']  * train.iloc[idx]['width']
      h = bbox['height'] * train.iloc[idx]['height']
      rect = patches.Rectangle((x, y), w, h, linewidth=1, edgecolor='r', facecolor='none')
    
      # Add the patch to the Axes
      ax.add_patch(rect)
    
    plt.title('
    '.join([bbox['label'] for bbox in bboxes]))
    
    plt.show()
    

    🖼️Image examples

    Human-labeled images

    https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F4480292%2Fd40f36b2ba3215770d0fc9beab9fc852%2Foutput4.png?generation=1717895975115361&alt=media" alt="image_2">

    https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F4480292%2F3909319e543566a039378e094a3144c9%2Foutput3.png?generation=1717895989389635&alt=media" alt="image_3">

    * It can be seen that data isn't perfect. The word Лого in the first picture is unlabeled. The second picture is missing the road sign signatures - 40 and 4,5 м.

    Synthetic images

    https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F4480292%2Fede8feae4c8e521409a1c8a7a4333a90%2Foutput.png?generation=1717895678470045&alt=media" alt="image_0">

    https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F4480292%2F508d10c660328c510cdd4fc66c68a5d0%2Foutput1.png?generation=1717895741343654&alt=media" alt="image_1">

  9. h

    ru-image-captions

    • huggingface.co
    Updated Apr 9, 2024
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    Svetlana Gorovaia (2024). ru-image-captions [Dataset]. https://huggingface.co/datasets/gorovuha/ru-image-captions
    Explore at:
    CroissantCroissant is a format for machine-learning datasets. Learn more about this at mlcommons.org/croissant.
    Dataset updated
    Apr 9, 2024
    Authors
    Svetlana Gorovaia
    Description

    Image Caprioning for Russian language

    This dataset is a Russian part of dinhanhx/crossmodal-3600

      Dataset Details
    

    3.11k rows. Two description for each picture. Cracked pictures were deleted from the original source. The main feature is that all the descriptions are written by the native russian speakers.

    Paper [https://google.github.io/crossmodal-3600/]

      Uses
    

    It is intended to be used for fine-tuning image captioning models.

  10. D

    Replication Data for: Analyzing GPT-4 Misinterpretations of Russian...

    • dataverse.no
    txt
    Updated Nov 1, 2024
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    Timofei Plotnikov; Timofei Plotnikov (2024). Replication Data for: Analyzing GPT-4 Misinterpretations of Russian Grammatical Constructions [Dataset]. http://doi.org/10.18710/8CAPJM
    Explore at:
    txt(309713), txt(188586), txt(3414), txt(39370), txt(480667), txt(51973), txt(87956), txt(442461)Available download formats
    Dataset updated
    Nov 1, 2024
    Dataset provided by
    DataverseNO
    Authors
    Timofei Plotnikov; Timofei Plotnikov
    License

    Attribution 4.0 (CC BY 4.0)https://creativecommons.org/licenses/by/4.0/
    License information was derived automatically

    Time period covered
    Mar 1, 2024 - Apr 5, 2024
    Area covered
    Russia
    Dataset funded by
    UiT The Arctic University of Norway
    Description

    GPT-4 interpretations of the dataset of 2,227 examples gathered from Russian Constructicon (https://constructicon.github.io/russian/)

  11. Data from: Twitter Dataset on the Russo-Ukrainian War

    • zenodo.org
    • data.niaid.nih.gov
    • +1more
    Updated Oct 20, 2023
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    Alexander Shevtsov; Alexander Shevtsov; Despoina Antonakaki; Despoina Antonakaki; Ioannis Lamprou; Sotiris Ioannidis; Sotiris Ioannidis; Polyvios Pratikakis; Polyvios Pratikakis; Ioannis Lamprou (2023). Twitter Dataset on the Russo-Ukrainian War [Dataset]. http://doi.org/10.5281/zenodo.8431047
    Explore at:
    Dataset updated
    Oct 20, 2023
    Dataset provided by
    Zenodohttp://zenodo.org/
    Authors
    Alexander Shevtsov; Alexander Shevtsov; Despoina Antonakaki; Despoina Antonakaki; Ioannis Lamprou; Sotiris Ioannidis; Sotiris Ioannidis; Polyvios Pratikakis; Polyvios Pratikakis; Ioannis Lamprou
    Time period covered
    Feb 23, 2022
    Area covered
    Ukraine
    Description

    On 24 February 2022, Russia invaded Ukraine, also known now as the Russo-Ukrainian War. We obtained our dataset through Twitter API from 23 February of 2022 until 23 June of 2023. The collected dataset has 127.275.386 tweets, shared in the form of anonymized text, where the tweet/user IDs and user mentions are anonymized and do not provide any personal information. The provided dataset contains user discussion in more than 70 languages, where the 20 most popular are : 'eng', 'fr', 'de', 'mix', 'it', 'es', 'ja', 'ru', 'pl', 'uk', 'tr', 'th', 'hi', 'qme', 'qht', 'nl', 'fi', 'ar', 'zh' and 'pt'. For the purpose of the information integrity tweets are separated and stored in different files ordered by creation date. The provided dataset is shared for further research purposes. Additionally, we provide the list of tweets IDs at the GitHub repository which can be retracted via Twitter API. Furthermore, we also manage to execute some initial analysis including: volume/activity, hashtags popularity, sentiment and military intelligence and publish the results in the web portal.

  12. a

    Mos.Hub Code Dataset

    • academictorrents.com
    bittorrent
    Updated Jan 9, 2026
    + more versions
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    nyuuzyou (2026). Mos.Hub Code Dataset [Dataset]. https://academictorrents.com/details/991f0d7eaa11bfda7f08e9bd82466458982cd430
    Explore at:
    bittorrent(554021034)Available download formats
    Dataset updated
    Jan 9, 2026
    Dataset authored and provided by
    nyuuzyou
    License

    https://academictorrents.com/nolicensespecifiedhttps://academictorrents.com/nolicensespecified

    Description

    Mos.Hub Code Dataset ## Dataset Description This dataset was compiled from code repositories hosted on Mos.Hub (hub.mos.ru), a code hosting platform operated by the Moscow Government. Mos.Hub is a service for storing and working with source code, based on the Git version control system, primarily used by Russian developers and government-related projects. ### Dataset Summary | Statistic | Value | |—————-|———-| | Total Files | 15,740,580 | | Total Repositories | 16,130 | | Total Size | 529 MB (compressed Parquet) | | Uncompressed Size | ~29 GB | | Programming Languages | 297 | | File Format | Parquet (single file) | ### Key Features - Russian code corpus: Contains code from repositories hosted on Moscow s official code platform, featuring Russian comments and documentation - Diverse language coverage: Spans 297 programming languages identified by [github-linguist](

  13. Data from: MiDe22: An Annotated Multi-Event Tweet Dataset for Misinformation...

    • zenodo.org
    • data.niaid.nih.gov
    zip
    Updated Jun 14, 2023
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    Cagri Toraman; Cagri Toraman; Oguzhan Ozcelik; Furkan Şahinuç; Fazli Can; Oguzhan Ozcelik; Furkan Şahinuç; Fazli Can (2023). MiDe22: An Annotated Multi-Event Tweet Dataset for Misinformation Detection [Dataset]. http://doi.org/10.5281/zenodo.8032136
    Explore at:
    zipAvailable download formats
    Dataset updated
    Jun 14, 2023
    Dataset provided by
    Zenodohttp://zenodo.org/
    Authors
    Cagri Toraman; Cagri Toraman; Oguzhan Ozcelik; Furkan Şahinuç; Fazli Can; Oguzhan Ozcelik; Furkan Şahinuç; Fazli Can
    License

    Attribution 4.0 (CC BY 4.0)https://creativecommons.org/licenses/by/4.0/
    License information was derived automatically

    Description

    The dataset is composed of 10,348 tweets: 5,284 for English and 5,064 for Turkish. Tweets in the dataset are human-annotated in terms of "false", "true", or "other". The dataset covers multiple topics: the Russia-Ukraine war, COVID-19 pandemic, Refugees, and additional miscellaneous events. The details can be found at https://github.com/avaapm/mide22

  14. WikiConv - Russian

    • figshare.com
    txt
    Updated May 31, 2023
    + more versions
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    Lucas Dixon; Nithum Thain; Dario Taraborelli; Cristian Danescu-Niculescu-Mizil; Jeffrey Sorensen; Yiqing Hua (2023). WikiConv - Russian [Dataset]. http://doi.org/10.6084/m9.figshare.7376015.v1
    Explore at:
    txtAvailable download formats
    Dataset updated
    May 31, 2023
    Dataset provided by
    Figsharehttp://figshare.com/
    figshare
    Authors
    Lucas Dixon; Nithum Thain; Dario Taraborelli; Cristian Danescu-Niculescu-Mizil; Jeffrey Sorensen; Yiqing Hua
    License

    Attribution 4.0 (CC BY 4.0)https://creativecommons.org/licenses/by/4.0/
    License information was derived automatically

    Description

    WikiConv (Russian): A Corpus of the Complete Conversational History of a Large Online Collaborative CommunityThis directory contains the WikiConv Corpus which encompasses the full history of conversations on Wikipedia Talk Pages.The project webpage for this work is at: https://github.com/conversationai/wikidetox/tree/master/wikiconvThe dataset and reconstruction process for this corpus has been published in the paper WikiConv: A Corpus of the Complete Conversational History of a Large OnlineCollaborative Community, presented at EMNLP 2018.The work has also been presented at the June 2018 Wikipedia researchshowcase (the first half describes our work, using an earlier version of this dataset to predict conversations going awry.The meta-data in this corpus is governed by the CC0 license v1.0, and the content of the comments is governed by the CC-SA license v3.0.

  15. Rossiyskaya Gazeta Papers (Russian legal texts)

    • kaggle.com
    zip
    Updated Feb 13, 2023
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    athugodage (2023). Rossiyskaya Gazeta Papers (Russian legal texts) [Dataset]. https://www.kaggle.com/datasets/athugodage/russian-legal-text-parallel-corpus
    Explore at:
    zip(27158041 bytes)Available download formats
    Dataset updated
    Feb 13, 2023
    Authors
    athugodage
    Area covered
    Russia
    Description

    Context

    This dataset was introduced at READi workshop (LREC-COLING 2024). This dataset was proposed to train a model for Russian legal text simplification. This dataset has already helped to train GPT and T5 models for this task, as well as in Dynamic Topic Modelling task to analyze the history of Russian law from 2009 to 2022 .

    We have collected our data from Rossiyskaya Gazeta website. It's a Russian newspaper published by the Government of Russia. The daily newspaper serves as the official government gazette of the Government of the Russian Federation, publishing government-related affairs such as official decrees, statements and documents of state bodies, the promulgation of newly approved laws, Presidential decrees, and government announcements. Rossiyskaya Gazeta provides legal text descriptions for common people called "comments". But these descriptions are made only for important documents, so while there are hundreds of thousands of legal documents in Russia, only a couple of thousands has a "comment". We used this "comment" as simplified version of the document.

    Content

    Overall there are 2963 pairs of original documents and simplified ones. Dataset contains documents from December 31, 2008 up to November 28, 2022 - thus it contains COVID-19-related laws too.

    Dataset has 5 columns: 1. Название документа (Document Title) 2. Ссылка (Link to the original document) 3. Текст (Original document text) 4. Комментарий РГ (Rossiyskaya Gazeta comment) 5. Дата (Publication date)

    Example of original document text (2nd article in row): https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F11047041%2F625daafe6319319700f754f5ac1d89e4%2Forig.png?generation=1676324147067453&alt=media" alt="">

    Example of Rossiyskaya Gazeta comment: https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F11047041%2F3230f4df60713b417826434ddacc8882%2Fcomm.png?generation=1676324185146892&alt=media" alt="">

    The photo on the headline was taken from Roscosmos official website. `

  16. Z

    Data from: Russian Financial Statements Database: A firm-level collection of...

    • data.niaid.nih.gov
    Updated Mar 14, 2025
    + more versions
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    Bondarkov, Sergey; Ledenev, Victor; Skougarevskiy, Dmitriy (2025). Russian Financial Statements Database: A firm-level collection of the universe of financial statements [Dataset]. https://data.niaid.nih.gov/resources?id=zenodo_14622208
    Explore at:
    Dataset updated
    Mar 14, 2025
    Dataset provided by
    European University at St. Petersburg
    European University at St Petersburg
    Authors
    Bondarkov, Sergey; Ledenev, Victor; Skougarevskiy, Dmitriy
    License

    Attribution-NonCommercial-ShareAlike 4.0 (CC BY-NC-SA 4.0)https://creativecommons.org/licenses/by-nc-sa/4.0/
    License information was derived automatically

    Description

    The Russian Financial Statements Database (RFSD) is an open, harmonized collection of annual unconsolidated financial statements of the universe of Russian firms:

    • 🔓 First open data set with information on every active firm in Russia.

    • 🗂️ First open financial statements data set that includes non-filing firms.

    • 🏛️ Sourced from two official data providers: the Rosstat and the Federal Tax Service.

    • 📅 Covers 2011-2023 initially, will be continuously updated.

    • 🏗️ Restores as much data as possible through non-invasive data imputation, statement articulation, and harmonization.

    The RFSD is hosted on 🤗 Hugging Face and Zenodo and is stored in a structured, column-oriented, compressed binary format Apache Parquet with yearly partitioning scheme, enabling end-users to query only variables of interest at scale.

    The accompanying paper provides internal and external validation of the data: http://arxiv.org/abs/2501.05841.

    Here we present the instructions for importing the data in R or Python environment. Please consult with the project repository for more information: http://github.com/irlcode/RFSD.

    Importing The Data

    You have two options to ingest the data: download the .parquet files manually from Hugging Face or Zenodo or rely on 🤗 Hugging Face Datasets library.

    Python

    🤗 Hugging Face Datasets

    It is as easy as:

    from datasets import load_dataset import polars as pl

    This line will download 6.6GB+ of all RFSD data and store it in a 🤗 cache folder

    RFSD = load_dataset('irlspbru/RFSD')

    Alternatively, this will download ~540MB with all financial statements for 2023# to a Polars DataFrame (requires about 8GB of RAM)

    RFSD_2023 = pl.read_parquet('hf://datasets/irlspbru/RFSD/RFSD/year=2023/*.parquet')

    Please note that the data is not shuffled within year, meaning that streaming first n rows will not yield a random sample.

    Local File Import

    Importing in Python requires pyarrow package installed.

    import pyarrow.dataset as ds import polars as pl

    Read RFSD metadata from local file

    RFSD = ds.dataset("local/path/to/RFSD")

    Use RFSD_dataset.schema to glimpse the data structure and columns' classes

    print(RFSD.schema)

    Load full dataset into memory

    RFSD_full = pl.from_arrow(RFSD.to_table())

    Load only 2019 data into memory

    RFSD_2019 = pl.from_arrow(RFSD.to_table(filter=ds.field('year') == 2019))

    Load only revenue for firms in 2019, identified by taxpayer id

    RFSD_2019_revenue = pl.from_arrow( RFSD.to_table( filter=ds.field('year') == 2019, columns=['inn', 'line_2110'] ) )

    Give suggested descriptive names to variables

    renaming_df = pl.read_csv('local/path/to/descriptive_names_dict.csv') RFSD_full = RFSD_full.rename({item[0]: item[1] for item in zip(renaming_df['original'], renaming_df['descriptive'])})

    R

    Local File Import

    Importing in R requires arrow package installed.

    library(arrow) library(data.table)

    Read RFSD metadata from local file

    RFSD <- open_dataset("local/path/to/RFSD")

    Use schema() to glimpse into the data structure and column classes

    schema(RFSD)

    Load full dataset into memory

    scanner <- Scanner$create(RFSD) RFSD_full <- as.data.table(scanner$ToTable())

    Load only 2019 data into memory

    scan_builder <- RFSD$NewScan() scan_builder$Filter(Expression$field_ref("year") == 2019) scanner <- scan_builder$Finish() RFSD_2019 <- as.data.table(scanner$ToTable())

    Load only revenue for firms in 2019, identified by taxpayer id

    scan_builder <- RFSD$NewScan() scan_builder$Filter(Expression$field_ref("year") == 2019) scan_builder$Project(cols = c("inn", "line_2110")) scanner <- scan_builder$Finish() RFSD_2019_revenue <- as.data.table(scanner$ToTable())

    Give suggested descriptive names to variables

    renaming_dt <- fread("local/path/to/descriptive_names_dict.csv") setnames(RFSD_full, old = renaming_dt$original, new = renaming_dt$descriptive)

    Use Cases

    🌍 For macroeconomists: Replication of a Bank of Russia study of the cost channel of monetary policy in Russia by Mogiliat et al. (2024) — interest_payments.md

    🏭 For IO: Replication of the total factor productivity estimation by Kaukin and Zhemkova (2023) — tfp.md

    🗺️ For economic geographers: A novel model-less house-level GDP spatialization that capitalizes on geocoding of firm addresses — spatialization.md

    FAQ

    Why should I use this data instead of Interfax's SPARK, Moody's Ruslana, or Kontur's Focus?hat is the data period?

    To the best of our knowledge, the RFSD is the only open data set with up-to-date financial statements of Russian companies published under a permissive licence. Apart from being free-to-use, the RFSD benefits from data harmonization and error detection procedures unavailable in commercial sources. Finally, the data can be easily ingested in any statistical package with minimal effort.

    What is the data period?

    We provide financials for Russian firms in 2011-2023. We will add the data for 2024 by July, 2025 (see Version and Update Policy below).

    Why are there no data for firm X in year Y?

    Although the RFSD strives to be an all-encompassing database of financial statements, end users will encounter data gaps:

    We do not include financials for firms that we considered ineligible to submit financial statements to the Rosstat/Federal Tax Service by law: financial, religious, or state organizations (state-owned commercial firms are still in the data).

    Eligible firms may enjoy the right not to disclose under certain conditions. For instance, Gazprom did not file in 2022 and we had to impute its 2022 data from 2023 filings. Sibur filed only in 2023, Novatek — in 2020 and 2021. Commercial data providers such as Interfax's SPARK enjoy dedicated access to the Federal Tax Service data and therefore are able source this information elsewhere.

    Firm may have submitted its annual statement but, according to the Uniform State Register of Legal Entities (EGRUL), it was not active in this year. We remove those filings.

    Why is the geolocation of firm X incorrect?

    We use Nominatim to geocode structured addresses of incorporation of legal entities from the EGRUL. There may be errors in the original addresses that prevent us from geocoding firms to a particular house. Gazprom, for instance, is geocoded up to a house level in 2014 and 2021-2023, but only at street level for 2015-2020 due to improper handling of the house number by Nominatim. In that case we have fallen back to street-level geocoding. Additionally, streets in different districts of one city may share identical names. We have ignored those problems in our geocoding and invite your submissions. Finally, address of incorporation may not correspond with plant locations. For instance, Rosneft has 62 field offices in addition to the central office in Moscow. We ignore the location of such offices in our geocoding, but subsidiaries set up as separate legal entities are still geocoded.

    Why is the data for firm X different from https://bo.nalog.ru/?

    Many firms submit correcting statements after the initial filing. While we have downloaded the data way past the April, 2024 deadline for 2023 filings, firms may have kept submitting the correcting statements. We will capture them in the future releases.

    Why is the data for firm X unrealistic?

    We provide the source data as is, with minimal changes. Consider a relatively unknown LLC Banknota. It reported 3.7 trillion rubles in revenue in 2023, or 2% of Russia's GDP. This is obviously an outlier firm with unrealistic financials. We manually reviewed the data and flagged such firms for user consideration (variable outlier), keeping the source data intact.

    Why is the data for groups of companies different from their IFRS statements?

    We should stress that we provide unconsolidated financial statements filed according to the Russian accounting standards, meaning that it would be wrong to infer financials for corporate groups with this data. Gazprom, for instance, had over 800 affiliated entities and to study this corporate group in its entirety it is not enough to consider financials of the parent company.

    Why is the data not in CSV?

    The data is provided in Apache Parquet format. This is a structured, column-oriented, compressed binary format allowing for conditional subsetting of columns and rows. In other words, you can easily query financials of companies of interest, keeping only variables of interest in memory, greatly reducing data footprint.

    Version and Update Policy

    Version (SemVer): 1.0.0.

    We intend to update the RFSD annualy as the data becomes available, in other words when most of the firms have their statements filed with the Federal Tax Service. The official deadline for filing of previous year statements is April, 1. However, every year a portion of firms either fails to meet the deadline or submits corrections afterwards. Filing continues up to the very end of the year but after the end of April this stream quickly thins out. Nevertheless, there is obviously a trade-off between minimization of data completeness and version availability. We find it a reasonable compromise to query new data in early June, since on average by the end of May 96.7% statements are already filed, including 86.4% of all the correcting filings. We plan to make a new version of RFSD available by July.

    Licence

    Creative Commons License Attribution 4.0 International (CC BY 4.0).

    Copyright © the respective contributors.

    Citation

    Please cite as:

    @unpublished{bondarkov2025rfsd, title={{R}ussian {F}inancial {S}tatements {D}atabase}, author={Bondarkov, Sergey and Ledenev, Victor and Skougarevskiy, Dmitriy}, note={arXiv preprint arXiv:2501.05841}, doi={https://doi.org/10.48550/arXiv.2501.05841}, year={2025}}

    Acknowledgments and Contacts

    Data collection and processing: Sergey Bondarkov, sbondarkov@eu.spb.ru, Viktor Ledenev, vledenev@eu.spb.ru

    Project conception, data validation, and use cases: Dmitriy Skougarevskiy, Ph.D.,

  17. h

    sakha-russian-parallel

    • huggingface.co
    Updated Nov 21, 2025
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    Artificial Intelligence Laboratory of the Republic of Sakha (Yakutia) (2025). sakha-russian-parallel [Dataset]. https://huggingface.co/datasets/ailabykt/sakha-russian-parallel
    Explore at:
    Dataset updated
    Nov 21, 2025
    Dataset authored and provided by
    Artificial Intelligence Laboratory of the Republic of Sakha (Yakutia)
    License

    Attribution 4.0 (CC BY 4.0)https://creativecommons.org/licenses/by/4.0/
    License information was derived automatically

    Description

    The texts in the sah column were generated using OCR and may contain errors or artifacts. Please take this into account when using the data for training or evaluation. The dataset was aligned using the Lingtrain Aligner library (https://github.com/averkij/lingtrain-aligner), created by @averoo

  18. A

    ‘Russia Covid Vaccination’ analyzed by Analyst-2

    • analyst-2.ai
    Updated Sep 30, 2021
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    Analyst-2 (analyst-2.ai) / Inspirient GmbH (inspirient.com) (2021). ‘Russia Covid Vaccination’ analyzed by Analyst-2 [Dataset]. https://analyst-2.ai/analysis/kaggle-russia-covid-vaccination-f17a/6bc77f10/?iid=004-781&v=presentation
    Explore at:
    Dataset updated
    Sep 30, 2021
    Dataset authored and provided by
    Analyst-2 (analyst-2.ai) / Inspirient GmbH (inspirient.com)
    License

    Attribution 4.0 (CC BY 4.0)https://creativecommons.org/licenses/by/4.0/
    License information was derived automatically

    Area covered
    Russia
    Description

    Analysis of ‘Russia Covid Vaccination’ provided by Analyst-2 (analyst-2.ai), based on source dataset retrieved from https://www.kaggle.com/rajkumarl/russia-covid-vaccination on 30 September 2021.

    --- Dataset description provided by original source is as follows ---

    Context

    Coronavirus spread and the deaths caused are mitigated in Russia by Vaccination and other measures. Vaccinations are provided to people in two doses as like the rest of the world. This dataset provides data of number of people vaccinated in Russia in a daily basis. This dataset is updated in a weekly basis.

    Content

    The features available are Date, The source of data (URL), The type of vaccine, Cumulative number of people vaccinated daily both the first dose and the second dose (in three separate features: aggregate, first dose, second dose).

    Acknowledgements

    Our World in Data https://ourworldindata.org/covid-vaccinations https://github.com/owid/covid-19-data

    Inspiration

    The trend of vaccination will help accurately predict the future of coronavirus spread along with https://www.kaggle.com/rajkumarl/coronavirus-spread-global (coronavirus spread data).

    --- Original source retains full ownership of the source dataset ---

  19. m

    Sanctions and Russian Online Prices - Replication Data

    • data.mendeley.com
    Updated Aug 20, 2024
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    Luigi Palumbo (2024). Sanctions and Russian Online Prices - Replication Data [Dataset]. http://doi.org/10.17632/n3zjsfdbvr.2
    Explore at:
    Dataset updated
    Aug 20, 2024
    Authors
    Luigi Palumbo
    License

    Attribution-NonCommercial-ShareAlike 4.0 (CC BY-NC-SA 4.0)https://creativecommons.org/licenses/by-nc-sa/4.0/
    License information was derived automatically

    Description

    Data on prices and quantity available for sale (weekly averages) acquired via web scraping from a Russian retail chain. Part of the replication package for the paper "Sanctions and Russian Online Prices", published on Journal of Economic Behavior and Organization (2024). Replication code also available on GitHub: https://github.com/paluigi/sanctions-russian-online-prices

    Please cite this work as: Benchimol, J., and Palumbo, L., 2024. Sanctions and Russian online prices. Journal of Economic Behavior & Organization, vol. 225, pages 483-521. doi: 10.1016/j.jebo.2024.07.013

    Bibtex entry: @article{BenchimolPalumbo2024a, title={{Sanctions and Russian online prices}}, author={Benchimol, Jonathan and Palumbo, Luigi}, journal={Journal of Economic Behavior & Organization}, year=2024, volume={225}, pages={483-521}}

  20. Z

    Database of Russian names, surnames and midnames for gender identification

    • data.niaid.nih.gov
    Updated Jan 24, 2020
    Share
    FacebookFacebook
    TwitterTwitter
    Email
    Click to copy link
    Link copied
    Close
    Cite
    Ivan Begtin (2020). Database of Russian names, surnames and midnames for gender identification [Dataset]. https://data.niaid.nih.gov/resources?id=zenodo_2747010
    Explore at:
    Dataset updated
    Jan 24, 2020
    Dataset provided by
    Infoculture
    Authors
    Ivan Begtin
    License

    Attribution 4.0 (CC BY 4.0)https://creativecommons.org/licenses/by/4.0/
    License information was derived automatically

    Area covered
    Russia
    Description

    Database of names, surnames and midnames across the Russian federation used as source to teach algorithms for gender identification by fullname.

    Dataset prepared for MongoDB database. It has MongoDB dump and dump of tables as JSON lines files.

    Used in gender identification and fullname parsing software https://github.com/datacoon/russiannames

    Available under Creative Commons CC-BY SA by default.

Share
FacebookFacebook
TwitterTwitter
Email
Click to copy link
Link copied
Close
Cite
Mikhail Nefedov (2018). Datasets for evaluation of keyword extraction in Russian [Dataset]. https://github.com/mannefedov/ru_kw_eval_datasets

Datasets for evaluation of keyword extraction in Russian

Explore at:
Dataset updated
May 27, 2018
Authors
Mikhail Nefedov
Description

Datasets for evaluation of keyword extraction in Russian

Search
Clear search
Close search
Google apps
Main menu