Functional Verification for Pre-Alignment Filtering in Memory used for the evaluation of the papers titled RattlesnakeJake: A Fast and Accurate Pre-Alignment Filter Suitable for Computation-in-Memory, SieveMem : A Computation-in-Memory Architecture for Fast and Accurate Pre-Alignment , and the M.Sc. thesis of Michael Miao at QCE department.

Analytical Model for Pre-Alignment Filtering in Memory used for the evaluation of the paper titled RattlesnakeJake: A Fast and Accurate Pre-Alignment Filter Suitable for Computation-in-Memory and the M.Sc. thesis of Michael Miao at QCE department. This analytical model is used to calculate the resource requirements (tile sizes, etc.) for the architectures in the research presented in the paper and the thesis given a genomic dataset.

Meteorological data recorded at all stations of the Network of Automatic Meteorological Stations (XEMA) of the Meteorological Service of Catalonia. This base date contains variables measured less frequently than daily, usually semi-hourly. The entire dataset could be downloaded as CSV file but because of its large size in-memory filtering could be difficult if it is not loaded into SQL database first. Data are even available within Socrata API access which allows to filter by station, variable, time range, etc.

The semiconductor gas filter market is experiencing robust growth, projected to reach a market size of $1531 million in 2025 and exhibiting a Compound Annual Growth Rate (CAGR) of 9.5% from 2025 to 2033. This expansion is driven by the increasing demand for advanced semiconductor devices in various applications, including electronics, solar energy, and data centers. The rising complexity of semiconductor manufacturing processes necessitates highly efficient and precise gas filtration systems to maintain product purity and yield. Key trends shaping this market include the adoption of advanced filter materials like nickel and stainless steel to meet stringent purity requirements and the growing focus on minimizing environmental impact through sustainable filter design and disposal methods. While increasing raw material costs present a restraint, the long-term outlook remains positive, fueled by continuous advancements in semiconductor technology and the expanding global semiconductor industry. The market is segmented by filter type (Point-of-Use Filters, Surface Mount Filters, Stainless Steel Gas Filter, Nickel Gas Filter, AMC Filter, Gasket Filters, Other) and application (Semiconductor Foundry Manufacturing, Memory Manufacturing, Solar Semiconductor Manufacturing), reflecting the diverse needs of different semiconductor manufacturing processes. Major players, including Pall, Entegris, and others, are actively investing in research and development to enhance filter performance and expand their product portfolios to capture this significant market opportunity. The significant players in this market are leveraging strategic partnerships and acquisitions to strengthen their market presence and enhance their product offerings. The geographical distribution reveals a strong presence across North America, Europe, and Asia Pacific, reflecting the concentration of semiconductor manufacturing facilities in these regions. Growth in emerging economies, particularly in Asia Pacific, further contributes to the overall expansion of the market. Future growth will depend on technological advancements in filter technology, the development of more sustainable and cost-effective filtration solutions, and the continuous expansion of the global semiconductor industry. The increasing focus on reducing particle contamination and improving process efficiency within semiconductor fabrication plants is a primary catalyst for growth within the forecast period.

The global market for filters in the semiconductor industry is poised for significant expansion, projected to reach an estimated USD 1188.6 million by 2025 and subsequently grow at a robust Compound Annual Growth Rate (CAGR) of 10.0% during the forecast period of 2025-2033. This remarkable growth trajectory is primarily fueled by the escalating demand for advanced semiconductor devices across various sectors, including consumer electronics, automotive, and artificial intelligence. The increasing complexity and miniaturization of semiconductor manufacturing processes necessitate stringent purity standards, driving the adoption of sophisticated filtration solutions for both gases and liquids. Emerging trends such as the development of next-generation microchips, the expansion of 5G infrastructure, and the surging demand for high-performance computing are further augmenting the need for highly effective filtration technologies to prevent contamination and ensure product yield and reliability. The market is strategically segmented into two primary types: Semiconductor Gas Filters and Semiconductor Liquid Filters, catering to the distinct purification needs within semiconductor fabrication. Key applications driving this demand include Semiconductor Foundry Manufacturing, Memory Manufacturing, and Solar Semiconductor Manufacturing, all of which rely heavily on ultra-pure process materials. Geographically, the Asia Pacific region is anticipated to dominate the market, driven by its strong manufacturing base, particularly in China, Japan, and South Korea, coupled with substantial investments in advanced semiconductor production facilities. North America and Europe also represent significant markets, bolstered by ongoing research and development initiatives and the presence of major semiconductor manufacturers. While the market presents a compelling growth opportunity, potential restraints such as the high cost of advanced filtration systems and the need for specialized maintenance could pose challenges for some market players, necessitating strategic investments in technological innovation and cost optimization. The semiconductor filters market is characterized by a high concentration of end-users within key manufacturing hubs, particularly in Asia (e.g., Taiwan, South Korea, China) and North America (e.g., USA). This concentration stems from the significant capital investment required for wafer fabrication plants (fabs). Innovation in this sector is driven by the relentless pursuit of higher purity, smaller feature sizes, and increased yields. Key characteristics of innovation include the development of advanced membrane materials capable of trapping sub-nanometer particles, novel cleaning methodologies for filter longevity, and smart filter technologies for real-time contamination monitoring. The impact of regulations, such as stringent environmental protection laws and increasingly demanding quality standards from chip manufacturers like TSMC and Intel, directly influences filter design and material selection. These regulations mandate lower particle counts and chemical purity, pushing filter manufacturers to invest heavily in R&D. Product substitutes are limited in highly specialized semiconductor applications, where the performance and reliability of dedicated filters are paramount. However, in less critical areas, more generalized filtration solutions might be considered, though they rarely meet the ultra-high purity requirements. End-user concentration is a defining feature, with a few dominant chip manufacturers accounting for a substantial portion of filter demand. This creates strong relationships between filter suppliers and their customers, fostering co-development and long-term supply agreements. The level of M&A activity within the semiconductor filter industry is moderate to high, driven by the desire of larger players like Entegris and Pall to expand their product portfolios, gain access to new technologies, and consolidate market share. Smaller, innovative companies are often acquired to accelerate their growth and market penetration.

Required amounts of memory for the pre-filter.

Explore the booming Semiconductor Filter market analysis, CAGR 12%, drivers, trends, restraints, and key players. Discover market size projections for Semiconductor Foundry Manufacturing, Memory Manufacturing, and Solar Semiconductor Manufacturing.

Economic analysis is often based on pre-filtered, de-trended, or seasonally adjusted data. Underlying filtering methods make strong assumptions about the memory of the series to be filtered, and inference about the memory is limited particularly when persistent cyclical variation overshadows the trend. This article introduces a data-driven method for filtering persistent series that requires no prior assumptions about the memory, thus, is robust to the actual memory of the data. It makes three primary contributions: first, it generalizes unobserved components (UC) models to fractionally integrated trends, making prior assumptions about the trend memory redundant while retaining the advantages of the state space structure of UC models; second, it establishes the asymptotic estimation theory for fractional UC models under mild assumptions; and third, it presents a computationally efficient estimator for the trend by deriving the closed-form solution to the Kalman filter optimization problem.

The global semiconductor CMP (Chemical Mechanical Planarization) filters market is experiencing robust growth, driven by the increasing demand for advanced semiconductor devices and the miniaturization of integrated circuits. The market, estimated at $2.5 billion in 2025, is projected to exhibit a Compound Annual Growth Rate (CAGR) of 8% from 2025 to 2033. This growth is fueled by several key factors: the expanding adoption of advanced node technologies in logic and memory chips, necessitating higher-performance filtration solutions; the increasing prevalence of 5G and AI applications, which drive demand for high-performance computing; and the ongoing expansion of semiconductor manufacturing capacity globally, particularly in regions like Asia-Pacific. Key applications include wafer fabrication and chip production, with wafer fabrication currently holding a larger market share due to its stringent filtration requirements. Major players like Pall Filter, Hangzhou Cobetter Filtration Equipment, and Entegris are actively engaged in innovation and expanding their product portfolios to capitalize on this growth, focusing on improved filter efficiency, longer lifespan, and reduced contamination risks. The market's growth trajectory is influenced by several trends including the rising adoption of advanced materials in semiconductor manufacturing, the shift toward larger wafer sizes, and the growing adoption of automation and process optimization within fabs. Restraints include the cyclical nature of the semiconductor industry, the volatility in raw material prices, and the need for continuous investment in research and development to meet evolving technological demands. The regional breakdown reveals strong growth in Asia-Pacific, driven by the concentration of semiconductor manufacturing facilities in China, South Korea, and Taiwan. North America and Europe also represent significant markets, with ongoing investments in advanced semiconductor manufacturing capabilities. The market segmentation based on application (wafer fabrication and chip production) highlights a strong focus on meeting the critical filtration needs of both areas, with ongoing innovation in filter materials and design tailored to specific requirements. The forecast period of 2025-2033 indicates continued expansion, suggesting significant opportunities for market participants.

STL files of the structures from 3D examples in "An Advection-Diffusion based Filter for Machinable Designs in Topology Optimization".The bracket designs contain the optimized part of the designs only. Use the included passive pin and passive rings files to obtain the full model. Abstract cited from DOI: 10.1016/j.cma.2021.114488 :"This paper introduces a simple formulation for topology optimization problems ensuring manufacturability by machining. The method distinguishes itself from existing methods by using the advection-diffusion equation with Robin boundary conditions to perform a filtering of the design variables. Furthermore, the approach is easy to implement on unstructured meshes and in a distributed memory setting. Finally, the proposed approach can be performed with few to no continuation steps in any system parameters. Applications are demonstrated with topology optimization on unstructured meshes with up to 64 million elements and up to 29 milling tool directions."

The global electronics and semiconductor process filter market is projected to reach a value of $1,236 million by 2033, exhibiting a CAGR of 9.7% during the forecast period (2025-2033). The growing demand for electronics and semiconductors, particularly in the automotive, healthcare, and telecommunications industries, is driving the market growth. Additionally, the increasing adoption of semiconductor devices in the production of electric and hybrid vehicles and the rising demand for advanced computing and data analytics are contributing to the market expansion. Key market trends include the adoption of advanced filtration technologies and the increasing use of microfiltration and ultrafiltration systems to meet the stringent requirements of the electronics and semiconductor manufacturing processes. The market is segmented into applications, such as semiconductor foundry manufacturing, memory manufacturing, and solar semiconductor manufacturing, and types, including gas filters and liquid filters. Major companies operating in the market include Pall, Entegris, Nippon Seisen, Exyte Technology, Camfil, Ecopro, CoorsTek, YESIANG Enterprise, Donaldson Company, AAF International, Purafil, Porvair, Dan-Takuma Technologies, Cobetter Filtration Group, Critical Process Filtration, Mott Corporation, SV Techsol, and Advantec Group. Geographically, North America, Europe, and Asia Pacific are the prominent regions in the electronics and semiconductor process filter market. The Electronics and Semiconductor Process Filter market is a highly specialized segment within the larger filtration industry. These filters play a critical role in manufacturing semiconductors and electronics by removing contaminants from gases and liquids used in the production process. The global Electronics and Semiconductor Process Filter market is valued at approximately $500 million and is expected to grow to $800 million by 2028, exhibiting a CAGR of 6%.

The semiconductor filter element market, valued at $695 million in 2025, is projected to experience robust growth, driven by the burgeoning semiconductor industry and increasing demand for advanced filtration technologies. The Compound Annual Growth Rate (CAGR) of 10.5% from 2025 to 2033 indicates a significant expansion of the market, exceeding $1.5 billion by 2033. Key drivers include the rising adoption of advanced semiconductor manufacturing processes (like EUV lithography), necessitating higher purity levels and stringent filtration requirements. Furthermore, the increasing prevalence of miniaturization and the growing demand for high-performance computing and 5G technologies are fueling the need for more efficient and reliable filter elements. The market is segmented by filter type (e.g., HEPA, ULPA, membrane filters), application (e.g., cleanrooms, chemical delivery systems), and end-use industry (e.g., logic, memory, foundry). Competitive landscape analysis reveals a mix of established players like Pall, Entegris, and 3M, alongside several specialized filtration companies. Challenges include maintaining consistent filter quality and managing the cost of advanced materials and manufacturing processes. The significant growth potential is largely due to the continuous innovation within the semiconductor industry, leading to the development of more sophisticated filter elements. The stringent quality control and regulatory requirements related to semiconductor manufacturing further bolster market growth, pushing manufacturers to invest in high-quality, reliable filtration solutions. While the initial investment in advanced filtration technologies can be high, the long-term benefits, including reduced manufacturing defects, improved yield, and enhanced product lifespan, make these investments compelling. Future market expansion will likely be influenced by ongoing technological advancements in filtration technology, the emergence of new materials, and the increasing adoption of automation within semiconductor manufacturing facilities. This analysis provides a foundation for understanding the growth trajectory of this crucial component of the semiconductor industry.

https://pypi-camo.freetls.fastly.net/90015efe39b09fb0e961bec1cf937228660265da/68747470733a2f2f757365722d696d616765732e67697468756275736572636f6e74656e742e636f6d2f3839313636342f3232373130333039302d36363234626637642d393532342d346530352d396432632d6332386435643435313438312e706e67" alt="logo">

Chroma - the open-source embedding database. The fastest way to build Python or JavaScript LLM apps with memory!

import chromadb
# setup Chroma in-memory, for easy prototyping. Can add persistence easily!
client = chromadb.Client()

# Create collection. get_collection, get_or_create_collection, delete_collection also available!
collection = client.create_collection("all-my-documents")

# Add docs to the collection. Can also update and delete. Row-based API coming soon!
collection.add(
  documents=["This is document1", "This is document2"], # we handle tokenization, embedding, and indexing automatically. You can skip that and add your own embeddings as well
  metadatas=[{"source": "notion"}, {"source": "google-docs"}], # filter on these!
  ids=["doc1", "doc2"], # unique for each doc
)

Query/search 2 most similar results. You can also .get by id

results = collection.query(
  query_texts=["This is a query document"],
  n_results=2,
  # where={"metadata_field": "is_equal_to_this"}, # optional filter
  # where_document={"$contains":"search_string"} # optional filter
)

Features Simple: Fully-typed, fully-tested, fully-documented == happiness Integrations: 🦜️🔗 LangChain (python and js), 🦙 LlamaIndex and more soon Dev, Test, Prod: the same API that runs in your python notebook, scales to your cluster Feature-rich: Queries, filtering, density estimation and more Free & Open Source: Apache 2.0 Licensed

TCAM Memory Market Outlook

According to the latest research conducted in 2025, the global TCAM Memory market size is valued at USD 1.42 billion in 2024, reflecting a robust industry growth trajectory. The market is projected to expand at a CAGR of 12.8% during the forecast period, reaching a value of USD 4.24 billion by 2033. This impressive growth is primarily fueled by the surging demand for high-speed networking solutions, exponential data traffic, and the advancing adoption of next-generation security infrastructure across various sectors. The proliferation of data centers, rapid digital transformation, and the increasing complexity of network architectures are among the critical factors propelling the TCAM Memory market forward.

One of the key growth drivers for the TCAM Memory market is the escalating need for high-performance networking equipment capable of handling massive, real-time data processing. As enterprises and service providers deploy advanced networking technologies, such as 5G, edge computing, and software-defined networking (SDN), the requirement for efficient and rapid packet classification becomes paramount. TCAM (Ternary Content Addressable Memory) technology, with its unparalleled ability to perform parallel searches and support complex lookup operations, is increasingly being integrated into routers, switches, and security appliances. This is particularly crucial in environments where low latency and high throughput are non-negotiable, such as financial trading platforms, cloud data centers, and mission-critical government networks.

Another significant factor contributing to the market’s expansion is the growing emphasis on network security and the need for advanced threat detection mechanisms. The proliferation of sophisticated cyber threats and the volume of encrypted traffic necessitate hardware-based solutions that can rapidly match and filter data packets against extensive rule sets. TCAM Memory, renowned for its speed and flexibility in handling access control lists (ACLs) and routing tables, is increasingly favored in security devices, including firewalls and intrusion prevention systems. Furthermore, as organizations migrate to hybrid and multi-cloud environments, the demand for scalable, high-speed memory solutions such as TCAM is expected to intensify, further bolstering market growth.

Technological advancements and ongoing R&D investments are also shaping the TCAM Memory landscape. Innovations in semiconductor manufacturing, power efficiency, and integration techniques are enabling the development of more compact, energy-efficient, and cost-effective TCAM solutions. These advancements are making TCAM Memory accessible to a broader range of applications beyond traditional networking, including artificial intelligence, big data analytics, and IoT infrastructure. Additionally, the emergence of embedded TCAM architectures is facilitating seamless integration into application-specific integrated circuits (ASICs) and system-on-chips (SoCs), expanding the addressable market and driving adoption in emerging use cases.

From a regional perspective, Asia Pacific is emerging as the fastest-growing market for TCAM Memory, driven by rapid digitization, expanding telecommunications infrastructure, and the presence of major data center hubs in countries such as China, India, and Singapore. North America continues to dominate the market in terms of revenue share, owing to its mature IT ecosystem, early adoption of advanced networking technologies, and the concentration of leading technology vendors. Europe is witnessing steady growth, fueled by investments in smart city initiatives, 5G rollouts, and stringent data privacy regulations. Meanwhile, Latin America and the Middle East & Africa are gradually embracing TCAM Memory solutions, supported by modernization efforts in telecommunications and government networks.

Type Analysis

The TCAM Memory market is segmented by type into Standalone TCAM and Embedded TCAM, each serving distinct application requirements and end-user preferences. Standalone TCAM, traditionally used in high-end networking equipment, offers dedicated memory resources for fast and flexible search operations. These devices are particularly valued in scenarios where maximum performance and scalability are critical, such as core routers, large data centers, and high-frequency trading systems. The ability to handle extensive rule sets and support

Mnemonic-enhanced memory has been observed for negative events. Here, we investigate its association with spatiotemporal attention, consolidation, and age. An ingenious method to study visual attention for emotional stimuli is eye tracking. Twenty young adults and twenty-one older adults encoded stimuli depicting neutral faces, angry faces, and houses while eye movements were recorded. The encoding phase was followed by an immediate and delayed (48 h) recognition assessment. Linear mixed model analyses of recognition performance with group, emotion, and their interaction as fixed effects revealed increased performance for angry compared to neutral faces in the young adults group only. Furthermore, young adults showed enhanced memory for angry faces compared to older adults. This effect was associated with a shorter fixation duration for angry faces compared to neutral faces in the older adults group. Furthermore, the results revealed that total fixation duration was a strong predictor for face memory performance. Methods Participants Forty-one subjects participated in our study. They were recruited by advertisements for participation in an eye-tracker memory experiment. Participants did not receive financial compensation for their participation. Inclusion criteria consisted of (1) 18–30-year age range (young adults group) or 50–90-year age range (older adults group) and (2) an MMSE score above 25. The young adults and older adults group consisted of 20 participants [7 males (35%); mean age ± SD = 22 ± 2 years, range 18–29] and 21 participants [9 males (43%); mean age ± SD = 69 ± 7 years, range 53–87], respectively. One participant from the older adults group was not included in the eye movement analysis due to technical issues. Participants completed the Addenbrooke’s Cognitive Examination III (ACE-III), which includes the Mini–Mental State Examination (MMSE). All participants had an ACE-III score above 71. Eye Tracker and Eye Movement Recordings Eye movement data were collected during the encoding phase at a sampling rate of 120 Hz using the Tobii eye tracker TX300 and processed with Tobii Studio 3.4.7. During recording, the eye tracker collects raw eye movement data points, which are processed into fixations and used to calculate eye-tracking metrics, by applying a fixation filter to the data. We applied default settings, including the Tobii fixation filter, with a velocity threshold of 0.84 pixels/ms (35 pixels) and a distant threshold (distance between two consecutive fixations) of 35 pixels (default). In short, peak values are identified, i.e., the values that are greater than both of its two closest neighbors. The list of peaks is then processed into fixations, where the start and end points of a fixation are set by two consecutive peaks. The spatial positions of the fixations are calculated by taking the median of the unfiltered data points in that interval. Secondly, the Euclidean distances between all the fixations are calculated and if the distance between two consecutive fixations falls below a second user-defined threshold, the two fixations are merged into a single fixation. The process is repeated until no fixation points are closer to each other than the threshold. A detailed description of the Tobii fixation can be found in the Tobii Studio user manual (https://www.tobiipro.com/siteassets/tobii-pro/user-manuals/tobii-pro-studio-user-manual.pdf). Statistical Analysis Behavioral Analyses Behavioral results were analyzed according to signal detection theory. R-Score Plus was used to calculate d’ for confidence rating designs. D’ was calculated as a function of category (face vs. house), emotion (angry vs. neutral), interval (IR vs. DR), and group (older adults and young adults). We calculated the mean interval between the encoding phase and DR (lag) for every participant. To evaluate the anticipated outcomes for group differences in d’ in the IR phase, we performed the following general multivariate regression model, which takes repeated measures within subjects into account. Let Yi be a vector with repeated measures for the ith subject (i … N). This general multivariate regression model assumes that Yi satisfies the following regression model: Yi = Xiβ + εi with Xi being a matrix of covariates (e.g., intercept, group, emotion condition, and group x emotion condition), β is a vector of regression coefficients, and εi is a vector of error components with εi∼N(0, Σ). For the variance/covariance structure Σ of each subject, we considered a compound symmetry and unstructured variance/covariance matrix. Selection of the adequate variance/covariance matrix was based on a likelihood-ratio test. Reference coding was used for group (2 levels: older adults = 1 vs. young adults = 0) and emotion (2 levels: neutral = 1 vs. angry = 0). To evaluate main and interaction effects, Bonferroni-corrected post hoc tests were used. It may be noted that this model is a special case of a linear mixed model (Verbeke and Molenberghs, 2000) and that the mean structure Xiβ (the parameters of interest) can be interpreted as that in a classical ANOVA or regression model. Second, we performed a similar model but with category ((2 levels: neutral = 1 vs. angry = 0)) instead of emotion as predictor. These analyses were performed for the two different memory stages (IR and DR) separately. Lastly, we performed a similar model but with intervals (2 levels: IR = 1 vs. DR = 0) for the different conditions (house, face, angry face, neutral face) separately. Finally, a similar model was used with groups (2 levels: older adults = 1 vs. young adults = 0), intervals (2 levels: IR = 1 vs. DR = 0), and group x interval as predictors. All analyses were performed in SPSS. Eye-Tracker Analyses Eye movement data were calculated for house, face, and the three areas of interest: mouth, nose, and eyes. For every participant, two indices for eye movement data were recorded: total fixation duration and fixation count. Total fixation duration represents the total time of fixation as it measures the sum of the duration (seconds) for all fixations within an area of interest for all test stimuli throughout the experiment. Fixation count measures the number of fixations in each area of interest for all test stimuli throughout the experiment. If during the recording the participant leaves and returns to the same media element, this is counted as a new fixation. A detailed description of the metric measures can be found in the Tobii Studio user manual (https://www.tobiipro.com/siteassets/tobii-pro/user-manuals/tobii-pro-studio-user-manual.pdf). We exported the gaze data from Tobii Studio to SPSS for further analysis. Statistical tests on the gaze data were preceded by a normality check on the distributions of the respective residuals by means of a Shapiro–Wilk test. In case normality could not be assumed, non-parametric tests were performed (Mann–Whitney and Wilcoxon tests). In order to investigate the association between the behavioral data and eye movements, we performed Spearman correlations. We computed correlations between d’ (IR, DR) and eye tracker data (total fixation duration and fixation count) during encoding for both groups separately (young adults and older adults).

The Russian Financial Statements Database (RFSD) is an open, harmonized collection of annual unconsolidated financial statements of the universe of Russian firms:

• 🔓 First open data set with information on every active firm in Russia.

• 🗂️ First open financial statements data set that includes non-filing firms.

• 🏛️ Sourced from two official data providers: the Rosstat and the Federal Tax Service.

• 📅 Covers 2011-2023 initially, will be continuously updated.

• 🏗️ Restores as much data as possible through non-invasive data imputation, statement articulation, and harmonization.

The RFSD is hosted on 🤗 Hugging Face and Zenodo and is stored in a structured, column-oriented, compressed binary format Apache Parquet with yearly partitioning scheme, enabling end-users to query only variables of interest at scale.

The accompanying paper provides internal and external validation of the data: http://arxiv.org/abs/2501.05841.

Here we present the instructions for importing the data in R or Python environment. Please consult with the project repository for more information: http://github.com/irlcode/RFSD.

Importing The Data

You have two options to ingest the data: download the .parquet files manually from Hugging Face or Zenodo or rely on 🤗 Hugging Face Datasets library.

Python

🤗 Hugging Face Datasets

It is as easy as:

from datasets import load_dataset import polars as pl

This line will download 6.6GB+ of all RFSD data and store it in a 🤗 cache folder

RFSD = load_dataset('irlspbru/RFSD')

Alternatively, this will download ~540MB with all financial statements for 2023# to a Polars DataFrame (requires about 8GB of RAM)

RFSD_2023 = pl.read_parquet('hf://datasets/irlspbru/RFSD/RFSD/year=2023/*.parquet')

Please note that the data is not shuffled within year, meaning that streaming first n rows will not yield a random sample.

Local File Import

Importing in Python requires pyarrow package installed.

import pyarrow.dataset as ds import polars as pl

Read RFSD metadata from local file

RFSD = ds.dataset("local/path/to/RFSD")

Use RFSD_dataset.schema to glimpse the data structure and columns' classes

print(RFSD.schema)

Load full dataset into memory

RFSD_full = pl.from_arrow(RFSD.to_table())

Load only 2019 data into memory

RFSD_2019 = pl.from_arrow(RFSD.to_table(filter=ds.field('year') == 2019))

Load only revenue for firms in 2019, identified by taxpayer id

RFSD_2019_revenue = pl.from_arrow( RFSD.to_table( filter=ds.field('year') == 2019, columns=['inn', 'line_2110'] ) )

Give suggested descriptive names to variables

renaming_df = pl.read_csv('local/path/to/descriptive_names_dict.csv') RFSD_full = RFSD_full.rename({item[0]: item[1] for item in zip(renaming_df['original'], renaming_df['descriptive'])})

R

Local File Import

Importing in R requires arrow package installed.

library(arrow) library(data.table)

Read RFSD metadata from local file

RFSD <- open_dataset("local/path/to/RFSD")

Use schema() to glimpse into the data structure and column classes

schema(RFSD)

Load full dataset into memory

scanner <- Scanner$create(RFSD) RFSD_full <- as.data.table(scanner$ToTable())

Load only 2019 data into memory

scan_builder <- RFSD$NewScan() scan_builder$Filter(Expression$field_ref("year") == 2019) scanner <- scan_builder$Finish() RFSD_2019 <- as.data.table(scanner$ToTable())

Load only revenue for firms in 2019, identified by taxpayer id

scan_builder <- RFSD$NewScan() scan_builder$Filter(Expression$field_ref("year") == 2019) scan_builder$Project(cols = c("inn", "line_2110")) scanner <- scan_builder$Finish() RFSD_2019_revenue <- as.data.table(scanner$ToTable())

Give suggested descriptive names to variables

renaming_dt <- fread("local/path/to/descriptive_names_dict.csv") setnames(RFSD_full, old = renaming_dt$original, new = renaming_dt$descriptive)

Use Cases

🌍 For macroeconomists: Replication of a Bank of Russia study of the cost channel of monetary policy in Russia by Mogiliat et al. (2024) — interest_payments.md

🏭 For IO: Replication of the total factor productivity estimation by Kaukin and Zhemkova (2023) — tfp.md

🗺️ For economic geographers: A novel model-less house-level GDP spatialization that capitalizes on geocoding of firm addresses — spatialization.md

FAQ

Why should I use this data instead of Interfax's SPARK, Moody's Ruslana, or Kontur's Focus?hat is the data period?

To the best of our knowledge, the RFSD is the only open data set with up-to-date financial statements of Russian companies published under a permissive licence. Apart from being free-to-use, the RFSD benefits from data harmonization and error detection procedures unavailable in commercial sources. Finally, the data can be easily ingested in any statistical package with minimal effort.

What is the data period?

We provide financials for Russian firms in 2011-2023. We will add the data for 2024 by July, 2025 (see Version and Update Policy below).

Why are there no data for firm X in year Y?

Although the RFSD strives to be an all-encompassing database of financial statements, end users will encounter data gaps:

We do not include financials for firms that we considered ineligible to submit financial statements to the Rosstat/Federal Tax Service by law: financial, religious, or state organizations (state-owned commercial firms are still in the data).

Eligible firms may enjoy the right not to disclose under certain conditions. For instance, Gazprom did not file in 2022 and we had to impute its 2022 data from 2023 filings. Sibur filed only in 2023, Novatek — in 2020 and 2021. Commercial data providers such as Interfax's SPARK enjoy dedicated access to the Federal Tax Service data and therefore are able source this information elsewhere.

Firm may have submitted its annual statement but, according to the Uniform State Register of Legal Entities (EGRUL), it was not active in this year. We remove those filings.

Why is the geolocation of firm X incorrect?

We use Nominatim to geocode structured addresses of incorporation of legal entities from the EGRUL. There may be errors in the original addresses that prevent us from geocoding firms to a particular house. Gazprom, for instance, is geocoded up to a house level in 2014 and 2021-2023, but only at street level for 2015-2020 due to improper handling of the house number by Nominatim. In that case we have fallen back to street-level geocoding. Additionally, streets in different districts of one city may share identical names. We have ignored those problems in our geocoding and invite your submissions. Finally, address of incorporation may not correspond with plant locations. For instance, Rosneft has 62 field offices in addition to the central office in Moscow. We ignore the location of such offices in our geocoding, but subsidiaries set up as separate legal entities are still geocoded.

Why is the data for firm X different from https://bo.nalog.ru/?

Many firms submit correcting statements after the initial filing. While we have downloaded the data way past the April, 2024 deadline for 2023 filings, firms may have kept submitting the correcting statements. We will capture them in the future releases.

Why is the data for firm X unrealistic?

We provide the source data as is, with minimal changes. Consider a relatively unknown LLC Banknota. It reported 3.7 trillion rubles in revenue in 2023, or 2% of Russia's GDP. This is obviously an outlier firm with unrealistic financials. We manually reviewed the data and flagged such firms for user consideration (variable outlier), keeping the source data intact.

Why is the data for groups of companies different from their IFRS statements?

We should stress that we provide unconsolidated financial statements filed according to the Russian accounting standards, meaning that it would be wrong to infer financials for corporate groups with this data. Gazprom, for instance, had over 800 affiliated entities and to study this corporate group in its entirety it is not enough to consider financials of the parent company.

Why is the data not in CSV?

The data is provided in Apache Parquet format. This is a structured, column-oriented, compressed binary format allowing for conditional subsetting of columns and rows. In other words, you can easily query financials of companies of interest, keeping only variables of interest in memory, greatly reducing data footprint.

Version and Update Policy

Version (SemVer): 1.0.0.

We intend to update the RFSD annualy as the data becomes available, in other words when most of the firms have their statements filed with the Federal Tax Service. The official deadline for filing of previous year statements is April, 1. However, every year a portion of firms either fails to meet the deadline or submits corrections afterwards. Filing continues up to the very end of the year but after the end of April this stream quickly thins out. Nevertheless, there is obviously a trade-off between minimization of data completeness and version availability. We find it a reasonable compromise to query new data in early June, since on average by the end of May 96.7% statements are already filed, including 86.4% of all the correcting filings. We plan to make a new version of RFSD available by July.

Licence

Creative Commons License Attribution 4.0 International (CC BY 4.0).

Copyright © the respective contributors.

Citation

Please cite as:

@unpublished{bondarkov2025rfsd, title={{R}ussian {F}inancial {S}tatements {D}atabase}, author={Bondarkov, Sergey and Ledenev, Victor and Skougarevskiy, Dmitriy}, note={arXiv preprint arXiv:2501.05841}, doi={https://doi.org/10.48550/arXiv.2501.05841}, year={2025}}

Acknowledgments and Contacts

Data collection and processing: Sergey Bondarkov, sbondarkov@eu.spb.ru, Viktor Ledenev, vledenev@eu.spb.ru

Project conception, data validation, and use cases: Dmitriy Skougarevskiy, Ph.D.,

Inference for high-dimensional, large scale and long series dynamic systems is a challenging task in modern data science. The existing algorithms, such as particle filter or sequential importance sampler, do not scale well to the dimension of the system and the sample size of the dataset, and often suffers from the sample degeneracy issue for long series data. The recently proposed Langevinized ensemble Kalman filter (LEnKF) addresses these difficulties in a coherent way. However, it cannot be applied to the case that the dynamic system contains unknown parameters. This article proposes the so-called stochastic approximation-LEnKF for jointly estimating the states and unknown parameters of the dynamic system, where the parameters are estimated on the fly based on the state variables simulated by the LEnKF under the framework of stochastic approximation Markov chain Monte Carlo (MCMC). Under mild conditions, we prove its consistency in parameter estimation and ergodicity in state variable simulations. The proposed algorithm can be used in uncertainty quantification for long series, large scale, and high-dimensional dynamic systems. Numerical results indicate its superiority over the existing algorithms. We employ the proposed algorithm in state-space modeling of the sea surface temperature with a long short term memory (LSTM) network, which indicates its great potential in statistical analysis of complex dynamic systems encountered in modern data science. Supplementary materials for this article are available online.

The spin filter market is experiencing robust growth, driven by increasing demand across diverse applications. The market's expansion is fueled primarily by the burgeoning electric vehicle (EV) sector, where spin filters play a crucial role in improving motor efficiency and performance. The semiconductor and data storage industries are also significant contributors, with spin filters enabling advancements in high-density memory and faster data processing. Furthermore, the emerging field of quantum computing relies heavily on advanced spintronic components, further boosting market demand. While precise market sizing data is unavailable, considering the CAGR (let's assume a conservative 15% based on similar emerging tech markets) and a current market value (estimated at $500 million in 2025), the market is projected to reach approximately $1.1 billion by 2033. The market is segmented by application (Data Storage, Electric Vehicles, Industrial Motors, Semiconductor Lasers, Microwave Devices, Quantum Computing, Other) and type (Clockwise Spin, Counter Clockwise Spin). While the Electric Vehicle segment is currently the fastest-growing, the potential of Quantum Computing holds immense long-term promise, promising exponential market expansion in the latter half of the forecast period. Geographic distribution shows significant presence in North America and Europe, driven by established technological hubs and robust R&D investments. Asia-Pacific is expected to witness considerable growth, fueled by increasing electronics manufacturing and adoption of EVs in emerging economies. However, challenges remain, including high manufacturing costs and the need for further technological advancements to enhance spin filter efficiency and scalability.

The electronics & semiconductor process filter market is booming, projected to reach $2185 million by 2025 with a 9.4% CAGR. Discover key trends, drivers, and top companies shaping this dynamic industry. Explore regional market shares and future growth projections in this in-depth analysis.