The Survey of Consumer Finances (SCF) is normally a triennial cross-sectional survey of U.S. families. The survey data include information on families' balance sheets, pensions, income, and demographic characteristics.

🏛️ Federal Reserve Document Scraper

1. Dataset description

This repository contains code for downloading and organizing Federal Reserve documents from the official Federal Reserve Board website.

These files were used as part of my NLP project. While collecting data, my data collection code is inspired by centralbank_analysis by yukit-k. However, that implementation had some limitations:

❌ Incomplete handling of newer HTML structures on the Fed website

❌ No support for Greenbook/Tealbook files

❌ File naming and folder structure not ideal for downstream processing

❌ No handling of failed downloads or noisy formatting

So I made som key Improvements:

✅ Supports both Greenbook and Minutes. You can choose which type to download

✅ Automatic directory organization. Files are saved using a consistent format as:

FOMC_[document type]_YYYY-MM-DD

✅ Duplicate check & resume support: Prevents redundant downloads and handles broken links gracefully

✅ Modular and extensible codebase Easy to extend for other Fed documents (e.g., SEP, transcripts)

2. Data detail

This repository contains modules for downloading and processing various official publications of the Federal Open Market Committee (FOMC). These documents, produced and released by the Federal Reserve, provide detailed insight into U.S. monetary policy formation, communication, and economic analysis over time.

Below is a reference guide to the major FOMC document types represented in this repository.

📅 FomcAgenda.py – Meeting Agendas

Agendas are created by the FOMC Secretariat in coordination with the Chair and outline the topics of discussion for each meeting, including standard items (e.g., open market operations, economic outlook) and special topics. Participants receive the agenda about one week in advance.

📄 FomcStatement.py – Policy Statements

FOMC statements are brief summaries of monetary policy decisions released immediately after each meeting. These statements have become a key communication tool since 1994 and are now issued after every scheduled meeting, even if policy remains unchanged.

📝 FomcMinutes.py – Meeting Minutes

Minutes provide a concise, narrative summary of policy discussions and rationales. Since 2004, they are released three weeks after each meeting. The minutes include details on voting outcomes and dissenting views, and are eventually included in the Fed’s Annual Report.

🧾 FomcPresConfScript.py – Press Conference Transcripts

Beginning in 2011, the Fed Chair has held press conferences following certain FOMC meetings. These transcripts document the Chair’s remarks and responses to journalists, offering additional context and forward guidance. Released shortly after the meeting.

🗣️ FomcMeetingScript.py – Meeting Transcripts

Verbatim transcripts of FOMC meetings, produced from audio recordings and lightly edited for readability. They are released with a 5-year delay. For meetings prior to 1994, transcripts were reconstructed from raw records and may contain transcription uncertainties.

📚 FomcGreenbook.py – Greenbook (1964–2010)

The Greenbook, officially titled Current Economic and Financial Conditions, was prepared by Board staff and delivered to FOMC members six days before each meeting. It provided forecasts, data analyses, and economic outlooks.

Part 1: Summary and forecast

Part 2: Detailed breakdowns

Supplement: Late-breaking updates

📘 FomcBlueBook.py – Bluebook (1965–2010)

The Bluebook, titled Monetary Policy Alternatives, outlined potential policy options and risks. It was distributed shortly after the Greenbook and informed FOMC decisions. The document evolved from earlier versions like Money Market and Reserve Relationships.

🧠 FomcTealbook.py – Tealbook (2010–Present)

The Tealbook replaced both the Greenbook and Bluebook in June 2010. It is split into two parts:

Tealbook A: Current Situation and Outlook — Forecasts and financial developments

Tealbook B: Strategies and Alternatives — Policy options and simulations

Both are released with a 5-year lag.

📙 FomcBeigeBook.py – Beige Book

The Beige Book, published eight times a year, summarizes anecdotal economic conditions across the 12 Federal Reserve Districts. Based on business surveys, interviews, and internal reports, it is released ~two weeks before each meeting.

🧾 FomcTestimony.py – Testimony before Congress

This includes the Chair’s Semiannual Monetary Policy Report to Congress and other testimonies. These communications explain the Fed’s outlook and policies directly to lawmakers and the public.

📚 References

Federal Reserve – FOMC Archive

Wikipedia – Federal Open Market Committee

View data of PCE, an index that measures monthly changes in the price of consumer goods and services as a means of analyzing inflation.

Content

Source ID: FL313066220.Q

For more information about the Flow of Funds tables, see: https://www.federalreserve.gov/apps/fof/Default.aspx

For a detailed description, including how this series is constructed, see: https://www.federalreserve.gov/apps/fof/SeriesAnalyzer.aspx?s=FL313066220&t=

Context

This is a dataset from the Federal Reserve hosted by the Federal Reserve Economic Database (FRED). FRED has a data platform found here and they update their information according to the frequency that the data updates. Explore the Federal Reserve using Kaggle and all of the data sources available through the Federal Reserve organization page!

• Update Frequency: This dataset is updated daily.

• Observation Start: 1945-10-01

• Observation End : 2019-04-01

Acknowledgements

This dataset is maintained using FRED's API and Kaggle's API.

Cover photo by Michael on Unsplash
Unsplash Images are distributed under a unique Unsplash License.

Foreign Exchange Reserves in China increased to 3343000 USD Million in October from 3339000 USD Million in September of 2025. This dataset provides - China Foreign Exchange Reserves - actual values, historical data, forecast, chart, statistics, economic calendar and news.

Foreign Exchange Reserves in Bangladesh increased to 32335.20 USD Million in October from 31426.80 USD Million in September of 2025. This dataset provides - Bangladesh Foreign Exchange Reserves - actual values, historical data, forecast, chart, statistics, economic calendar and news.

The Small Business Credit Survey is a national sample of small businesses, or firms with fewer than 500 employees, aimed at providing insight into firms' financing and debt needs and experiences. Analysis of this dataset is issued through a series of reports.

This table represents the breakdown of tax refunds by recipient (individual vs business) and type (check vs electronic funds transfer). Tax refunds are also represented as withdrawals in the Deposits and Withdrawals of Operating Cash table. All figures are rounded to the nearest million. As of February 14, 2023, Table VI Income Tax Refunds Issued was renamed to Table V Income Tax Refunds Issued within the published report.

Content

Reserve balance requirements series is the portion of the reserve requirements not satisfied by vault cash. The series is calulated as Required reserves (https://fred.stlouisfed.org/series/REQRESNS) less Vault cash used to satisfy required reserves (https://fred.stlouisfed.org/series/VAULT).

Context

This is a dataset from the Federal Reserve hosted by the Federal Reserve Economic Database (FRED). FRED has a data platform found here and they update their information according to the frequency that the data updates. Explore the Federal Reserve using Kaggle and all of the data sources available through the Federal Reserve organization page!

• Update Frequency: This dataset is updated daily.

• Observation Start: 1984-02-01

• Observation End : 2019-11-01

Acknowledgements

This dataset is maintained using FRED's API and Kaggle's API.

Cover photo by David Clode on Unsplash
Unsplash Images are distributed under a unique Unsplash License.

The interest rate set by the Federal Reserve is a crucial tool for promoting economic conditions that meet the mandate established by the United States Congress, which includes high employment, low and stable inflation, sustainable economic growth, and the moderation of long-term interest rates. The interest rates determined by the Fed directly influence the cost of credit, making financing either more accessible or more restrictive. When interest rates are low, there is a greater incentive for consumers to purchase homes through mortgages, finance automobiles, or undertake home renovations. Additionally, businesses are encouraged to invest in expanding their operations, whether by purchasing new equipment, modernizing facilities, or hiring more workers. Conversely, higher interest rates tend to curb such activity, discouraging borrowing and slowing economic expansion.

The dataset analyzed contains information on the economic conditions in the United States on a monthly basis since 1954, including the federal funds rate, which represents the percentage at which financial institutions trade reserves held at the Federal Reserve with each other in the interbank market overnight. This rate is determined by the market but is directly influenced by the Federal Reserve through open market operations to reach the established target. The Federal Open Market Committee (FOMC) meets eight times a year to determine the federal funds rate target, which has been defined within a range with upper and lower limits since December 2008.

Furthermore, real Gross Domestic Product (GDP) is calculated based on the seasonally adjusted quarterly rate of change in the economy, using chained 2009 dollars as a reference. The unemployment rate represents the seasonally adjusted percentage of the labor force that is unemployed. Meanwhile, the inflation rate is determined by the monthly change in the Consumer Price Index, excluding food and energy prices for a more stable analysis of core inflation.

The interest rate data was sourced from the Federal Reserve Bank of St. Louis' economic data portal, while GDP information was provided by the U.S. Bureau of Economic Analysis, and unemployment and inflation data were made available by the U.S. Bureau of Labor Statistics.

The analysis of this data helps to understand how economic growth, the unemployment rate, and inflation influence the Federal Reserve’s monetary policy decisions. Additionally, it allows for a study of the evolution of interest rate policies over time and raises the question of how predictable the Fed’s future decisions may be. Based on observed trends, it is possible to speculate whether the target range set in March 2017 will be maintained, lowered, or increased, considering the prevailing economic context and the challenges faced in conducting U.S. monetary policy.

Content

Weighted average of the rates received on the interest-bearing assets included in M2. The interest-bearing assets include size of the other checkable deposits, thrift saving deposits, money market mutual fund holdings, and small time deposits that are weighted using their corresponding rates.

The construction of this series was discontinued as of July 12, 2019. The underlying data can be accessed through the following sources: size of the assets can be obtained from the H.6 release (https://www.federalreserve.gov/releases/h6/current/default.htm) published by the Board of Governors, rate on the money market mutual funds from iMoneyNet (https://financialintelligence.informa.com/products-and-services/data-analysis-and-tools/imoneynet), and the remaining rates from the Weekly National Rates and Rate Caps (https://www.fdic.gov/regulations/resources/rates/index.html) from the FDIC website. Listing of the sources is provided for informational purposes only: the Federal Reserve Bank of St. Louis is not associated with any listed private entities and cannot guarantee that the listed data sources will provide the data in the future.

Context

This is a dataset from the Federal Reserve hosted by the Federal Reserve Economic Database (FRED). FRED has a data platform found here and they update their information according to the frequency that the data updates. Explore the Federal Reserve using Kaggle and all of the data sources available through the Federal Reserve organization page!

• Update Frequency: This dataset is updated daily.

• Observation Start: 1959-02-01

• Observation End : 2019-06-01

Acknowledgements

This dataset is maintained using FRED's API and Kaggle's API.

Cover photo by Leon Skibitzki on Unsplash
Unsplash Images are distributed under a unique Unsplash License.

U.S. Economic and Financial Dataset

Dataset Description

This dataset combines historical U.S. economic and financial indicators, spanning the last 50 years, to facilitate time series analysis and uncover patterns in macroeconomic trends. It is designed for exploring relationships between interest rates, inflation, economic growth, stock market performance, and industrial production.

Key Features

• Frequency: Monthly
• Time Period: Last 50 years from Nov-24
• Sources:
  • Federal Reserve Economic Data (FRED)
  • Yahoo Finance

Dataset Feature Description

1. Interest Rate (Interest_Rate):

   • The effective federal funds rate, representing the interest rate at which depository institutions trade federal funds overnight.

2. Inflation (Inflation):

   • The Consumer Price Index for All Urban Consumers, an indicator of inflation trends.

3. GDP (GDP):

   • Real GDP measures the inflation-adjusted value of goods and services produced in the U.S.

4. Unemployment Rate (Unemployment):

   • The percentage of the labor force that is unemployed and actively seeking work.

5. Stock Market Performance (S&P500):

   • Monthly average of the adjusted close price, representing stock market trends.

6. Industrial Production (Ind_Prod):

   • A measure of real output in the industrial sector, including manufacturing, mining, and utilities.

Dataset Statistics

1. Total Entries: 599
2. Columns: 6
3. Memory usage: 37.54 kB
4. Data types: float64

Feature Overview

• Columns:
  • Interest_Rate: Monthly Federal Funds Rate (%)
  • Inflation: CPI (All Urban Consumers, Index)
  • GDP: Real GDP (Billions of Chained 2012 Dollars)
  • Unemployment: Unemployment Rate (%)
  • Ind_Prod: Industrial Production Index (2017=100)
  • S&P500: Monthly Average of S&P 500 Adjusted Close Prices

Executive Summary

This project explores the interconnected dynamics of key macroeconomic indicators and financial market trends over the past 50 years, leveraging data from the Federal Reserve Economic Data (FRED) and Yahoo Finance. The dataset integrates critical variables such as the Federal Funds Rate, Inflation (CPI), Real GDP, Unemployment Rate, Industrial Production, and the S&P 500 Index, providing a holistic view of the U.S. economy and financial markets.

The analysis focuses on uncovering relationships between these variables through time-series visualization, correlation analysis, and trend decomposition. Key findings are included in the Insights section. This project serves as a robust resource for understanding long-term economic trends, policy impacts, and market behavior. It is particularly valuable for students, researchers, policymakers, and financial analysts seeking to connect macroeconomic theory with real-world data.

Potential Use Cases

• Economic Analysis: Examine relationships between interest rates, inflation, GDP, and unemployment.
• Stock Market Prediction: Study how macroeconomic indicators influence stock market trends.
• Time Series Modeling: Perform ARIMA, VAR, or other models to forecast economic trends.
• Cyclic Pattern Analysis: Identify how economic shocks and recoveries impact key indicators.

Snap of Power Analysis

https://github.com/user-attachments/assets/1b40e0ca-7d2e-4fbc-8cfd-df3f09e4fdb8">

To ensure sufficient power, the dataset covers last 50 years of monthly data i.e., around 600 entries.

Key Insights derived through EDA, time-series visualization, correlation analysis, and trend decomposition

• Interest Rate and Inflation Dynamics: The interest Rate and inflation exhibit an inverse relationship, especially during periods of aggressive monetary tightening by the Federal Reserve.
• Economic Growth and Market Performance: GDP growth and the S&P 500 Index show a positive correlation, reflecting how market performance often aligns with overall economic health.
• Labor Market and Industrial Output: Unemployment and industrial production demonstrate a strong inverse relationship. Higher industrial output is typically associated with lower unemployment
• Market Behavior During Economic Shocks: The S&P 500 experienced sharp declines during significant crises, such as the 2008 financial crash and the COVID-19 pandemic in 2020. These events also triggered increased unemployment and contractions in GDP, highlighting the interplay between markets and the broader economy.
• Correlation Highlights: S&P 500 and GDP have a strong positive correlation. Interest rates negatively correlate with GDP and inflation, reflecting monetary policy impacts. Unemployment is negatively correlated with industrial production but positively correlated with interest rates.

Link to GitHub Repo

https:/...

List of US Counties (including FIPS State and FIPS County codes) and the respective Federal Reserve District they belong to.

This spreadsheet extends the Excel file "U.S. FIPS County Codes" by MDR Education to include 'Federal Reserve District Boundaries' based on the 1996 document published by the Federal Reserve Bank of St. Louis. It also includes the US territories that are under the Federal Reserve System. There may be some differences in county lists as minor changes to county names have occurred since 1996.

Content

Reverse repurchase agreements are transactions in which securities are sold to a set of counterparties under an agreement to buy them back from the same party on a specified date at the same price plus interest. Reverse repurchase agreements may be conducted with foreign official and international accounts as a service to the holders of these accounts. All other reverse repurchase agreements, including transactions with primary dealers and a set of eligible money market funds, are open market operations intended to manage the supply of reserve balances; reverse repurchase agreements absorb reserve balances from the banking system for the length of the agreement. As with repurchase agreements, the naming convention used here reflects the transaction from the counterparties' perspective; the Federal Reserve receives cash in a reverse repurchase agreement and provides collateral to the counterparties.

Context

This is a dataset from the Federal Reserve hosted by the Federal Reserve Economic Database (FRED). FRED has a data platform found here and they update their information according to the frequency that the data updates. Explore the Federal Reserve using Kaggle and all of the data sources available through the Federal Reserve organization page!

• Update Frequency: This dataset is updated daily.

• Observation Start: 2002-12-18

• Observation End : 2019-12-18

Acknowledgements

This dataset is maintained using FRED's API and Kaggle's API.

Cover photo by Andrew Pons on Unsplash
Unsplash Images are distributed under a unique Unsplash License.

The Survey of Consumer Finances (SCF) dataset, provided by the Federal Reserve, offers comprehensive insights into the financial condition of U.S. households. This dataset is invaluable for researchers, policymakers, and analysts interested in understanding consumer behavior, wealth distribution, and economic trends in the United States.

The SCF dataset includes detailed information on household income, assets, liabilities, and various demographic characteristics. It is collected every three years and serves as a crucial resource for analyzing the financial well-being of American families.

Key Features: Income Data: Information on various sources of income, including wages, investments, and government assistance. Asset Ownership: Detailed accounts of household assets, such as real estate, retirement accounts, stocks, and other investments. Liabilities:Comprehensive details on household debts, including mortgages, credit card debts, and student loans. Demographics: Data covering age, education, race, and family structure, allowing for nuanced analysis of financial trends across different segments of the population.

Use Cases: Economic research and analysis, Policy formulation and assessment, Understanding wealth inequality, Consumer behavior studies

Citing the Dataset:

When using this dataset in your research, please ensure to cite the Federal Reserve Board and the SCF as the original source.

Note: The dataset is intended for educational and research purposes. Users are encouraged to adhere to ethical guidelines when analyzing and interpreting the data.

Content

This series is calculated by the Federal Reserve Bank of St. Louis as Total Borrowings of Depository Institutions from the Federal Reserve (BORROW) (https://fred.stlouisfed.org/series/BORROW) minus Term Auction Credit (TERMAUC) (https://fred.stlouisfed.org/series/TERMAUC). The Term Auction Credit (TERMAUC) (https://fred.stlouisfed.org/series/TERMAUC) was discontinued in May 2011.

The values are monthly averages.

Please note breaks in data: Data prior to 2003-01-01 include adjustment, extended, and seasonal credit. From 2003-01-01 to 2008-04-03, the data include primary, secondary, and seasonal credit. As of 2008-04-10, data include primary, secondary, and seasonal credit, primary dealer credit facility, and other credit extensions. Data from 2008-09-01 are loans to depository institutions for primary, secondary, and seasonal credit, primary dealer and other broker-dealer credit. This category also includes the asset-backed commercial paper money market mutual fund liquidity facility, credit extended to American International Group, Inc., and other credit extensions. Data were modified to include term asset-backed securities loan facility with the release on 2009-03-26.

Context

This is a dataset from the Federal Reserve hosted by the Federal Reserve Economic Database (FRED). FRED has a data platform found here and they update their information according to the frequency that the data updates. Explore the Federal Reserve using Kaggle and all of the data sources available through the Federal Reserve organization page!

• Update Frequency: This dataset is updated daily.

• Observation Start: 1959-01-01

• Observation End : 2019-11-01

Acknowledgements

This dataset is maintained using FRED's API and Kaggle's API.

Cover photo by Francesco Ungaro on Unsplash
Unsplash Images are distributed under a unique Unsplash License.

The Income Distributions and Dynamics in America project seeks to foster new research and analysis on income disparities by providing statistics on income percentiles, shares, growth rates, persistence, and more for many U.S. demographic groups at national and state levels.

files included 1. Percentiles of income module 2. Top income shares module 3. Top income population shares module 4. Income change distributions module 5. Income transition matrix module

Kondo, Illenin, Brandon Hawkins, Kevin Rinz, John Voorheis, Andrew Goodman-Bacon, Natalie Gubbay, and Abigail Wozniak. (2023). Income Distributions and Dynamics in America: Version 1.0 [Dataset]. Minneapolis, MN: Federal Reserve Bank of Minneapolis. https://doi.org/10.21034/data.idda

FOMC Meeting Minutes Dataset (Year 1993+, updated following each release)

Overview

This dataset contains the detailed minutes of the meetings held by the Federal Open Market Committee (FOMC) from 1993 onwards. The FOMC, a key component of the U.S. Federal Reserve System, is responsible for setting national monetary policy. The minutes provide a comprehensive record of the committee's discussions, including reviews of economic and financial conditions, deliberations on policy options, the range of participants' views, the rationale behind policy decisions, and the specific votes cast by each member. They offer significantly more detail than the policy statements released immediately after the meetings.

Background on Meeting Minutes

The minutes of each regularly scheduled meeting of the Committee provide a timely summary of significant policy issues addressed by meeting participants. The minutes record all decisions taken by the Committee with respect to these policy issues and explain the reasoning behind these decisions. From their emergence in their present form in February 1993 until December 2004, the minutes were published approximately three days after the Committee's subsequent meeting. In December 2004, the Committee decided to expedite the release of its minutes. Since then, the minutes have been made available to the public three weeks after the date of the policy decision, thus reducing the lag in their release by an average of about three weeks. The minutes are subsequently published in the Board's Annual Report.

Data Description

Each row in the dataset represents the minutes from a specific FOMC meeting. The dataset includes the following fields:

• Date: The last day of the FOMC meeting.
• Text: The full, official text of the meeting minutes, often including lists of attendees, summaries of staff reports, detailed accounts of committee discussions, policy directives, and voting records (including dissents).
• Chair: The Chair of the Federal Reserve who presided over the meeting documented in the minutes.

Context

This dataset lists the natural rate of unemployment (NAIRU) in the U.S., which is the rate of unemployment arising from all sources except fluctuations in aggregate demand. Estimates of potential GDP are based on the long-term natural rate. The short-term natural rate incorporates structural factors that are temporarily boosting the natural rate beginning in 2008. The short-term natural rate is used to gauge the amount of current and projected slack in labor markets, which is a key input into CBO's projections of inflation.

Content

Data includes the date of the quarterly collection and the natural rate of unemployment from January 1, 1949 through October 1, 2016.

Inspiration

• What is the general trend of unemployment?
• Can you compare this unemployment data with other factors found in any of the BLS databases, such as manufacturing employment rates and GDP?

Acknowledgement

This dataset is part of the US Department of Labor Bureau of Labor Statistics Datasets (the Federal Reserve Economic Data database), and the original source can be found here.

Content

Household is an occupied housing unit. Householder is a person in whose name the housing unit is rented or owned. This person must be at least 15 years old. Family household is a household in which there is at least 1 person present who is related to the householder by birth, marriage or adoption. Family is used to refer to a family household. In general, family consists of those related to each other by birth, marriage or adoption.

This data uses the householder's person weight to describe characteristics of people living in households. As a result, estimates of the number of households do not match estimates of households from the Housing Vacancy Survey (HVS). The HVS is weighted to housing units, rather than the population, in order to more accurately estimate the number of occupied and vacant housing units. For more information about the source and accuracy statement of the Annual Social and Economic Supplement (ASEC) of the Current Population Survey (CPS) see the technical documentation accessible at: http://www.census.gov/programs-surveys/cps/technical-documentation/complete.html

Context

This is a dataset from the U.S. Census Bureau hosted by the Federal Reserve Economic Database (FRED). FRED has a data platform found here and they update their information according the amount of data that is brought in. Explore the U.S. Census Bureau using Kaggle and all of the data sources available through the U.S. Census Bureau organization page!

• Update Frequency: This dataset is updated daily.

• Observation Start: 1950-01-01

• Observation End : 2019-01-01

Acknowledgements

This dataset is maintained using FRED's API and Kaggle's API.