Dataset

This dataset was created by achal2703

Released under Apache 2.0

Contents

The objective of the study is to explore the relationship between country governance practices along with political stability and Economic policy uncertainty, and stock market performance of two different economies, Pakistan and Kurdistan region of Iraq. To meet our objectives, we used the 25 years past data from 1996 to 2021. Data is collected from the DataStream database. The regression analysis is used as the method of estimation for linear and moderation effect. Our results show that regulatory quality, rules of law and political stability has significant positive relationship with stock market performance of Pakistan, but all the governance indicators have significant positive relationship with stock market performance of the Kurdistan Region of Iraq. Moreover, political stability has significant moderating impact between the governance practices and the performance of the stock markets of both economies indicating that the governance practices perform well with the political stability that leads to rise in the stock market indices of selected countries. Economic policy uncertainty has significant negative moderation impact due to creating the risk in both economies that decrease the performance of the stock markets of the selected economies. Finally, our study advocated some implications for the investors to increase their confidence on the stock of high political stability and low economic policy uncertainty economies. Government can take significant measures to control the uncertainty of the policy and portfolio managers can adjust their risk on the ground of the political stability and efficient governance practices countries.

** Overview** This dataset contains stock price data for 5 different stocks along with major market indices (Dow Jones, NASDAQ, and S&P 500). The data has been enhanced with various technical indicators and features commonly used in financial analysis and algorithmic trading.

Dataset Statistics

• Number of rows: 2569
• Number of columns: 221
• Date range: 2015-01-05 to 2025-03-21
• Number of stocks: 5
• Number of market indices: 3

Feature Naming Conventions

• Features ending with numbers (e.g., return_1, close_2) refer to specific stocks (1-5)
• Features with X_Y format (e.g., ma10_3, beta_2_nasdaq_20) have the following pattern:
  • First number/name refers to the parameter or stock
  • Second number/name refers to the stock or index
  • Third number (if present) refers to the time window
• Correlation features (e.g., corr_1_2) show correlation between two stocks (stock 1 and stock 2)

Feature Categories

Basic Price Data

• Date: Trading date in YYYY-MM-DD format
• return_X: Daily return (percentage price change) for stock X (where X is 1-5)
• open_X: Opening price for stock X
• high_X: Highest price during the trading day for stock X
• low_X: Lowest price during the trading day for stock X
• close_X: Closing price for stock X
• adjusted_X: Adjusted closing price for stock X (accounts for dividends and splits)
• volume_X: Trading volume (number of shares traded) for stock X

Market Index Data

• returns_dj: Daily return for Dow Jones Industrial Average
• close_dj: Closing price for Dow Jones Industrial Average
• returns_nasdaq: Daily return for NASDAQ Composite Index
• close_nasdaq: Closing price for NASDAQ Composite Index
• returns_SP500: Daily return for S&P 500 Index
• close_SP500: Closing price for S&P 500 Index

Moving Averages and Trend Indicators

• maX_Y: X-day simple moving average of closing price for stock Y
• emaX_Y: X-day exponential moving average of closing price for stock Y
• envelope_upper_X: Upper price envelope (5% above MA10) for stock X
• envelope_lower_X: Lower price envelope (5% below MA10) for stock X

Momentum and Volatility Indicators

• rocX_Y: X-day Rate of Change (percentage) for stock Y
• volatility_X: 20-day rolling standard deviation of returns for stock X
• rsi_X: 14-day Relative Strength Index for stock X (momentum indicator, 0-100)
• macd_X: Moving Average Convergence Divergence for stock X (ema12 - ema26)
• macd_signal_X: 9-day EMA of MACD for stock X
• macd_hist_X: MACD histogram for stock X (macd - macd_signal)

Volume Indicators

• volume_ma10_X: 10-day moving average of trading volume for stock X
• volume_ratio_X: Ratio of current volume to 10-day volume MA for stock X

Price Ratio Indicators

• high_low_ratio_X: Ratio of high price to low price for stock X (daily range)
• close_open_ratio_X: Ratio of close price to open price for stock X (intraday movement)

Correlation and Beta Indicators

• beta_X_Y_Z: Z-day rolling beta of stock X to index Y (measure of volatility relative to market)
• corr_X_Y: 20-day rolling correlation between returns of stock X and stock Y (ranges from -1 to 1)

Example Usage

import pandas as pd
import matplotlib.pyplot as plt

# Load the dataset
df = pd.read_csv('enhanced_stock_dataset.csv')
df['Date'] = pd.to_datetime(df['Date'])

# Plot closing prices for all stocks
plt.figure(figsize=(12, 6))
for i in range(1, 6):
  plt.plot(df['Date'], df[f'close_{i}'], label=f'Stock {i}')
plt.title('Stock Closing Prices')
plt.xlabel('Date')
plt.ylabel('Price')
plt.legend()
plt.grid(True)
plt.show()

Notes

• This dataset contains engineered features that can be directly used for machine learning models
• All NaN values have been filled using forward and backward filling methods
• The correlation features (corr_X_Y) show the relationship between different stocks
• Beta values show the relationship between each stock and the market indices

This data set contains intraday (1 hour format) correlations for one month (December 2021) from more than 2000 Stocks, Indices, Forex and Futures of major Stock exchanges world wide. It is an example of the outcome from data processing inside Infore project. The data set contains more than 2 million files.

U.S. Economic and Financial Dataset

Dataset Description

This dataset combines historical U.S. economic and financial indicators, spanning the last 50 years, to facilitate time series analysis and uncover patterns in macroeconomic trends. It is designed for exploring relationships between interest rates, inflation, economic growth, stock market performance, and industrial production.

Key Features

• Frequency: Monthly
• Time Period: Last 50 years from Nov-24
• Sources:
  • Federal Reserve Economic Data (FRED)
  • Yahoo Finance

Dataset Feature Description

1. Interest Rate (Interest_Rate):

   • The effective federal funds rate, representing the interest rate at which depository institutions trade federal funds overnight.

2. Inflation (Inflation):

   • The Consumer Price Index for All Urban Consumers, an indicator of inflation trends.

3. GDP (GDP):

   • Real GDP measures the inflation-adjusted value of goods and services produced in the U.S.

4. Unemployment Rate (Unemployment):

   • The percentage of the labor force that is unemployed and actively seeking work.

5. Stock Market Performance (S&P500):

   • Monthly average of the adjusted close price, representing stock market trends.

6. Industrial Production (Ind_Prod):

   • A measure of real output in the industrial sector, including manufacturing, mining, and utilities.

Dataset Statistics

1. Total Entries: 599
2. Columns: 6
3. Memory usage: 37.54 kB
4. Data types: float64

Feature Overview

• Columns:
  • Interest_Rate: Monthly Federal Funds Rate (%)
  • Inflation: CPI (All Urban Consumers, Index)
  • GDP: Real GDP (Billions of Chained 2012 Dollars)
  • Unemployment: Unemployment Rate (%)
  • Ind_Prod: Industrial Production Index (2017=100)
  • S&P500: Monthly Average of S&P 500 Adjusted Close Prices

Executive Summary

This project explores the interconnected dynamics of key macroeconomic indicators and financial market trends over the past 50 years, leveraging data from the Federal Reserve Economic Data (FRED) and Yahoo Finance. The dataset integrates critical variables such as the Federal Funds Rate, Inflation (CPI), Real GDP, Unemployment Rate, Industrial Production, and the S&P 500 Index, providing a holistic view of the U.S. economy and financial markets.

The analysis focuses on uncovering relationships between these variables through time-series visualization, correlation analysis, and trend decomposition. Key findings are included in the Insights section. This project serves as a robust resource for understanding long-term economic trends, policy impacts, and market behavior. It is particularly valuable for students, researchers, policymakers, and financial analysts seeking to connect macroeconomic theory with real-world data.

Potential Use Cases

• Economic Analysis: Examine relationships between interest rates, inflation, GDP, and unemployment.
• Stock Market Prediction: Study how macroeconomic indicators influence stock market trends.
• Time Series Modeling: Perform ARIMA, VAR, or other models to forecast economic trends.
• Cyclic Pattern Analysis: Identify how economic shocks and recoveries impact key indicators.

Snap of Power Analysis

https://github.com/user-attachments/assets/1b40e0ca-7d2e-4fbc-8cfd-df3f09e4fdb8">

To ensure sufficient power, the dataset covers last 50 years of monthly data i.e., around 600 entries.

Key Insights derived through EDA, time-series visualization, correlation analysis, and trend decomposition

• Interest Rate and Inflation Dynamics: The interest Rate and inflation exhibit an inverse relationship, especially during periods of aggressive monetary tightening by the Federal Reserve.
• Economic Growth and Market Performance: GDP growth and the S&P 500 Index show a positive correlation, reflecting how market performance often aligns with overall economic health.
• Labor Market and Industrial Output: Unemployment and industrial production demonstrate a strong inverse relationship. Higher industrial output is typically associated with lower unemployment
• Market Behavior During Economic Shocks: The S&P 500 experienced sharp declines during significant crises, such as the 2008 financial crash and the COVID-19 pandemic in 2020. These events also triggered increased unemployment and contractions in GDP, highlighting the interplay between markets and the broader economy.
• Correlation Highlights: S&P 500 and GDP have a strong positive correlation. Interest rates negatively correlate with GDP and inflation, reflecting monetary policy impacts. Unemployment is negatively correlated with industrial production but positively correlated with interest rates.

Link to GitHub Repo

https:/...

A dataset of key technical indicators for CBOE Implied Correlation Index, including RSI and MACD, used for technical analysis.

Pearson and Kendall’s correlation indices.

Correlation coefficients between two indices and hearing outcomes.

The 500 most cited papers in the computer sciences published between January 2013 and December 2017 were downloaded from the Web of Science (WoS). Data on the number of citations, number of authors, article length and subject sub-discipline were extracted and analyzed in order to identify trends, relationships and common features. Correlations between common factors were analyzed. The 500 papers were cited a total of 10,926 times: the average number of citations per paper was 21.82 citations.

For further information, kindly refer to this paper

Teh P.L., Heard P. (2021) Five Hundred Most-Cited Papers in the Computer Sciences: Trends, Relationships and Common Factors. In: Rocha Á., Adeli H., Dzemyda G., Moreira F., Ramalho Correia A.M. (eds) Trends and Applications in Information Systems and Technologies. WorldCIST 2021. Advances in Intelligent Systems and Computing, vol 1366. Springer, Cham. https://doi.org/10.1007/978-3-030-72651-5_2

The datasets for the Role of Financial Investors on Commodity Futures Risk Premium are weekly datasets for the period from 1995 to 2015 for three commodities in the energy market: crude oil (WTI), heating oil, and natural gas. These datasets contain futures prices for different maturities, open interest positions for each commodity (long and short open interest positions), and S&P 500 composite index. The selected commodities are traded on the New York Mercantile Exchange (NYMEX). The data comes from the Thomson Reuters Datastream and from the Commodity Futures Trading Commission (CFTC).

The topological indices used for determining the value distributions and correlation plots.

1) Data Introduction • The AI Global Index Dataset is a comprehensive index that benchmarks 62 countries based on the level of AI investment, innovation, and implementation, including seven key indicators (human resources, infrastructure, operational environment, research, development, government strategy, commercialization) and general information by country (region, cluster, income group, political system).

2) Data Utilization (1) AI Global Index Dataset has characteristics that: • This dataset consists of a total of 13 columns with 5 categorical variables (regions, clusters, etc.) and 8 numerical variables (scores for each indicator), covering 62 countries. • The seven key indicators are classified into three pillars: △ implementation (human resources/infrastructure/operational environment) △ innovation (R&D) △ investment (government strategy/commercialization), and assess each country's overall AI ecosystem capabilities in multiple dimensions. (2) AI Global Index Dataset can be used to: • Global AI leadership pattern analysis: Correlation analysis between seven indicators can identify AI strengths and weaknesses by country and perform group comparisons by region and income level. • Machine learning-based predictive model: It can be used for data science education and application, such as country-specific index prediction through regression analysis or classification of AI development types through clustering.

Spearman's rank correlation coefficient of annual indices in bag and IWC data (N = 23).

MB-N: Microbial biomass nitrogen. TRAP: Acid Phosphatase. PPO: Polyphenol Oxidase.

This is a simulated data set containing 1000 genotypes and 25 highly correlated traits, to be used in the Monte Carlo simulation of the draft paper "MGIDI: towards an effective multivariate selection in biological experiments" by Tiago Olivoto and Maicon Nardino

The dataset contains a high correlation network of stocks in the Nasdaq100 index. Correlation is calculated based on the closing price of the stocks from 1/04/2020 to 31/03/2022.

Pearson correlation of HRV indices with potential factors and physical mobility, as assessed through regression analyses.

The US_Stock_Data.csv dataset offers a comprehensive view of the US stock market and related financial instruments, spanning from January 2, 2020, to February 2, 2024. This dataset includes 39 columns, covering a broad spectrum of financial data points such as prices and volumes of major stocks, indices, commodities, and cryptocurrencies. The data is presented in a structured CSV file format, making it easily accessible and usable for various financial analyses, market research, and predictive modeling. This dataset is ideal for anyone looking to gain insights into the trends and movements within the US financial markets during this period, including the impact of major global events.

Key Features and Data Structure

The dataset captures daily financial data across multiple assets, providing a well-rounded perspective of market dynamics. Key features include:

• Commodities: Prices and trading volumes for natural gas, crude oil, copper, platinum, silver, and gold.
• Cryptocurrencies: Prices and volumes for Bitcoin and Ethereum, including detailed 5-minute interval data for Bitcoin.
• Stock Market Indices: Data for major indices such as the S&P 500 and Nasdaq 100.
• Individual Stocks: Prices and volumes for major companies including Apple, Tesla, Microsoft, Google, Nvidia, Berkshire Hathaway, Netflix, Amazon, and Meta.

The dataset’s structure is designed for straightforward integration into various analytical tools and platforms. Each column is dedicated to a specific asset's daily price or volume, enabling users to perform a wide range of analyses, from simple trend observations to complex predictive models. The inclusion of intraday data for Bitcoin provides a detailed view of market movements.

Applications and Usability

This dataset is highly versatile and can be utilized for various financial research purposes:

• Market Analysis: Track the performance of key assets, compare volatility, and study correlations between different financial instruments.
• Risk Assessment: Analyze the impact of commodity price movements on related stock prices and evaluate market risks.
• Educational Use: Serve as a resource for teaching market trends, asset correlation, and the effects of global events on financial markets.

The dataset’s daily updates ensure that users have access to the most current data, which is crucial for real-time analysis and decision-making. Whether for academic research, market analysis, or financial modeling, the US_Stock_Data.csv dataset provides a valuable foundation for exploring the complexities of financial markets over the specified period.

Acknowledgements:

This dataset would not be possible without the contributions of Dhaval Patel, who initially curated the US stock market data spanning from 2020 to 2024. Full credit goes to Dhaval Patel for creating and maintaining the dataset. You can find the original dataset here: US Stock Market 2020 to 2024.

Context

This dataset was actually made to check the correlations between a housing price index and its crime rate. Rise and fall of housing prices can be due to various factors with obvious reasons being the facilities of the house and its neighborhood. Think of a place like Detroit where there are hoodlums and you don't want to end up buying a house in the wrong place. This data set will serve as historical data for crime rate data and this in turn can be used to predict whether the housing price will rise or fall. Rise in housing price will suggest decrease in crime rate over the years and vice versa.

Content

The headers are self explanatory. index_nsa is the housing price non seasonal index.

Acknowledgements

Thank you to my team who helped in achieving this.

Inspiration

https://www.kaggle.com/marshallproject/crime-rates https://catalog.data.gov/dataset/fhfa-house-price-indexes-hpis Data was collected from these 2 sources and merged to get the resulting dataset.

Drawing on a long history in macroecology, correlation analysis of microbiome datasets is becoming a common practice for identifying relationships or shared ecological niches among bacterial taxa. However, many of the statistical issues that plague such analyses in macroscale communities remain unresolved for microbial communities. Here, we discuss problems in the analysis of microbial species correlations based on presence-absence data. We focus on presence-absence data because this information is more readily obtainable from sequencing studies, especially for whole-genome sequencing, where abundance estimation is still in its infancy. First, we show how Pearson’s correlation coefficient (r) and Jaccard’s index (J)–two of the most common metrics for correlation analysis of presence-absence data–can contradict each other when applied to a typical microbiome dataset. In our dataset, for example, 14% of species-pairs predicted to be significantly correlated by r were not predicted to be significantly correlated using J, while 37.4% of species-pairs predicted to be significantly correlated by J were not predicted to be significantly correlated using r. Mismatch was particularly common among species-pairs with at least one rare species (