About:

This dataset encompasses the historical data of major stock indices from around the world, sourced directly from Yahoo Finance. With data reaching back to the early 1920s (where available), it serves as an invaluable repository for academic researchers, financial analysts, and market enthusiasts. Users can delve into trends across decades, evaluate historical market behaviors, or even design and validate predictive financial models.

Photo by Tötös Ádám on Unsplash

Info on CSVs:

1. all_indices_data.csv:

   • Description: A consolidated dataset merging all the stock indices from Yahoo Finance.
   • Columns:
     • date: The date of the data point (formatted as YYYY-MM-DD).
     • open: The opening value of the index on that date.
     • high: The highest value of the index during the trading session.
     • low: The lowest value of the index during the trading session.
     • close: The closing value of the index.
     • volume: The trading volume of the index on that date.
     • ticker: The ticker symbol of the stock index.

2. individual_indices_data/[SYMBOL]_data.csv:

   • Description: Individual datasets for each stock index, where [SYMBOL] denotes the ticker symbol of the respective stock index. Each dataset is curated from Yahoo Finance's historical data archives.
   • Columns:
     • date: The date of the data point (formatted as YYYY-MM-DD).
     • open: The opening value of the index on that date.
     • high: The highest value of the index during the trading session.
     • low: The lowest value of the index during the trading session.
     • close: The closing value of the index.
     • volume: The trading volume of the index on that date.

What is this data?

This is a yahoo finance mapper for world indices. You can use this file to fetch the historical data using the YFinance API.

Original data?

https://finance.yahoo.com/world-indices

This is the name of the 38 global main stock indexes in the world. We collected from Yahoo! Finance. For the convenience of expression and computation later, we numbered it. For each item, the front is its serial number, followed by the corresponding stock index.

Context:

The provided dataset is extracted from yahoo finance using pandas and yahoo finance library in python. This deals with stock market index of the world best economies. The code generated data from Jan 01, 2003 to Jun 30, 2023 that’s more than 20 years. There are 18 CSV files, dataset is generated for 16 different stock market indices comprising of 7 different countries. Below is the list of countries along with number of indices extracted through yahoo finance library, while two CSV files deals with annualized return and compound annual growth rate (CAGR) has been computed from the extracted data.

Number of Countries & Index:

https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F15657145%2F90ce8a986761636e3edbb49464b304d8%2FNumber%20of%20Index.JPG?generation=1688490342207096&alt=media" alt="">

Content:

Unit of analysis: Stock Market Index Analysis

This dataset is useful for research purposes, particularly for conducting comparative analyses involving capital market performance and could be used along with other economic indicators.

There are 18 distinct CSV files associated with this dataset. First 16 CSV files deals with number of indices and last two CSV file deals with annualized return of each year and CAGR of each index. If data in any column is blank, it portrays that index was launch in later years, for instance: Bse500 (India), this index launch in 2007, so earlier values are blank, similarly China_Top300 index launch in year 2021 so early fields are blank too.

The extraction process involves applying different criteria, like in 16 CSV files all columns are included, Adj Close is used to calculate annualized return. The algorithm extracts data based on index name (code given by the yahoo finance) according start and end date.

Annualized return and CAGR has been calculated and illustrated in below image along with machine readable file (CSV) attached to that.

To extract the data provided in the attachment, various criteria were applied:

1. Content Filtering: The data was filtered based on several attributes, including the index name, start and end date. This filtering process ensured that only relevant data meeting the specified criteria.

2. Collaborative Filtering: Another filtering technique used was collaborative filtering using yahoo finance, which relies on index similarity. This approach involves finding indices that are similar to other index or extended dataset scope to other countries or economies. By leveraging this method, the algorithm identifies and extracts data based on similarities between indices.

In the last two CSV files, one belongs to annualized return, that was calculated based on the Adj close column and new DataFrame created to store its outcome. Below is the image of annualized returns of all index (if unreadable, machine-readable or CSV format is attached with the dataset).

Annualized Return:

As far as annualised rate of return is concerned, most of the time India stock market indices leading, followed by USA, Canada and Japan stock market indices.

https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F15657145%2F37645bd90623ea79f3708a958013c098%2FAnnualized%20Return.JPG?generation=1688525901452892&alt=media" alt="">

Compound Annual Growth Rate (CAGR):

The best performing index based on compound growth is Sensex (India) that comprises of top 30 companies is 15.60%, followed by Nifty500 (India) that is 11.34% and Nasdaq (USA) all is 10.60%.

The worst performing index is China top300, however this is launch in 2021 (post pandemic), so would not possible to examine at that stage (due to less data availability). Furthermore, UK and Russia indices are also top 5 in the worst order.

https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F15657145%2F58ae33f60a8800749f802b46ec1e07e7%2FCAGR.JPG?generation=1688490409606631&alt=media" alt="">

Geography: Stock Market Index of the World Top Economies

Time period: Jan 01, 2003 – June 30, 2023

Variables: Stock Market Index Title, Open, High, Low, Close, Adj Close, Volume, Year, Month, Day, Yearly_Return and CAGR

File Type: CSV file

Inspiration:

• Time series prediction model
• Investment opportunities in world best economies
• Comparative Analysis of past data with other stock market indices or other indices

Disclaimer:

This is not a financial advice; due diligence is required in each investment decision.

Dataset

This dataset was created by EL Younes

Contents

Collected from Yahoo Finance, Investing.com and WJS, this dataset consists of the following indices ranging from July 17, 2017 to July 22, 2022:

1. DJI (Dow Jones Industrial Average)
2. SNP (Standard and Poor's 500)
3. IXIC (Nasdaq Composite)
4. VIX (CBOE Volatility Index)
5. FTSE (Financial Times Stock Exchange)
6. FCHI (CAC 40 Paris Index)
7. STOXX (The STOXX Europe 600)
8. AEX (Amsterdam Exchange Index)
9. IBEX (Iberian Index, Madrid)
10. MOEX (Russia Index)
11. BIST (Istanbul Index)
12. HSI (Hang Seng Index)
13. SSE (Shanghai Composite Index)
14. STI (Straits Times Index)
15. SZSE (Shenzhen Stock Exchange)
16. NIK (Nikkei 225 Index)
17. TWII (Taiwan Weighted)
18. JKSE (Jakarta Composite Index)

Content

Daily price data for World indices stock exchanges from all over the world (United States, China, Canada, Germany, Japan, and more). The data was all collected from Yahoo Finance, which had several decades of data available for most exchanges. Prices are quoted in terms of the USD currency of where each exchange is located.

Acknowledgement

Data collected from Yahoo Finance.

Context

Stocks and financial instrument trading is a lucrative proposition. Stock markets across the world facilitate such trades and thus wealth exchanges hands. Stock prices move up and down all the time and having ability to predict its movement has immense potential to make one rich. Stock price prediction has kept people interested from a long time. There are hypothesis like the Efficient Market Hypothesis, which says that it is almost impossible to beat the market consistently and there are others which disagree with it.

There are a number of known approaches and new research going on to find the magic formula to make you rich. One of the traditional methods is the time series forecasting. Fundamental analysis is another method where numerous performance ratios are analyzed to assess a given stock. On the emerging front, there are neural networks, genetic algorithms, and ensembling techniques.

Another challenging problem in stock price prediction is Black Swan Event, unpredictable events that cause stock market turbulence. These are events that occur from time to time, are unpredictable and often come with little or no warning.

A black swan event is an event that is completely unexpected and cannot be predicted. Unexpected events are generally referred to as black swans when they have significant consequences, though an event with few consequences might also be a black swan event. It may or may not be possible to provide explanations for the occurrence after the fact – but not before. In complex systems, like economies, markets and weather systems, there are often several causes. After such an event, many of the explanations for its occurrence will be overly simplistic.

#
#

https://www.visualcapitalist.com/wp-content/uploads/2020/03/mm3_black_swan_events_shareable.jpg"> #
#
New bleeding age state-of-the-art deep learning models stock predictions is overcoming such obstacles e.g. "Transformer and Time Embeddings". An objectives are to apply these novel models to forecast stock price.

Content

Stock price prediction is the task of forecasting the future value of a given stock. Given the historical daily close price for S&P 500 Index, prepare and compare forecasting solutions. S&P 500 or Standard and Poor's 500 index is an index comprising of 500 stocks from different sectors of US economy and is an indicator of US equities. Other such indices are the Dow 30, NIFTY 50, Nikkei 225, etc. For the purpose of understanding, we are utilizing S&P500 index, concepts, and knowledge can be applied to other stocks as well.

Dataset

The historical stock price information is also publicly available. For our current use case, we will utilize the pandas_datareader library to get the required S&P 500 index history using Yahoo Finance databases. We utilize the closing price information from the dataset available though other information such as opening price, adjusted closing price, etc., are also available. We prepare a utility function get_raw_data() to extract required information in a pandas dataframe. The function takes index ticker name as input. For S&P 500 index, the ticker name is ^GSPC. The following snippet uses the utility function to get the required data.(See Simple LSTM Regression)

Features and Terminology: In stock trading, the high and low refer to the maximum and minimum prices in a given time period. Open and close are the prices at which a stock began and ended trading in the same period. Volume is the total amount of trading activity. Adjusted values factor in corporate actions such as dividends, stock splits, and new share issuance.

Starter Kernel(s)

• Simple LSTM Regression

Acknowledgements

Mining and updating of this dateset will depend upon Yahoo Finance .

Inspiration

Sort of variation of sequence modeling and bleeding age e.g. attention can be applied for research and forecasting

Some Readings

• Applications of deep learning in stock market prediction: recent progress
• Stock predictions with state-of-the-art Transformer and Time Embeddings
• https://arxiv.org/pdf/1712.02136.pdf
• https://catanacapital.com/blog/unpredictable-black-swan-event-stock-market/
• https://par.nsf.gov/servlets/purl/10149715
• https://arxiv.org/pdf/1712.02136.pdf

*If you download and find the data useful your upvote is an explicit feedback for future works*

Overview This dataset provides daily snapshots of cryptocurrency, stock market, and forex data.

Sources Yahoo Finance (via yfinance)

Features Automated daily updates Covers major global indices and top cryptocurrencies Includes sentiment analysis for financial news

Use Cases Financial market analysis Machine learning for price prediction Trading strategy research

License Data compiled from public APIs for educational and analytical use.

This dataset is refreshed using yahoo finance backend api executed from my analytics application. I use the data to build graphical models for various indicators such as SMA, Stochastics, RSI. The information stored in the dataset is also used to develop buy and sell strategy. The application also shows various interactive graphs using the dataset. All graphs are developed using google graphs.

Content

Top Companies of NASDAQ 100 in 2022

Here are the top companies on the NASDAQ 100 index in 2022. NASDAQ 100 is one of the most prominent large-cap growth indices in the world.

Many companies listed in the NASDAQ 100 operate in the tech sector. That is why many investors who are focused investing in tech stocks also invest in NASDAQ index to grow their funds

What is NASDAQ 100?

NASDAQ 100 is a stock market index composed of the 100 largest and most actively traded companies in the United States of America in the non- financial sector and are segmented under technology, retail, industrial, biotechnology, health care, telecom, transportation, media and services sectors.

Acknowledgement

Data collected from Yahoo Finance.

This dataset consists of five CSV files that provide detailed data on a stock portfolio and related market performance over the last 5 years. It includes portfolio positions, stock prices, and major U.S. market indices (NASDAQ, S&P 500, and Dow Jones). The data is essential for conducting portfolio analysis, financial modeling, and performance tracking.

1. Portfolio

This file contains the portfolio composition with details about individual stock positions, including the quantity of shares, sector, and their respective weights in the portfolio. The data also includes the stock's closing price.

• Columns:
  • Ticker: The stock symbol (e.g., AAPL, TSLA)
  • Quantity: The number of shares in the portfolio
  • Sector: The sector the stock belongs to (e.g., Technology, Healthcare)
  • Close: The closing price of the stock
  • Weight: The weight of the stock in the portfolio (as a percentage of total portfolio)

2. Portfolio Prices

This file contains historical pricing data for the stocks in the portfolio. It includes daily open, high, low, close prices, adjusted close prices, returns, and volume of traded stocks.

• Columns:
  • Date: The date of the data point
  • Ticker: The stock symbol
  • Open: The opening price of the stock on that day
  • High: The highest price reached on that day
  • Low: The lowest price reached on that day
  • Close: The closing price of the stock
  • Adjusted: The adjusted closing price after stock splits and dividends
  • Returns: Daily percentage return based on close prices
  • Volume: The volume of shares traded that day

3. NASDAQ

This file contains historical pricing data for the NASDAQ Composite index, providing similar data as in the Portfolio Prices file, but for the NASDAQ market index.

• Columns:
  • Date: The date of the data point
  • Ticker: The stock symbol (for NASDAQ index, this will be "IXIC")
  • Open: The opening price of the index
  • High: The highest value reached on that day
  • Low: The lowest value reached on that day
  • Close: The closing value of the index
  • Adjusted: The adjusted closing value after any corporate actions
  • Returns: Daily percentage return based on close values
  • Volume: The volume of shares traded

4. S&P 500

This file contains similar historical pricing data, but for the S&P 500 index, providing insights into the performance of the top 500 U.S. companies.

• Columns:
  • Date: The date of the data point
  • Ticker: The stock symbol (for S&P 500 index, this will be "SPX")
  • Open: The opening price of the index
  • High: The highest value reached on that day
  • Low: The lowest value reached on that day
  • Close: The closing value of the index
  • Adjusted: The adjusted closing value after any corporate actions
  • Returns: Daily percentage return based on close values
  • Volume: The volume of shares traded

5. Dow Jones

This file contains similar historical pricing data for the Dow Jones Industrial Average, providing insights into one of the most widely followed stock market indices in the world.

• Columns:
  • Date: The date of the data point
  • Ticker: The stock symbol (for Dow Jones index, this will be "DJI")
  • Open: The opening price of the index
  • High: The highest value reached on that day
  • Low: The lowest value reached on that day
  • Close: The closing value of the index
  • Adjusted: The adjusted closing value after any corporate actions
  • Returns: Daily percentage return based on close values
  • Volume: The volume of shares traded

Personal Portfolio Data

This data is received using a custom framework that fetches real-time and historical stock data from Yahoo Finance. It provides the portfolio’s data based on user-specific stock holdings and performance, allowing for personalized analysis. The personal framework ensures the portfolio data is automatically retrieved and updated with the latest stock prices, returns, and performance metrics.

This part of the dataset would typically involve data specific to a particular user’s stock positions, weights, and performance, which can be integrated with the other files for portfolio performance analysis.

📝 Description This dataset contains the daily trading history of the BSE SENSEX index from January 1, 2000 through December 31, 2024, sourced via Yahoo Finance. Each record includes: 🍀 Open, High, Low, Close index levels 🍀 Adjusted Close (to account for corporate actions) 🍀 Volume of shares traded

Key features: 🍀 Coverage: ~6,150 trading days (Mon–Fri, excluding exchange holidays) 🍀 Format: Single CSV file (sensex_2000_2024.csv) with a Date column and six numeric columns 🍀 Use cases: 🍀 Back‑testing equity strategies 🍀 Teaching time‑series and econometrics 🍀 Correlating Indian markets with global indices 🍀 Building financial dashboards and visualizations

The dataset contains 41265 observations and 21 variables. Each row represents a specific observation or data point. The variables in the dataset include: hpi_type: Type of housing price index data (e.g., traditional, developmental, distress-free, non-metro). hpi_flavor: Flavor of the housing price index data (e.g., purchase-only, all-transactions, expanded-data). frequency: Frequency of the data (e.g., monthly, quarterly). level: Level of geography (e.g., USA or Census Division, State, MSA, Puerto Rico). place_name: Name of the place (e.g., region, state, metropolitan area). place_id: Identifier for the place (e.g., abbreviation, CBSA code). yr: Year of the data. period: Period of the data (e.g., month, quarter). index_nsa: Index, non seasonally adjusted. index_sa: Index, seasonally adjusted. Gross domestic product, constant prices: GDP at constant prices in national currency. Gross domestic product per capita, constant prices: GDP per capita at constant prices. Gross domestic product per capita, current prices: GDP per capita at current prices. Gross domestic product based on purchasing-power-parity (PPP) share of world total: GDP based on PPP as a share of world total GDP. Inflation, average consumer prices: Average consumer price inflation index. Volume of imports of goods and services: Volume change in imports of goods and services. Volume of exports of goods and services: Volume change in exports of goods and services. Unemployment rate: Percentage of total labor force unemployed. Current account balance: Balance of payments current account balance. Date: Date of the data. GSPC.Close: Closing price of the S&P 500 index.

Context

Time series modelling for the prediction of stocks prices is a challenging task. Political events, market expectations and economic factors are just a few known factors that can impact financial market behaviour. The financial market is a complex, noisy, evolutionary and chaotic field of study that attracts many enthusiasts and researches — the first, usually driven by the economic benefit of it, the latter, inspired by the challenge of handling such complex data.

This project aims to predict Facebook (FB) next day stock price direction with machine learning algorithms. Technical indicators and global market indexes are used, and their influence on the forecast accuracy is analysed.

Content

Daily values were retrieved (volume, open, close, low and high prices) from Yahoo! Finance website. For Facebook (FB), July 2012 was the earliest data available. The date range is July 2012 to November 2018.

The closing price of current day C(t) and closing price from the previous day C(t-1) are compared to build the initial dataset. The objective is to define if the price trend is going up or down by analysing these two values. For each instance, a comparison was made and recorded. If the price is going up, C(t) > C(t-1), class “1” is assigned. Class “0” is assigned for the opposite case.

• ID: Sample ID
• Close: Closing value of previous day
• Low: Lowest value of previous day
• High: Highest value of previous day
• Volume: Volume value of previous day

Research was initiated to understand which features could help the model to forecast the stock direction. Three main routes were found: Lag features, Technical Indicators and Global Market Indexes. Below is an explanation of each group of features.

Lag features are features that contain the closing price and direction of previous days and it is a common strategy for Time Series models. The following features were added:

• C(t-5): Closing price of 5 days before
• C(t-4): Closing price of 4 days before
• C(t-3): Closing price of 3 days before
• C(t-2): Closing price of 2 days before
• C_up_4: Output 1 if closing price went up 4 days ago
• C_up_3: Output 1 if closing price went up 3 days ago
• C_up_2: Output 1 if closing price went up 2 days ago
• C_up_1: Output 1 if closing price went up 1 day ago

Technical indicators are used by researches and financial market analysts to support stock market trend forecasting. Common indicators retrieved from the literature were selected and calculated for Facebook stock. Techical Indicators added:

• MA-10: Moving Average considering previous 10 days
• MA-5: Moving Average considering previous 5 days
• WMA-10: Weighted Moving Average considering previous 10 days
• SO: Stochastic Oscillator
• M: Momentum as the difference in closing price in a 10 days interval
• SSO: Slow Stochastic Oscillator
• EMA: Exponential Moving Average for a 10 day period
• MACD_Sline_9: MACD Signal Line for a 9 day period
• RSI: Relative Strength Index
• CCI: Commodity Channel Index
• ADO: Accumulation Distribution Oscillator

Technical indicators provide a suggestion of the stock price movement. Additional features were created for each technical indicator by analysing its daily value and assigning a class according to their meaning. Class “1” is given if the indicator numerical value suggests upper trend, class “0” for a downtrend. In other words, financial market analysis is performed at a simplistic level, in the attempt to translate what the continuous value means.

• MA-10>C: If MA-10 is higher than Closing price output 1
• MA-5>C: If MA-5 is higher than Closing price output 1
• WMA-10>C: If WMA-10 is higher than Closing price output 1
• SO>SOt-1: Output is 1 if SO current value is higher than previous day
• M>0: A positive momentum outputs 1
• SSO>SSOt-1: SSO current value is higher than previous day
• EMA>C: If EMA is higher than Closing price output 1
• MACD>MACDt-1: If MACD current value is higher than previous day output 1
• RSI70-30: If RSI is above 70, output 0. Values below 30 output is one. For values within this range it compares to previous day and outputs 1 if value has increased
• CCI200-200: Similar to RSI, but if threshold set for 200 and -200.
• ADO>ADOt-1: Output is 1 if ADO current value is higher than previous day

For a given country or region, the stock market index characterises the performance of its financial market and the overall local economy. For this reason, the same day performance of these markets could contribute to the machine learning model predictions. Six global indexes were added as features, with their closing direction as up or down, class “1” or “0”, respectively. Data for these indexes (Nikkei, Hang Seng, All Ordinaries, Euronext 100, SSE and DAX) were also retrieved from Yahoo! Finance.

The S&P 100 index is a collection of 101 constituent companies representing a diverse range of industries and sectors, including technology, finance, healthcare, and more. These companies, such as Apple, Nvidia, and Accenture, among others, have been selected based on their market capitalization, liquidity, and other factors.

The dataset constructed from Yahoo Finance data offers a comprehensive view of the daily close price variations of all 101 constituent companies over a span of 23 years, from 2000 to the present day. This dataset provides valuable insights into the performance and volatility of individual companies as well as the overall index.

Analyzing this dataset can reveal trends, patterns, and correlations within and across industries. It allows investors, analysts, and researchers to study the dynamics of the S&P 100 index and make informed decisions based on historical price movements.

By examining the historical price data, one can identify periods of growth, market fluctuations, and potential opportunities for investment. This dataset offers a wealth of information that can be leveraged for quantitative analysis, modeling, and developing trading strategies.

Overall, the S&P 100 dataset provides a captivating journey into the world of finance, offering a rich and comprehensive resource for understanding the performance of major companies across various sectors over a significant timeframe.

With this extensive dataset at your disposal, you can: - delve into long-term analysis of the performance of these prominent companies - gain valuable insights into the ebb and flow of daily price fluctuations and uncover patterns, trends, and market dynamics that have shaped the S&P 100 index over the years. - do forecasting analysis

Whether you are a seasoned investor, financial analyst, or data enthusiast, this dataset provides a valuable resource for studying the evolution of stock prices, conducting trend analysis, and making informed decisions in the ever-changing landscape of finance.

Embark on an enlightening journey of discovery as you explore the rich history and intricate price movements of the S&P 100 companies, and leverage this dataset to gain insights that can fuel your investment strategies, quantitative models, and forecasting techniques

About Company:

Founded in 1946 in Alba (Italy) by the young chocolatier Pietro Ferrero, Michele’s father, the Ferrero Group is a family-run business and one of the largest chocolates and confectionery companies in the world. Today, our reach extends across 55 countries on 5 continents with iconic brands distributed in over 170 countries. With his global vision, built upon our heritage, Giovanni Ferrero continues to run the company successfully. Generation after generation, our commitment to creating value forms the basis for crafting our much-loved products in an ethical and socially conscious manner.

About Dataset:

• Date: A date is a particular day of the month.
• Open: It is the price at which the financial security opens in the market when trading begins. It may or may not be different from the previous day's closing price. Thus, the price in the beginning of trading sessions is called open price or simply open.
• High: Today's high refers to a security's intraday highest trading price. It is represented by the highest point on a day's stock chart. This can be contrasted with today's low, which is the trading day's intraday low price.
• Low: The low is the minimum price of a stock in a period, while high is the maximum value reached by the stock in the same period.
• Close: The close is a reference to the end of a trading session in the financial markets when the markets close for the day. The close can also refer to the process of exiting a trade or the final procedure in a financial transaction in which contract documents are signed and recorded.
• Adj Close: The adjusted closing price amends a stock's closing price to reflect that stock's value after accounting for any corporate actions. The closing price is the raw price, which is just the cash value of the last transacted price before the market closes. Volume: In capital markets, volume, or trading volume, is the amount of a security that was traded during a given period of time. In the context of a single stock trading on a stock exchange, the volume is commonly reported as the number of shares that changed hands during a given day.

How can you use this dataset:

• This dataset is present in csv format, you can use pandas to read and manipulate csv and can perform time series analysis on this dataset.
• You can also perform EDA to get insight of the dataset.

Source:

• This dataset is collected from Yahoo finance.