In the dynamic landscape of financial markets, accurate forecasting of stock indices remains a pivotal yet challenging task, essential for investors and policymakers alike. This study is motivated by the need to enhance the precision of predicting the Shanghai Composite Index’s opening price spread, a critical measure reflecting market volatility and investor sentiment. Traditional time series models like ARIMA have shown limitations in capturing the complex, nonlinear patterns inherent in stock price movements, prompting the exploration of advanced methodologies. The aim of this research is to bridge the gap in forecasting accuracy by developing a hybrid model that integrates the strengths of ARIMA with deep learning techniques, specifically Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) networks. This novel approach leverages the ARIMA model’s proficiency in linear trend analysis and the deep learning models’ capability in modeling nonlinear dependencies, aiming to provide a comprehensive tool for market prediction. Utilizing a comprehensive dataset covering the period from December 20, 1990, to June 2, 2023, the study develops and assesses the efficacy of ARIMA, LSTM, GRU, ARIMA-LSTM, and ARIMA-GRU models in forecasting the Shanghai Composite Index’s opening price spread. The evaluation of these models is based on key statistical metrics, including Mean Squared Error (MSE) and Mean Absolute Error (MAE), to gauge their predictive accuracy. The findings indicate that the hybrid models, ARIMA-LSTM and ARIMA-GRU, perform better in forecasting the opening price spread of the Shanghai Composite Index than their standalone counterparts. This outcome suggests that combining traditional statistical methods with advanced deep learning algorithms can enhance stock market prediction. The research contributes to the field by providing evidence of the potential benefits of integrating different modeling approaches for financial forecasting, offering insights that could inform investment strategies and financial decision-making.

This analysis presents a rigorous exploration of financial data, incorporating a diverse range of statistical features. By providing a robust foundation, it facilitates advanced research and innovative modeling techniques within the field of finance.

Short/Long Term Stocks: Dow Jones New Zealand Index Stock Forecast

Financial data:

• Historical daily stock prices (open, high, low, close, volume)

• Fundamental data (e.g., market capitalization, price to earnings P/E ratio, dividend yield, earnings per share EPS, price to earnings growth, debt-to-equity ratio, price-to-book ratio, current ratio, free cash flow, projected earnings growth, return on equity, dividend payout ratio, price to sales ratio, credit rating)

• Technical indicators (e.g., moving averages, RSI, MACD, average directional index, aroon oscillator, stochastic oscillator, on-balance volume, accumulation/distribution A/D line, parabolic SAR indicator, bollinger bands indicators, fibonacci, williams percent range, commodity channel index)

Machine learning features:

• Feature engineering based on financial data and technical indicators

• Sentiment analysis data from social media and news articles

• Macroeconomic data (e.g., GDP, unemployment rate, interest rates, consumer spending, building permits, consumer confidence, inflation, producer price index, money supply, home sales, retail sales, bond yields)

Potential Applications:

• Stock price prediction

• Portfolio optimization

• Algorithmic trading

• Market sentiment analysis

• Risk management

Use Cases:

• Researchers investigating the effectiveness of machine learning in stock market prediction

• Analysts developing quantitative trading Buy/Sell strategies

• Individuals interested in building their own stock market prediction models

• Students learning about machine learning and financial applications

Additional Notes:

• The dataset may include different levels of granularity (e.g., daily, hourly)

• Data cleaning and preprocessing are essential before model training

• Regular updates are recommended to maintain the accuracy and relevance of the data

This analysis presents a rigorous exploration of financial data, incorporating a diverse range of statistical features. By providing a robust foundation, it facilitates advanced research and innovative modeling techniques within the field of finance.

CPZ Calamos Long/Short Equity & Dynamic Income Trust Common Stock

Financial data:

• Historical daily stock prices (open, high, low, close, volume)

• Fundamental data (e.g., market capitalization, price to earnings P/E ratio, dividend yield, earnings per share EPS, price to earnings growth, debt-to-equity ratio, price-to-book ratio, current ratio, free cash flow, projected earnings growth, return on equity, dividend payout ratio, price to sales ratio, credit rating)

• Technical indicators (e.g., moving averages, RSI, MACD, average directional index, aroon oscillator, stochastic oscillator, on-balance volume, accumulation/distribution A/D line, parabolic SAR indicator, bollinger bands indicators, fibonacci, williams percent range, commodity channel index)

Machine learning features:

• Feature engineering based on financial data and technical indicators

• Sentiment analysis data from social media and news articles

• Macroeconomic data (e.g., GDP, unemployment rate, interest rates, consumer spending, building permits, consumer confidence, inflation, producer price index, money supply, home sales, retail sales, bond yields)

Potential Applications:

• Stock price prediction

• Portfolio optimization

• Algorithmic trading

• Market sentiment analysis

• Risk management

Use Cases:

• Researchers investigating the effectiveness of machine learning in stock market prediction

• Analysts developing quantitative trading Buy/Sell strategies

• Individuals interested in building their own stock market prediction models

• Students learning about machine learning and financial applications

Additional Notes:

• The dataset may include different levels of granularity (e.g., daily, hourly)

• Data cleaning and preprocessing are essential before model training

• Regular updates are recommended to maintain the accuracy and relevance of the data

This analysis presents a rigorous exploration of financial data, incorporating a diverse range of statistical features. By providing a robust foundation, it facilitates advanced research and innovative modeling techniques within the field of finance.

Short/Long Term Stocks: Tadawul All Share Index Stock Forecast

Financial data:

• Historical daily stock prices (open, high, low, close, volume)

• Fundamental data (e.g., market capitalization, price to earnings P/E ratio, dividend yield, earnings per share EPS, price to earnings growth, debt-to-equity ratio, price-to-book ratio, current ratio, free cash flow, projected earnings growth, return on equity, dividend payout ratio, price to sales ratio, credit rating)

• Technical indicators (e.g., moving averages, RSI, MACD, average directional index, aroon oscillator, stochastic oscillator, on-balance volume, accumulation/distribution A/D line, parabolic SAR indicator, bollinger bands indicators, fibonacci, williams percent range, commodity channel index)

Machine learning features:

• Feature engineering based on financial data and technical indicators

• Sentiment analysis data from social media and news articles

• Macroeconomic data (e.g., GDP, unemployment rate, interest rates, consumer spending, building permits, consumer confidence, inflation, producer price index, money supply, home sales, retail sales, bond yields)

Potential Applications:

• Stock price prediction

• Portfolio optimization

• Algorithmic trading

• Market sentiment analysis

• Risk management

Use Cases:

• Researchers investigating the effectiveness of machine learning in stock market prediction

• Analysts developing quantitative trading Buy/Sell strategies

• Individuals interested in building their own stock market prediction models

• Students learning about machine learning and financial applications

Additional Notes:

• The dataset may include different levels of granularity (e.g., daily, hourly)

• Data cleaning and preprocessing are essential before model training

• Regular updates are recommended to maintain the accuracy and relevance of the data

This dataset contains replication files for "The Surrogate Index: Combining Short-Term Proxies to Estimate Long-Term Treatment Effects More Rapidly and Precisely" by Susan Athey, Raj Chetty, Guido Imbens, and Hyunseung Kang. For more information, see https://opportunityinsights.org/paper/the-surrogate-index/. A summary of the related publication follows. The impacts of many policies, such as efforts to increase upward income mobility or improve health outcomes, are only observed with long delays. For example, it can take decades to see the effects of early childhood interventions on lifetime earnings. This problem has greatly limited researchers’ and policymakers’ ability to test and improve policies and arises frequently in our own work at Opportunity Insights on the determinants of economic opportunity. In this study, we develop a new method of estimating the long-term impacts of policies more rapidly and precisely using short-term proxies. We predict long-term outcomes (e.g., lifetime earnings) using short-term outcomes (e.g., earnings in early adulthood or test scores). We then show that the causal effects of policies on this predictive index (which we term a “surrogate index”, following terminology in the statistics literature) can help us learn about their long-term impacts more quickly under certain assumptions that are described in the full paper. We apply our method to analyze the long-term impacts of a job training experiment in California. Using short-term employment rates as surrogates, we show that one could have estimated the program’s impact on mean employment rates over a 9 year horizon within 1.5 years, with a 35% reduction in standard errors. The success of the surrogate index in this job training application suggests that our method could be applied to predict the long-term impacts of other programs as well. Going forward, we hope to build a public library of early indicators (surrogate indices) for social science by harnessing historical experiments along with the large-scale datasets we have built. If you would like to contribute to this effort by reporting a surrogate index that predicts long-term impacts estimated in an experiment, as in the GAIN program, please contact us.

In this paper we model Value-at-Risk (VaR) for daily asset returns using a collection of parametric univariate and multivariate models of the ARCH class based on the skewed Student distribution. We show that models that rely on a symmetric density distribution for the error term underperform with respect to skewed density models when the left and right tails of the distribution of returns must be modelled. Thus, VaR for traders having both long and short positions is not adequately modelled using usual normal or Student distributions. We suggest using an APARCH model based on the skewed Student distribution (combined with a time-varying correlation in the multivariate case) to fully take into account the fat left and right tails of the returns distribution. This allows for an adequate modelling of large returns defined on long and short trading positions. The performances of the univariate models are assessed on daily data for three international stock indexes and three US stocks of the Dow Jones index. In a second application, we consider a portfolio of three US stocks and model its long and short VaR using a multivariate skewed Student density.

The application of deep learning approaches to finance has received a great deal of attention from both investors and researchers. This study presents a novel deep learning framework where wavelet transforms (WT), stacked autoencoders (SAEs) and long-short term memory (LSTM) are combined for stock price forecasting. The SAEs for hierarchically extracted deep features is introduced into stock price forecasting for the first time. The deep learning framework comprises three stages. First, the stock price time series is decomposed by WT to eliminate noise. Second, SAEs is applied to generate deep high-level features for predicting the stock price. Third, high-level denoising features are fed into LSTM to forecast the next day’s closing price. Six market indices and their corresponding index futures are chosen to examine the performance of the proposed model. Results show that the proposed model outperforms other similar models in both predictive accuracy and profitability performance.

This analysis presents a rigorous exploration of financial data, incorporating a diverse range of statistical features. By providing a robust foundation, it facilitates advanced research and innovative modeling techniques within the field of finance.

Short/Long Term Stocks: LON:PMG Stock Forecast

Financial data:

• Historical daily stock prices (open, high, low, close, volume)

• Fundamental data (e.g., market capitalization, price to earnings P/E ratio, dividend yield, earnings per share EPS, price to earnings growth, debt-to-equity ratio, price-to-book ratio, current ratio, free cash flow, projected earnings growth, return on equity, dividend payout ratio, price to sales ratio, credit rating)

• Technical indicators (e.g., moving averages, RSI, MACD, average directional index, aroon oscillator, stochastic oscillator, on-balance volume, accumulation/distribution A/D line, parabolic SAR indicator, bollinger bands indicators, fibonacci, williams percent range, commodity channel index)

Machine learning features:

• Feature engineering based on financial data and technical indicators

• Sentiment analysis data from social media and news articles

• Macroeconomic data (e.g., GDP, unemployment rate, interest rates, consumer spending, building permits, consumer confidence, inflation, producer price index, money supply, home sales, retail sales, bond yields)

Potential Applications:

• Stock price prediction

• Portfolio optimization

• Algorithmic trading

• Market sentiment analysis

• Risk management

Use Cases:

• Researchers investigating the effectiveness of machine learning in stock market prediction

• Analysts developing quantitative trading Buy/Sell strategies

• Individuals interested in building their own stock market prediction models

• Students learning about machine learning and financial applications

Additional Notes:

• The dataset may include different levels of granularity (e.g., daily, hourly)

• Data cleaning and preprocessing are essential before model training

• Regular updates are recommended to maintain the accuracy and relevance of the data

This analysis presents a rigorous exploration of financial data, incorporating a diverse range of statistical features. By providing a robust foundation, it facilitates advanced research and innovative modeling techniques within the field of finance.

Dow Jones Industrial Average Index assigned short-term B1 & long-term Ba1 forecasted stock rating.

Financial data:

• Historical daily stock prices (open, high, low, close, volume)

• Fundamental data (e.g., market capitalization, price to earnings P/E ratio, dividend yield, earnings per share EPS, price to earnings growth, debt-to-equity ratio, price-to-book ratio, current ratio, free cash flow, projected earnings growth, return on equity, dividend payout ratio, price to sales ratio, credit rating)

• Technical indicators (e.g., moving averages, RSI, MACD, average directional index, aroon oscillator, stochastic oscillator, on-balance volume, accumulation/distribution A/D line, parabolic SAR indicator, bollinger bands indicators, fibonacci, williams percent range, commodity channel index)

Machine learning features:

• Feature engineering based on financial data and technical indicators

• Sentiment analysis data from social media and news articles

• Macroeconomic data (e.g., GDP, unemployment rate, interest rates, consumer spending, building permits, consumer confidence, inflation, producer price index, money supply, home sales, retail sales, bond yields)

Potential Applications:

• Stock price prediction

• Portfolio optimization

• Algorithmic trading

• Market sentiment analysis

• Risk management

Use Cases:

• Researchers investigating the effectiveness of machine learning in stock market prediction

• Analysts developing quantitative trading Buy/Sell strategies

• Individuals interested in building their own stock market prediction models

• Students learning about machine learning and financial applications

Additional Notes:

• The dataset may include different levels of granularity (e.g., daily, hourly)

• Data cleaning and preprocessing are essential before model training

• Regular updates are recommended to maintain the accuracy and relevance of the data

An index that can be used to gauge broad financial conditions and assess how these conditions are related to future economic growth. The index is broadly consistent with how the FRB/US model generally relates key financial variables to economic activity. The index aggregates changes in seven financial variables: the federal funds rate, the 10-year Treasury yield, the 30-year fixed mortgage rate, the triple-B corporate bond yield, the Dow Jones total stock market index, the Zillow house price index, and the nominal broad dollar index using weights implied by the FRB/US model and other models in use at the Federal Reserve Board. These models relate households' spending and businesses' investment decisions to changes in short- and long-term interest rates, house and equity prices, and the exchange value of the dollar, among other factors. These financial variables are weighted using impulse response coefficients (dynamic multipliers) that quantify the cumulative effects of unanticipated permanent changes in each financial variable on real gross domestic product (GDP) growth over the subsequent year. The resulting index is named Financial Conditions Impulse on Growth (FCI-G). One appealing feature of the FCI-G is that its movements can be used to measure whether financial conditions have tightened or loosened, to summarize how changes in financial conditions are associated with real GDP growth over the following year, or both.

This analysis presents a rigorous exploration of financial data, incorporating a diverse range of statistical features. By providing a robust foundation, it facilitates advanced research and innovative modeling techniques within the field of finance.

Short/Long Term Stocks: Karachi 100 Index Stock Forecast

Financial data:

• Historical daily stock prices (open, high, low, close, volume)

• Fundamental data (e.g., market capitalization, price to earnings P/E ratio, dividend yield, earnings per share EPS, price to earnings growth, debt-to-equity ratio, price-to-book ratio, current ratio, free cash flow, projected earnings growth, return on equity, dividend payout ratio, price to sales ratio, credit rating)

• Technical indicators (e.g., moving averages, RSI, MACD, average directional index, aroon oscillator, stochastic oscillator, on-balance volume, accumulation/distribution A/D line, parabolic SAR indicator, bollinger bands indicators, fibonacci, williams percent range, commodity channel index)

Machine learning features:

• Feature engineering based on financial data and technical indicators

• Sentiment analysis data from social media and news articles

• Macroeconomic data (e.g., GDP, unemployment rate, interest rates, consumer spending, building permits, consumer confidence, inflation, producer price index, money supply, home sales, retail sales, bond yields)

Potential Applications:

• Stock price prediction

• Portfolio optimization

• Algorithmic trading

• Market sentiment analysis

• Risk management

Use Cases:

• Researchers investigating the effectiveness of machine learning in stock market prediction

• Analysts developing quantitative trading Buy/Sell strategies

• Individuals interested in building their own stock market prediction models

• Students learning about machine learning and financial applications

Additional Notes:

• The dataset may include different levels of granularity (e.g., daily, hourly)

• Data cleaning and preprocessing are essential before model training

• Regular updates are recommended to maintain the accuracy and relevance of the data

This package contains the datasets and source codes used in the PhD thesis entitled Predicting the Brazilian stock market using sentiment analysis, technical indicators and stock prices. The following files are included: File Labeled.zip - financial news labeled in two classes (Positive and Negative), organized to train Sentiment Analysis models. Part of these news were initially presented in [1]. Besides the news in this file, in the related PhD thesis the training dataset was complemented with the labeled news presented in [2]. File Unlabeled.zip - general unlabeled financial news collected during the period 2010-2020 from the following online sources: G1, Folha de São Paulo and Estadão. This file contains news from the Bovespa index and from the following companies: Banco do Brasil, Itau, Gerdau and Ambev. File Stocks.zip - stock prices from the companies Banco do Brasil, Itau, Gerdau, Ambev, and the Bovespa index. The considered period ranges from 2010 to 2020. File Models.zip - contains the source codes of the models used in the PhD thesis (i.e., Multilayer Perceptron, Long Short-Term Memory, Bidirectional Long Short-Term Memory, Convolutional Neural Network, and Support Vector Machines). File Utils.zip - contains the source codes of the preprocessing step designed for the methodology of this work (i.e., load data and generate the word embeddings), alongside with stocks manipulation, and investment evaluation. [1] Carosia, A. E. D. O., Januário, B. A., da Silva, A. E. A., & Coelho, G. P. (2021). Sentiment Analysis Applied to News from the Brazilian Stock Market. IEEE Latin America Transactions, 100. DOI: 10.1109/TLA.2022.9667151 [2] MARTINS, R. F.; PEREIRA, A.; BENEVENUTO, F. An approach to sentiment analysis of web applications in portuguese. Proceedings of the 21st Brazilian Symposium on Multimedia and the Web, ACM, p. 105–112, 2015. DOI: 10.1145/2820426.2820446

Description of the input variables.

description: Shows index traders in selected agricultural markets. These traders are drawn from the noncommercial and commercial categories. The noncommercial category includes positions of managed funds, pension funds, and other investors that are generally seeking exposure to a broad index of commodity prices as an asset class in an unleveraged and passively-managed manner. The commercial category includes positions for entities whose trading predominantly reflects hedging of over-the-counter transactions involving commodity indices, for example, a swap dealer holding long futures positions to hedge a short commodity index exposure opposite institutional traders, such as pension funds.; abstract: Shows index traders in selected agricultural markets. These traders are drawn from the noncommercial and commercial categories. The noncommercial category includes positions of managed funds, pension funds, and other investors that are generally seeking exposure to a broad index of commodity prices as an asset class in an unleveraged and passively-managed manner. The commercial category includes positions for entities whose trading predominantly reflects hedging of over-the-counter transactions involving commodity indices, for example, a swap dealer holding long futures positions to hedge a short commodity index exposure opposite institutional traders, such as pension funds.

The Blended Index (BI) is a model which employs multiple potential indicators of drought and excess moisture, such as the Palmer drought index, rolling precipitation amounts and soil moisture, and combines them into a weighted, normalized value between 0 and 100. The inputs and weights used in this model are subject to change periodically as it is optimized to best represent extent, duration and severity of impactful weather conditions. The blended index is deployed as two variations; short term (st) focusing on 1 to 3 months, and long term (lt) focusing on 6 months to 5 years.

Predictive accuracy in developing markets.

Case Shiller Home Price Index in the United States increased to 338.39 points in March from 335.08 points in February of 2025. This dataset provides the latest reported value for - United States S&P Case-Shiller Home Price Index - plus previous releases, historical high and low, short-term forecast and long-term prediction, economic calendar, survey consensus and news.

This analysis presents a rigorous exploration of financial data, incorporating a diverse range of statistical features. By providing a robust foundation, it facilitates advanced research and innovative modeling techniques within the field of finance.

Short/Long Term Stocks: SPB Stock Forecast

Financial data:

• Historical daily stock prices (open, high, low, close, volume)

• Fundamental data (e.g., market capitalization, price to earnings P/E ratio, dividend yield, earnings per share EPS, price to earnings growth, debt-to-equity ratio, price-to-book ratio, current ratio, free cash flow, projected earnings growth, return on equity, dividend payout ratio, price to sales ratio, credit rating)

• Technical indicators (e.g., moving averages, RSI, MACD, average directional index, aroon oscillator, stochastic oscillator, on-balance volume, accumulation/distribution A/D line, parabolic SAR indicator, bollinger bands indicators, fibonacci, williams percent range, commodity channel index)

Machine learning features:

• Feature engineering based on financial data and technical indicators

• Sentiment analysis data from social media and news articles

• Macroeconomic data (e.g., GDP, unemployment rate, interest rates, consumer spending, building permits, consumer confidence, inflation, producer price index, money supply, home sales, retail sales, bond yields)

Potential Applications:

• Stock price prediction

• Portfolio optimization

• Algorithmic trading

• Market sentiment analysis

• Risk management

Use Cases:

• Researchers investigating the effectiveness of machine learning in stock market prediction

• Analysts developing quantitative trading Buy/Sell strategies

• Individuals interested in building their own stock market prediction models

• Students learning about machine learning and financial applications

Additional Notes:

• The dataset may include different levels of granularity (e.g., daily, hourly)

• Data cleaning and preprocessing are essential before model training

• Regular updates are recommended to maintain the accuracy and relevance of the data

Economic Optimism Index in the United States increased to 49.20 points in June from 47.90 points in May of 2025. This dataset provides the latest reported value for - United States IBD/TIPP Economic Optimism Index - plus previous releases, historical high and low, short-term forecast and long-term prediction, economic calendar, survey consensus and news.

LSTM (Long Short-Term Memory Network) is currently extensively utilized for forecasting financial time series, primarily due to its distinct advantages in separating the long-term from the short-term memory information within a sequence. However, the experimental results presented in this paper indicate that LSTM may struggle to clearly differentiate between these two types of information. To overcome this limitation, we propose the ARMA-RNN-LSTM Hybrid Model, aimed at enhancing the separation between the long-term and short-term memory information on top of LSTM framework. The experiment in this paper is inspired by an observation: when LSTMs and RNNs are respectively used to forecast the same time series that contains only short-term memory information, LSTMs exhibit significantly lower forecasting accuracy than RNNs, and we attributed this to LSTMs potentially misclassifying some short-term memory information as long-term during forecasting process. Further, we speculate that this confusion might also arise when LSTMs are used to forecast the time series containing both the long-term and short-term memory information. To verify the aforementioned hypothesis and improve the forecasting accuracy for financial time series, this paper combines RNNs with LSTMs, proposing a method of ARMA-RNN-LSTM Hybrid Modelling, and conducts an experiment with stock index prices. Eventually, the experiment results show that the ARMA-RNN-LSTM Hybrid Model outperforms standalone RNNs and LSTMs in forecasting stock index series containing both long-term and short-term memory information, confirming that the ARMA-RNN-LSTM Hybrid Model has effectively enhanced the separation between the long-term and short-term memory information within sequence. This hybrid modelling approach has innovatively addressed the issue of the confusion between the long-term and the short-term memory information in a sequence during LSTM’s forecasting process, improving the accuracy of forecasting financial time series, and demonstrates that neural network’s forecasting errors is a area worth to explore in the future.