In March 2024 Bitcoin BTC reached a new all-time high with prices exceeding 73000 USD marking a milestone for the cryptocurrency market This surge was due to the approval of Bitcoin exchange-traded funds ETFs in the United States allowing investors to access Bitcoin without directly holding it This development increased Bitcoin’s credibility and brought fresh demand from institutional investors echoing previous price surges in 2021 when Tesla announced its 15 billion investment in Bitcoin and Coinbase was listed on the Nasdaq By the end of 2022 Bitcoin prices dropped sharply to 15000 USD following the collapse of cryptocurrency exchange FTX and its bankruptcy which caused a loss of confidence in the market By August 2024 Bitcoin rebounded to approximately 64178 USD but remained volatile due to inflation and interest rate hikes Unlike fiat currency like the US dollar Bitcoin’s supply is finite with 21 million coins as its maximum supply By September 2024 over 92 percent of Bitcoin had been mined Bitcoin’s value is tied to its scarcity and its mining process is regulated through halving events which cut the reward for mining every four years making it harder and more energy-intensive to mine The next halving event in 2024 will reduce the reward to 3125 BTC from its current 625 BTC The final Bitcoin is expected to be mined around 2140 The energy required to mine Bitcoin has led to criticisms about its environmental impact with estimates in 2021 suggesting that one Bitcoin transaction used as much energy as Argentina Bitcoin’s future price is difficult to predict due to the influence of large holders known as whales who own about 92 percent of all Bitcoin These whales can cause dramatic market swings by making large trades and many retail investors still dominate the market While institutional interest has grown it remains a small fraction compared to retail Bitcoin is vulnerable to external factors like regulatory changes and economic crises leading some to believe it is in a speculative bubble However others argue that Bitcoin is still in its early stages of adoption and will grow further as more institutions and governments recognize its potential as a hedge against inflation and a store of value 2024 has also seen the rise of Bitcoin Layer 2 technologies like the Lightning Network which improve scalability by enabling faster and cheaper transactions These innovations are crucial for Bitcoin’s wider adoption especially for day-to-day use and cross-border remittances At the same time central bank digital currencies CBDCs are gaining traction as several governments including China and the European Union have accelerated the development of their own state-controlled digital currencies while Bitcoin remains decentralized offering financial sovereignty for those who prefer independence from government control The rise of CBDCs is expected to increase interest in Bitcoin as a hedge against these centralized currencies Bitcoin’s journey in 2024 highlights its growing institutional acceptance alongside its inherent market volatility While the approval of Bitcoin ETFs has significantly boosted interest the market remains sensitive to events like exchange collapses and regulatory decisions With the limited supply of Bitcoin and improvements in its transaction efficiency it is expected to remain a key player in the financial world for years to come Whether Bitcoin is currently in a speculative bubble or on a sustainable path to greater adoption will ultimately be revealed over time.

Bitcoin's blockchain size was close to reaching 5450 gigabytes in 2024, as the database saw exponential growth by nearly one gigabyte every few days. The Bitcoin blockchain contains a continuously growing and tamper-evident list of all Bitcoin transactions and records since its initial release in January 2009. Bitcoin has a set limit of 21 million coins, the last of which will be mined around 2140, according to a forecast made in 2017. Bitcoin mining: A somewhat uncharted world Despite interest in the topic, there are few accurate figures on how big Bitcoin mining is on a country-by-country basis. Bitcoin's design philosophy is at the heart of this. Created out of protest against governments and central banks, Bitcoin's blockchain effectively hides both the country of origin and the destination country within a (mining) transaction. Research involving IP addresses placed the United States as the world's most Bitcoin mining country in 2022 - but the source admits IP addresses can easily be manipulated using VPN. Note that mining figures are different from figures on Bitcoin trading: Africa and Latin America were more interested in buying and selling BTC than some of the world's developed economies. Bitcoin developments Bitcoin's trade volume slowed in the second quarter of 2023, after hitting a noticeable growth at the beginning of the year. The coin outperformed most of the market. Some attribute this to the announcement in June 203 that BlackRock filed for a Bitcoin ETF. This iShares Bitcoin Trust was to use Coinbase Custody as its custodian. Regulators in the United States had not yet approved any applications for spot ETFs on Bitcoin.

Context:

This dataset is an extra updating dataset for the G-Research Crypto Forecasting competition.

Introduction

This is a daily updated dataset, automaticlly collecting market data for G-Research crypto forecasting competition. The data is of the 1-minute resolution, collected for all competition assets and both retrieval and uploading are fully automated. see discussion topic.

The Data

For every asset in the competition, the following fields from Binance's official API endpoint for historical candlestick data are collected, saved, and processed.

1. **timestamp** - A timestamp for the minute covered by the row.
2. **Asset_ID** - An ID code for the cryptoasset.
3. **Count** - The number of trades that took place this minute.
4. **Open** - The USD price at the beginning of the minute.
5. **High** - The highest USD price during the minute.
6. **Low** - The lowest USD price during the minute.
7. **Close** - The USD price at the end of the minute.
8. **Volume** - The number of cryptoasset u units traded during the minute.
9. **VWAP** - The volume-weighted average price for the minute.
10. **Target** - 15 minute residualized returns. See the 'Prediction and Evaluation section of this notebook for details of how the target is calculated.
11. **Weight** - Weight, defined by the competition hosts [here](https://www.kaggle.com/cstein06/tutorial-to-the-g-research-crypto-competition)
12. **Asset_Name** - Human readable Asset name.

Indexing

The dataframe is indexed by timestamp and sorted from oldest to newest. The first row starts at the first timestamp available on the exchange, which is July 2017 for the longest-running pairs.

Usage Example

The following is a collection of simple starter notebooks for Kaggle's Crypto Comp showing PurgedTimeSeries in use with the collected dataset. Purged TimesSeries is explained here. There are many configuration variables below to allow you to experiment. Use either GPU or TPU. You can control which years are loaded, which neural networks are used, and whether to use feature engineering. You can experiment with different data preprocessing, model architecture, loss, optimizers, and learning rate schedules. The extra datasets contain the full history of the assets in the same format as the competition, so you can input that into your model too.

Baseline Example Notebooks:

• Neural Network Starter
• LightGBM Starter
• Catboost Starter
• XGBoost Starter
• TabNet Starter
• Reinforcement Learning (PPO) Starter

These notebooks follow the ideas presented in my "Initial Thoughts" here. Some code sections have been reused from Chris' great (great) notebook series on SIIM ISIC melanoma detection competition here

Loose-ends:

This is a work in progress and will be updated constantly throughout the competition. At the moment, there are some known issues that still needed to be addressed:

• VWAP: - At the moment VWAP calculation formula is still unclear. Currently the dataset uses an approximation calculated from the Open, High, Low, Close, Volume candlesticks. [Waiting for competition hosts input]
• Target Labeling: There exist some mismatches to the original target provided by the hosts at some time intervals. On all the others - it is the same. The labeling code can be seen here. [Waiting for competition hosts] input]
• Filtering: No filtration of 0 volume data is taken place.

Example Visualisations

Opening price with an added indicator (MA50): https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2234678%2Fb8664e6f26dc84e9a40d5a3d915c9640%2Fdownload.png?generation=1582053879538546&alt=media" alt="">

Volume and number of trades: https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2234678%2Fcd04ed586b08c1576a7b67d163ad9889%2Fdownload-1.png?generation=1582053899082078&alt=media" alt="">

License

This data is being collected automatically from the crypto exchange Binance.

This is a submission for Challenge #24 by Desights User

Click here for Challenge Details Note: This submission is in REVIEW state and is only accessible by Challenge Reviewers. So you might get errors when you try to download this asset directly from Ocean Market.

Submission Description

The cryptocurrency is not just a new form of value store and exchange, it is a revolution of its own. Beginning with its use to provide peer-to-peer payment network (or digital money) like Bitcoin, today’s cryptocurrency, or crypto for short, have evolved way beyond its humble start. Underlying the crypto world, there lies amazing technology called Blockchain. In simple term, blockchain is a decentralized and shared digital ledger that records transactions transparently and immutably across nodes in the network. Today’s Crypto community has slowly turned into industry of its own introducing a whole spectrum of enigmatic pattern, trends, and economic framework. In this report we will explore the trend, correlations, and dynamics related to 20 selected Crypto projects to derived insights and build models that predict the future of crypto. Key Findings: Our exploratory data analysis (EDA) underlines the span and general pattern of the Google Trend and Price related data. The data being analyzed span from the earliest entry on 2014-09-17 up to the latest on 2024-04-07. Time series decomposition was performed to extract trend, seasonal cycle, and residuals that made up the Google Interest Trend data. Analysis on the time-series decomposition help us distinguish cluster (a) with projects on the rise such as Solana, SingularityNet, Fetch.ai, and Ocean Protocol; and cluster (b) containing old project such as Dogecoin, Litecoin Filecoin, Tezos that are facing stagnant/downfall trend. Based on the Google Trends’s Correlation across projects we characterize Highly correlated projects cluster with correlation of about >0.8, and up to 0.92 with Bitcoin-Ethereum-Chainlink-Litecoin-Monero as the prominent group members. By introducing additional Google Trend data to understand Crypto Narrative, we worked toward building interpretable Event/Entity driving the market sentiment to explain our decomposed Time-series data. Based on Lag Characteristics in Correlation of Google Trend and Price/Trade Volume we highlight the tendency for the correlation to accumulate at longer lag time. Using NeuralProphet Framework we build forecasting models for Google Trend and Token Price for all 20 projects investigated here. We deployed these models to predict Trend and Price for all 20 projects for the following 52 weeks (up until April 2025). The developed models performed extraordinarily well with the R^2 value for most fall between the range of 0.75-0.88, while the highest goes up to 0.919. We highlight the correlation between Bitcoin, Ethereum, Ocean, with the rest of other projects. Ocean and Bitcoin, also Ethereum and Solana are the most correlated, both with correlation value of 0.89. The Kucoin’s KCS token is the least correlated with both Ocean and Bitcoin (0.31), while with Ethereum, Filecoin have the least correlation (0.41).

Conclusion This investigative study presents a thorough data analysis and exploration of correlations, time-lag characteristics, and time-series decomposition concerning Google Trends and token prices for 20 selected crypto/blockchain projects. By decomposing the time-series data, we have identified several clusters of crypto projects that is moving up in popularity such as Fetch.ai, SingularityNet, Solana, Ocean and some others that are stuck or in downfall trend, such as Dogecoin and Litecoin. Our analysis also includes a detailed exploration of various factors that contribute to understanding the data better, such as the incorporation of event-driven trends that explain outlier spikes in the residual data from our decomposed time-series.

In addition to our in-depth analysis, we build strong mini-library of forecasting models for predicting the Google Trend as well as price for the upcoming year with R^2 score that goes as high as 0.88 for most cases. Moreover, in order to demonstrate the utility of our exploratory data analysis tools and pipeline in full we also include all the results and analysis output produced in this work.

Looking ahead, we plan to expand our developed forecasting models and the presented data into a "CryptoForecast MiniApp." This application, based on the Streamlit package, will be hosted on a decentralized cloud (Akash) and connected to the Ocean marketplace and Predictoor, enhancing accessibility and utility for users interested in real-time data for Google Trends and Crypto Token Price forecasts.

The dataset contains all historical order book snapshots and blockchain network information used to generate the results for the paper "Building Trust takes Time". A blockchain replaces central counterparties with time-consuming consensus protocols to record the transfer of ownership. This settlement latency slows cross-exchange trading, exposing arbitrageurs to price risk. Off-chain settlement, instead, exposes arbitrageurs to costly default risk. We show with Bitcoin network and order book data that cross-exchange price differences coincide with periods of high settlement latency, asset flows chase arbitrage opportunities, and price differences across exchanges with low default risk are smaller. Blockchain-based trading thus faces a dilemma: reliable consensus protocols require time-consuming settlement latency, leading to arbitrage limits. Circumventing such arbitrage costs is possible only by reinstalling trusted intermediation, which mitigates default risk., This dataset provides three types of Bitcoin-related information:

Centralized crypto-exchange (CEX) characteristics which have been collected manually. High-frequency order book information. The data has been retrieved by regularly fetching order book information from major centralized crypto exchanges (CEX) in minute-level intervals from 2018 - 2019. We provide the entire order book history across the exchanges with this dataset. Corresponding information on the state of the Bitcoin blockchain, for instance, the number of outstanding transactions at every point in time. The data has been used to analyze arbitrage activity across CEXes in relation to the time it takes for validators to execute cross-CEX transactions. Code to replicate the data processing parts is publicly available on Github: www.github.com/voigtstefan/building-trust-takes-time

, , # Data to Replicate the paper Building Trust Takes Time: Limits to Arbitrage for Blockchain-Based Assets

We provide all datasets required to replicate the paper "Building Trust takes Time: Limits to Arbitrage for Blockchain-based Assets". A description of the data sources and preprocessing steps is provided in the paper. All code to generate the results is available on .

Description of the data and file structure

In principle, we offer three different types of crypto-currency-related data:

1. Centralized crypto-exchange (CEX) characteristics, which have been collected manually.
2. High-frequency order book information. The data has been retrieved by regularly fetching order book information from major centralized crypto exchanges (CEX) in minute-level intervals from 2018 - 2019. We provide the entire order book history across the exchanges with this dataset.
3. Corresponding information on the state of the Bitcoin blockchain, for instance, the number of outstanding transacti...

Interest in Bitcoin and cryptocurrencies in 2020 was seemingly higher in Africa and Latin America than some of the world's developed economies. This shows after analyzing Bitcoin trading volume against domestic currencies used for the transaction of the digital coin. In 2020, roughly 420 million U.S. dollars worth of Russian rubles were used to buy Bitcoin on an exchange, against 400 million U.S. dollars worth of Nigerian naira. The source assumes the currencies are mainly used by the domestic population - e.g. transactions made with British pounds are likely done by UK residents -, and makes the same assumption for the United States, despite the international appeal of the U.S. dollar on foreign exchange markets.

Africa and Latin America lead the way

Although the source does not mention all countries in Africa and Latin America, the few entries these regions do have in the list stand out. Bitcoin trading volume in Nigeria, for instance, was twice as high as that of the eurozone in 2020. Colombia's market size was twice that of Canada. Whether this interest is for actual payment use on a day-to-day basis or as a tool for investment is not really clear. Data from Statista's Global Consumer Survey on payment methods in Egypt reveals that eight percent of Egyptians either owned or used Bitcoin, but does not specify the exact use or purpose of the cryptocurrency.

Bitcoin: the "Renaissance"

Believed by some to fade into obscurity after hitting the news in 2017 and price declines that followed afterwards, the world's most well-known cryptocurrency witnessed a "rebirth" at the end of 2020: Within five days in January 2021, the price of Bitcoin soared from 30,000 U.S. dollars to 40,000 U.S. dollars. Bitcoin's market cap - calculated by multiplying the total number of Bitcoins in circulation against its price - grew as well, more than doubling in early January 2021 against November 2020

Since 2022, hacks were the main reason behind cryptocurrency losses. 2021 was the exception during this time period, when fraud led to the highest volume of funds lost. This is according to data from web3 crowdsourced security platform Immunefi. While the source does not clearly state how it estimated its figures, it claims that over 60 percent of all cryptocurrency losses between 2021 and 2024 can be attributed to hacks.