In March 2024 Bitcoin BTC reached a new all-time high with prices exceeding 73000 USD marking a milestone for the cryptocurrency market This surge was due to the approval of Bitcoin exchange-traded funds ETFs in the United States allowing investors to access Bitcoin without directly holding it This development increased Bitcoin’s credibility and brought fresh demand from institutional investors echoing previous price surges in 2021 when Tesla announced its 15 billion investment in Bitcoin and Coinbase was listed on the Nasdaq By the end of 2022 Bitcoin prices dropped sharply to 15000 USD following the collapse of cryptocurrency exchange FTX and its bankruptcy which caused a loss of confidence in the market By August 2024 Bitcoin rebounded to approximately 64178 USD but remained volatile due to inflation and interest rate hikes Unlike fiat currency like the US dollar Bitcoin’s supply is finite with 21 million coins as its maximum supply By September 2024 over 92 percent of Bitcoin had been mined Bitcoin’s value is tied to its scarcity and its mining process is regulated through halving events which cut the reward for mining every four years making it harder and more energy-intensive to mine The next halving event in 2024 will reduce the reward to 3125 BTC from its current 625 BTC The final Bitcoin is expected to be mined around 2140 The energy required to mine Bitcoin has led to criticisms about its environmental impact with estimates in 2021 suggesting that one Bitcoin transaction used as much energy as Argentina Bitcoin’s future price is difficult to predict due to the influence of large holders known as whales who own about 92 percent of all Bitcoin These whales can cause dramatic market swings by making large trades and many retail investors still dominate the market While institutional interest has grown it remains a small fraction compared to retail Bitcoin is vulnerable to external factors like regulatory changes and economic crises leading some to believe it is in a speculative bubble However others argue that Bitcoin is still in its early stages of adoption and will grow further as more institutions and governments recognize its potential as a hedge against inflation and a store of value 2024 has also seen the rise of Bitcoin Layer 2 technologies like the Lightning Network which improve scalability by enabling faster and cheaper transactions These innovations are crucial for Bitcoin’s wider adoption especially for day-to-day use and cross-border remittances At the same time central bank digital currencies CBDCs are gaining traction as several governments including China and the European Union have accelerated the development of their own state-controlled digital currencies while Bitcoin remains decentralized offering financial sovereignty for those who prefer independence from government control The rise of CBDCs is expected to increase interest in Bitcoin as a hedge against these centralized currencies Bitcoin’s journey in 2024 highlights its growing institutional acceptance alongside its inherent market volatility While the approval of Bitcoin ETFs has significantly boosted interest the market remains sensitive to events like exchange collapses and regulatory decisions With the limited supply of Bitcoin and improvements in its transaction efficiency it is expected to remain a key player in the financial world for years to come Whether Bitcoin is currently in a speculative bubble or on a sustainable path to greater adoption will ultimately be revealed over time.

Context

Bitcoin is the longest running and most well known cryptocurrency, first released as open source in 2009 by the anonymous Satoshi Nakamoto. Bitcoin serves as a decentralized medium of digital exchange, with transactions verified and recorded in a public distributed ledger (the blockchain) without the need for a trusted record keeping authority or central intermediary. Transaction blocks contain a SHA-256 cryptographic hash of previous transaction blocks, and are thus "chained" together, serving as an immutable record of all transactions that have ever occurred. As with any currency/commodity on the market, bitcoin trading and financial instruments soon followed public adoption of bitcoin and continue to grow. Included here is historical bitcoin market data for select bitcoin exchanges where trading takes place. Happy (data) mining!

CSV files for select bitcoin exchanges for the time period of September 2011 to June 2024, with updates of OHLC (Open, High, Low, Close), Volume in BTC and indicated currency, and weighted bitcoin price. Timestamps are in Unix time. Timestamps without any trades or activity have their data fields filled with NaNs. If a timestamp is missing, or if there are jumps, this may be because the exchange (or its API) was down, the exchange (or its API) did not exist, or some other unforeseen technical error in data reporting or gathering. All effort has been made to deduplicate entries and verify the contents are correct and complete to the best of my ability, but obviously trust at your own risk.

Acknowledgements and Inspiration

Bitcoin charts for the data. The various exchange APIs, for making it difficult or unintuitive enough to get OHLC and volume data that I set out on this data scraping project. Satoshi Nakamoto and the novel core concept of the blockchain, as well as its first execution via the bitcoin protocol. I'd also like to thank viewers like you! Can't wait to see what code or insights you all have to share.

Context

Bitcoin(₿) is a cryptocurrency invented in 2008 by an unknown person or group of people using the name Satoshi Nakamoto. The currency began use in 2009 when its implementation was released as open-source software.

Bitcoin is a decentralized digital currency, without a central bank or single administrator, that can be sent from user to user on the peer-to-peer bitcoin network without the need for intermediaries. Transactions are verified by network nodes through cryptography and recorded in a public distributed ledger called a blockchain. Bitcoins are created as a reward for a process known as mining. They can be exchanged for other currencies, products, and services.

On 30 November 2020, bitcoin hit a new all-time high of $19,860 topping the previous high from December 2017. On 19 January 2021 Elon Musk placed #Bitcoin in his Twitter profile tweeting “In retrospect, it was inevitable”, which caused the price to briefly rise about $5000 in an hour to $37,299.

Content

The tweets have #Bitcoin and #btc hashtag.. Collection star started on 6/2/2021, with an initial 100,000 tweets, and will continue on a daily basis.

Information regarding the data

The data totally consists of 1 lakh+ records with 13 columns. The description of the features is given below | No |Columns | Descriptions | | -- | -- | -- | | 1 | user_name | The name of the user, as they’ve defined it. | | 2 | user_location | The user-defined location for this account’s profile. | | 3 | user_description | The user-defined UTF-8 string describing their account. | | 4 | user_created | Time and date, when the account was created. | | 5 | user_followers | The number of followers an account currently has. | | 6 | user_friends | The number of friends an account currently has. | | 7 | user_favourites | The number of favorites an account currently has | | 8 | user_verified | When true, indicates that the user has a verified account | | 9 | date | UTC time and date when the Tweet was created | | 10 | text | The actual UTF-8 text of the Tweet | | 11 | hashtags | All the other hashtags posted in the tweet along with #Bitcoin & #btc | | 12 | source | Utility used to post the Tweet, Tweets from the Twitter website have a source value - web | | 13 | is_retweet | Indicates whether this Tweet has been Retweeted by the authenticating user. |

Inspiration

The tweets were extracted using tweepy, Refer to this notebook for the complete extraction process https://www.kaggle.com/kaushiksuresh147/twitter-data-extraction-for-ipl2020

You can use this data to dive into the subjects that use this hashtag, look to the geographical distribution, evaluate sentiments, looks at trends.

Bitcoin's blockchain size was close to reaching 652.93 gigabytes in June 2025, as the database saw exponential growth by nearly one gigabyte every few days. The Bitcoin blockchain contains a continuously growing and tamper-evident list of all Bitcoin transactions and records since its initial release in January 2009. Bitcoin has a set limit of 21 million coins, the last of which will be mined around 2140, according to a forecast made in 2017. Bitcoin mining: A somewhat uncharted world Despite interest in the topic, there are few accurate figures on how big Bitcoin mining is on a country-by-country basis. Bitcoin's design philosophy is at the heart of this. Created out of protest against governments and central banks, Bitcoin's blockchain effectively hides both the country of origin and the destination country within a (mining) transaction. Research involving IP addresses placed the United States as the world's most Bitcoin mining country in 2022 - but the source admits IP addresses can easily be manipulated using VPN. Note that mining figures are different from figures on Bitcoin trading: Africa and Latin America were more interested in buying and selling BTC than some of the world's developed economies. Bitcoin developments Bitcoin's trade volume slowed in the second quarter of 2023, after hitting a noticeable growth at the beginning of the year. The coin outperformed most of the market. Some attribute this to the announcement in June 2023 that BlackRock filed for a Bitcoin ETF. This iShares Bitcoin Trust was to use Coinbase Custody as its custodian. Regulators in the United States had not yet approved any applications for spot ETFs on Bitcoin.

This dataset contains historical Bitcoin (BTC/USDT) price data from Binance exchange with the following specifications:

Timezone Information: - All timestamps are in UTC (Coordinated Universal Time) - Open time format: YYYY-MM-DD HH:MM:SS.ffffff UTC - Close time format: YYYY-MM-DD HH:MM:SS.ffffff UTC

Daily Timeframe Specific: - Open time: Always shows 00:00:00.000000 UTC (start of day) - Close time: Always shows 23:59:59.999000 UTC (end of day)

Timeframes Available: - 15-minute intervals (15m) - 1-hour intervals (1h) - 4-hour intervals (4h) - 1-day intervals (1d)

Data Columns: - Open time: Opening timestamp in UTC - Open: Opening price - High: Highest price during period - Low: Lowest price during period - Close: Closing price - Volume: Trading volume - Close time: Closing timestamp in UTC - Quote asset volume: Volume in quote asset (USDT) - Number of trades: Number of trades during period - Taker buy base asset volume: Volume of taker buy orders - Taker buy quote asset volume: Volume of taker buy orders in quote asset - Ignore: Unused field

Data is automatically updated and maintained through automated scripts.

Electron (Bitcoin) Ang pagsubaybay sa kasaysayan ng presyo ay nagbibigay-daan sa mga crypto investor na madaling masubaybayan ang performance ng kanilang pamumuhunan. Maginhawa mong masusubaybayan ang opening value, high, at close sa Electron (Bitcoin) sa paglipas ng panahon, pati na rin ang trade volume. Bukod pa rito, maaari mong agad na tingnan ang pang-araw-araw na pagbabago bilang isang porsyento, na ginagawang effortless na tukuyin ang mga araw na may significant fluctuations. Ayon sa aming data ng history ng presyo ng Electron (Bitcoin), tumaas ang halaga nito sa hindi pa naganap na peak sa 2024-03-23, na lumampas sa $0.04001 USD. Sa kabilang banda, ang pinakamababang punto sa trajectory ng presyo ni Electron (Bitcoin), na karaniwang tinutukoy bilang "Electron (Bitcoin) all-time low", ay naganap noong 2025-01-09. Kung ang isa ay bumili ng Electron (Bitcoin) sa panahong iyon, kasalukuyan silang masisiyahan sa isang kahanga-hangang kita na 339%. Sa pamamagitan ng disenyo, ang 930,277,117 Electron (Bitcoin) ay malilikha. Sa ngayon, ang circulating supply ng Electron (Bitcoin) ay tinatayang 0. Ang lahat ng mga presyong nakalista sa pahinang ito ay nakuha mula sa Bitget, galing sa isang reliable source. Napakahalagang umasa sa iisang pinagmulan upang suriin ang iyong mga investment, dahil maaaring mag-iba ang mga halaga sa iba't ibang nagbebenta. Kasama sa aming makasaysayang Electron (Bitcoin) dataset ng presyo ang data sa pagitan ng 1 minuto, 1 araw, 1 linggo, at 1 buwan (bukas/mataas/mababa/close/volume). Ang mga dataset na ito ay sumailalim sa mahigpit na pagsubok upang matiyak ang consistency, pagkakumpleto, at accurancy. Ang mga ito ay partikular na idinisenyo para sa trade simulation at mga layunin ng backtesting, madaling magagamit para sa libreng pag-download, at na-update sa real-time.

Bitcoin's transaction volume was at its highest in December 2023, when the network processed over ******* coins on the same day. Bitcoin generally has a higher transaction activity than other cryptocurrencies, except Ethereum. This cryptocurrency is often processed more than *********** times per day. Note that the transaction volume here refers to transactions registered within the Bitcoin blockchain. It should not be confused with Bitcoin's 24-hour trade volume, a metric associated with crypto exchanges. The more Bitcoin transactions, the more it is used in B2C payments? A Bitcoin transaction recorded in the blockchain can be any transaction, including B2C but also P2P. While it is possible to see in the blockchain which address sent Bitcoin to whom, details on who this person is and where they are from are typically missing. Bitcoin was designed to go against monetary authorities and prides itself on being anonymous. An important argument against Bitcoin replacing cash or cards in payments is that the cryptocurrency was not allowed for such a task: Bitcoin ranks among the slowest cryptocurrencies in terms of transaction speed. Are cryptocurrencies taking over payments? Cryptocurrency payments are set to grow at a CAGR of nearly ** percent between 2022 and 2029, although the market is relatively small. The forecast is according to a market estimate made in early 2023, based on various conditions and sources available at that time. Research across ** countries during the same time suggested that the market share of cryptocurrency in e-commerce transactions was "less than *** percent" in all surveyed countries, with predictions being this would not change in the future.

Analysis of ‘Crypto Fear and Greed Index’ provided by Analyst-2 (analyst-2.ai), based on source dataset retrieved from https://www.kaggle.com/adelsondias/crypto-fear-and-greed-index on 13 February 2022.

--- Dataset description provided by original source is as follows ---

Crypto Fear and Greed Index

Each day, the website https://alternative.me/crypto/fear-and-greed-index/ publishes this index based on analysis of emotions and sentiments from different sources crunched into one simple number: The Fear & Greed Index for Bitcoin and other large cryptocurrencies.

Why Measure Fear and Greed?

The crypto market behaviour is very emotional. People tend to get greedy when the market is rising which results in FOMO (Fear of missing out). Also, people often sell their coins in irrational reaction of seeing red numbers. With our Fear and Greed Index, we try to save you from your own emotional overreactions. There are two simple assumptions:

• Extreme fear can be a sign that investors are too worried. That could be a buying opportunity.
• When Investors are getting too greedy, that means the market is due for a correction.

Therefore, we analyze the current sentiment of the Bitcoin market and crunch the numbers into a simple meter from 0 to 100. Zero means "Extreme Fear", while 100 means "Extreme Greed". See below for further information on our data sources.

Data Sources

We are gathering data from the five following sources. Each data point is valued the same as the day before in order to visualize a meaningful progress in sentiment change of the crypto market.

First of all, the current index is for bitcoin only (we offer separate indices for large alt coins soon), because a big part of it is the volatility of the coin price.

But let’s list all the different factors we’re including in the current index:

Volatility (25 %)

We’re measuring the current volatility and max. drawdowns of bitcoin and compare it with the corresponding average values of the last 30 days and 90 days. We argue that an unusual rise in volatility is a sign of a fearful market.

Market Momentum/Volume (25%)

Also, we’re measuring the current volume and market momentum (again in comparison with the last 30/90 day average values) and put those two values together. Generally, when we see high buying volumes in a positive market on a daily basis, we conclude that the market acts overly greedy / too bullish.

Social Media (15%)

While our reddit sentiment analysis is still not in the live index (we’re still experimenting some market-related key words in the text processing algorithm), our twitter analysis is running. There, we gather and count posts on various hashtags for each coin (publicly, we show only those for Bitcoin) and check how fast and how many interactions they receive in certain time frames). A unusual high interaction rate results in a grown public interest in the coin and in our eyes, corresponds to a greedy market behaviour.

Surveys (15%) currently paused

Together with strawpoll.com (disclaimer: we own this site, too), quite a large public polling platform, we’re conducting weekly crypto polls and ask people how they see the market. Usually, we’re seeing 2,000 - 3,000 votes on each poll, so we do get a picture of the sentiment of a group of crypto investors. We don’t give those results too much attention, but it was quite useful in the beginning of our studies. You can see some recent results here.

Dominance (10%)

The dominance of a coin resembles the market cap share of the whole crypto market. Especially for Bitcoin, we think that a rise in Bitcoin dominance is caused by a fear of (and thus a reduction of) too speculative alt-coin investments, since Bitcoin is becoming more and more the safe haven of crypto. On the other side, when Bitcoin dominance shrinks, people are getting more greedy by investing in more risky alt-coins, dreaming of their chance in next big bull run. Anyhow, analyzing the dominance for a coin other than Bitcoin, you could argue the other way round, since more interest in an alt-coin may conclude a bullish/greedy behaviour for that specific coin.

Trends (10%)

We pull Google Trends data for various Bitcoin related search queries and crunch those numbers, especially the change of search volumes as well as recommended other currently popular searches. For example, if you check Google Trends for "Bitcoin", you can’t get much information from the search volume. But currently, you can see that there is currently a +1,550% rise of the query „bitcoin price manipulation“ in the box of related search queries (as of 05/29/2018). This is clearly a sign of fear in the market, and we use that for our index.

There's a story behind every dataset and here's your opportunity to share yours.

Copyright disclaimer

This dataset is produced and maintained by the administrators of https://alternative.me/crypto/fear-and-greed-index/.

This published version is an unofficial copy of their data, which can be also collected using their API (e.g., GET https://api.alternative.me/fng/?limit=10&format=csv&date_format=us).

--- Original source retains full ownership of the source dataset ---

Bitcoin is a cryptocurrency and a large-scale payment system, in which all the transactions are publicly accessible. This dataset was use as part of our study, in which we used the Bitcoin Transaction Network Dataset published in 2013 by Ivan Brugere. The dataset includes over 37.4 million transactions, from January 2009 to April 2013, between public-key “addresses,” from which we created a directed network with over 6.3 million vertices and 16.3 million links over a period of 222 weeks.

More details on the data can be found in the following links: * Fire, Michael, and Carlos Guestrin. "The rise and fall of network stars: Analyzing 2.5 million graphs to reveal how high-degree vertices emerge over time." Information Processing & Management 57.2 (2020): 102041.

• Network Dynamics project's website

• Data4Good Lab's Website

Context

Bitcoin is the longest running and most well known cryptocurrency, first released as open source in 2009 by the anonymous Satoshi Nakamoto. Bitcoin serves as a decentralized medium of digital exchange, with transactions verified and recorded in a public distributed ledger (the blockchain) without the need for a trusted record keeping authority or central intermediary. Transaction blocks contain a SHA-256 cryptographic hash of previous transaction blocks, and are thus "chained" together, serving as an immutable record of all transactions that have ever occurred. As with any currency/commodity on the market, bitcoin trading and financial instruments soon followed public adoption of bitcoin and continue to grow. Included here is historical bitcoin market data at 1-min intervals for select bitcoin exchanges where trading takes place. Happy (data) mining!

Content

(See https://github.com/mczielinski/kaggle-bitcoin/ for automation/scraping script)

btcusd_1-min_data.csv

CSV files for select bitcoin exchanges for the time period of Jan 2012 to Present (Measured by UTC day), with minute to minute updates of OHLC (Open, High, Low, Close) and Volume in BTC.

If a timestamp is missing, or if there are jumps, this may be because the exchange (or its API) was down, the exchange (or its API) did not exist, or some other unforeseen technical error in data reporting or gathering. I'm not perfect, and I'm also busy! All effort has been made to deduplicate entries and verify the contents are correct and complete to the best of my ability, but obviously trust at your own risk.

Acknowledgements and Inspiration

Bitcoin charts for the data, originally. Now thank you to the Bitstamp API directly. The various exchange APIs, for making it difficult or unintuitive enough to get OHLC and volume data at 1-min intervals that I set out on this data scraping project. Satoshi Nakamoto and the novel core concept of the blockchain, as well as its first execution via the bitcoin protocol. I'd also like to thank viewers like you! Can't wait to see what code or insights you all have to share.

LuckyBit bets

Datasets used in the creation of "How Bitcoin’s Ups and Downs Are Changing the Way You Bet" by Máté Csaba Sándor and Barna Bakó of Corvinus University of Budapest

These datasets can be used to reproduce the results presented in (1)

The necessary scripts are available at: https://github.com/sampaat/luckybit_gambling

Producing the gambling dataset

The gambling dataset was created using the transactional data extractable from the bitcoin ledger. To make things simpler we have been using a formated dataset from (2), available currently at (3). To replicate the dataset, use the script data_preparation_from_blockchain_excrept.R. Do not run the script as it is, since first you need to download the appropriate datasets from above which takes considerable time. Also there are some variables that need manual adjustment, to avoid overwhelming most desktop computers.

A representative state of the game's website is observable as a web archive

The prepared gambling dataset

The excrept created, containing all transactions of the LuckyBit platform are featured in luckybit_bets_usered.csv

The columns featured in the dataset (names in the first row):

• txID transaction ID of the bet transaction [integer]
• addrID recieving address ID of the targeted game (see *luckybit_games_addresses.csv* for mapping them to games) [integer]
• addrID_in reciving/initiating address ID of the bet/answer transaction, the ID gathered from the dataset, not resolved to true bitcoin IDs [integer]
• value bet ammount (or wager) measured in satoshis (1 satoshi = 1e-8 BTC) [integer]
• block_timestamp blockchain block timestamp (UTC unixtime) used to time the bets [integer]
• userID assigned based on addrID_player using the methods and dataset of (**2**) [integer]

Supporting datasets

Luckybit addresses

Mapping table containing basic information about the possible LuckyBit games are featured in luckybit_games_address.csv

The columns featured in the dataset (names in the first row):

• Name color based identification of the game as shown in the web archive. [string]
• Address string format of the game's bitcoin address hash [string]
• addrID recieving address ID of the targeted game (see luckybit_games_addresses.csv for mapping them to games) [integer]
• ExpectedReturn expected payout of the game on unit bet [float]
• HousePercent expected loss of the game on unit bet [float]
• MaxWin maximum winning multiplier [integer]
• WinProb total probability of positive total payout (win) [float]
• LossProb total probability of negative total payout (loss) [float]
• FlatProb total probability of bet payback (no win or loss) [float]

Luckybit game table

Mapping table containing multipliers and probabilites of each game are featured luckybit_games_table.csv

The columns featured in the dataset (names in the first row):

• n label of result 1-17 [integer]
• blue multiplier of the n-th outcome for the game blue [float]
• green multiplier of the n-th outcome for the game green [float]
• yellow multiplier of the n-th outcome for the game yellow [float]
• red multiplier of the n-th outcome for the game red [float]
• p_win maximum winning multiplier [float]

Clustered users table

Mapping table containing the clusterings assigned to each user are featured clustered_users.csv

The columns featured in the dataset (names in the first row):

• userID assigned based on addrID_player using the methods and dataset of (**2**) [integer]
• cluster calculated in *player_clustering.Rmd* as 1 - All players, 2 - Casual, 3 - Regular, 4 - Extreme [integer]

Clustered users table

Bitcoin to USD exchange rate data featured in bitcoin_historical_data_coinmarketcap.csv

This dataset has been downloaded as of 2024-09-02 from Coinmarketcap

The columns featured in the dataset (names in the first row):

• timeOpen market open, datetime UTC [string]
• timeClose market close, datetime UTC [string]
• timeHigh timing of daily high, datetime UTC [string]
• timeLow timing of daily low, datetime UTC [string]
• name arbitrary ID from Coinmarketcap, uniform, dropable [integer]
• open daily opening exchange rate (USD/BTC) [float]
• high highest daily exchange rate (USD/BTC) [float]
• low lowest daily exchange rate (USD/BTC) [float]
• close daily closing exchange rate (USD/BTC) [float]
• volume daily total traded volume (BTC) [float]
• marketCap total Bitcoin market capitalization in USD [float]
• timeStamp timing of data recording, datetime UTC [string]

References

1. Bakó, B., Sándor, M.C. (2025). How Bitcoin’s Ups and Downs Are Changing the Way You Bet
2. Kondor, D., Pósfai, M., Csabai, I., & Vattay, G. (2014). Do the rich get richer? An empirical analysis of the BitCoin transaction network. PLoS ONE, 9(2), e86197.
3. https://doi.org/10.5061/dryad.qz612jmcf

The dataset contains all historical order book snapshots and blockchain network information used to generate the results for the paper "Building Trust takes Time". A blockchain replaces central counterparties with time-consuming consensus protocols to record the transfer of ownership. This settlement latency slows cross-exchange trading, exposing arbitrageurs to price risk. Off-chain settlement, instead, exposes arbitrageurs to costly default risk. We show with Bitcoin network and order book data that cross-exchange price differences coincide with periods of high settlement latency, asset flows chase arbitrage opportunities, and price differences across exchanges with low default risk are smaller. Blockchain-based trading thus faces a dilemma: reliable consensus protocols require time-consuming settlement latency, leading to arbitrage limits. Circumventing such arbitrage costs is possible only by reinstalling trusted intermediation, which mitigates default risk., This dataset provides three types of Bitcoin-related information:

Centralized crypto-exchange (CEX) characteristics which have been collected manually. High-frequency order book information. The data has been retrieved by regularly fetching order book information from major centralized crypto exchanges (CEX) in minute-level intervals from 2018 - 2019. We provide the entire order book history across the exchanges with this dataset. Corresponding information on the state of the Bitcoin blockchain, for instance, the number of outstanding transactions at every point in time. The data has been used to analyze arbitrage activity across CEXes in relation to the time it takes for validators to execute cross-CEX transactions. Code to replicate the data processing parts is publicly available on Github: www.github.com/voigtstefan/building-trust-takes-time

, , # Data to Replicate the paper Building Trust Takes Time: Limits to Arbitrage for Blockchain-Based Assets

We provide all datasets required to replicate the paper "Building Trust takes Time: Limits to Arbitrage for Blockchain-based Assets". A description of the data sources and preprocessing steps is provided in the paper. All code to generate the results is available on .

Description of the data and file structure

In principle, we offer three different types of crypto-currency-related data:

1. Centralized crypto-exchange (CEX) characteristics, which have been collected manually.
2. High-frequency order book information. The data has been retrieved by regularly fetching order book information from major centralized crypto exchanges (CEX) in minute-level intervals from 2018 - 2019. We provide the entire order book history across the exchanges with this dataset.
3. Corresponding information on the state of the Bitcoin blockchain, for instance, the number of outstanding transacti...

Bitcoin's blockchain size was close to reaching 5450 gigabytes in 2024, as the database saw exponential growth by nearly one gigabyte every few days. The Bitcoin blockchain contains a continuously growing and tamper-evident list of all Bitcoin transactions and records since its initial release in January 2009. Bitcoin has a set limit of 21 million coins, the last of which will be mined around 2140, according to a forecast made in 2017. Bitcoin mining: A somewhat uncharted world Despite interest in the topic, there are few accurate figures on how big Bitcoin mining is on a country-by-country basis. Bitcoin's design philosophy is at the heart of this. Created out of protest against governments and central banks, Bitcoin's blockchain effectively hides both the country of origin and the destination country within a (mining) transaction. Research involving IP addresses placed the United States as the world's most Bitcoin mining country in 2022 - but the source admits IP addresses can easily be manipulated using VPN. Note that mining figures are different from figures on Bitcoin trading: Africa and Latin America were more interested in buying and selling BTC than some of the world's developed economies. Bitcoin developments Bitcoin's trade volume slowed in the second quarter of 2023, after hitting a noticeable growth at the beginning of the year. The coin outperformed most of the market. Some attribute this to the announcement in June 203 that BlackRock filed for a Bitcoin ETF. This iShares Bitcoin Trust was to use Coinbase Custody as its custodian. Regulators in the United States had not yet approved any applications for spot ETFs on Bitcoin.

Context:

This dataset is an extra updating dataset for the G-Research Crypto Forecasting competition.

Introduction

This is a daily updated dataset, automaticlly collecting market data for G-Research crypto forecasting competition. The data is of the 1-minute resolution, collected for all competition assets and both retrieval and uploading are fully automated. see discussion topic.

The Data

For every asset in the competition, the following fields from Binance's official API endpoint for historical candlestick data are collected, saved, and processed.

1. **timestamp** - A timestamp for the minute covered by the row.
2. **Asset_ID** - An ID code for the cryptoasset.
3. **Count** - The number of trades that took place this minute.
4. **Open** - The USD price at the beginning of the minute.
5. **High** - The highest USD price during the minute.
6. **Low** - The lowest USD price during the minute.
7. **Close** - The USD price at the end of the minute.
8. **Volume** - The number of cryptoasset u units traded during the minute.
9. **VWAP** - The volume-weighted average price for the minute.
10. **Target** - 15 minute residualized returns. See the 'Prediction and Evaluation section of this notebook for details of how the target is calculated.
11. **Weight** - Weight, defined by the competition hosts [here](https://www.kaggle.com/cstein06/tutorial-to-the-g-research-crypto-competition)
12. **Asset_Name** - Human readable Asset name.

Indexing

The dataframe is indexed by timestamp and sorted from oldest to newest. The first row starts at the first timestamp available on the exchange, which is July 2017 for the longest-running pairs.

Usage Example

The following is a collection of simple starter notebooks for Kaggle's Crypto Comp showing PurgedTimeSeries in use with the collected dataset. Purged TimesSeries is explained here. There are many configuration variables below to allow you to experiment. Use either GPU or TPU. You can control which years are loaded, which neural networks are used, and whether to use feature engineering. You can experiment with different data preprocessing, model architecture, loss, optimizers, and learning rate schedules. The extra datasets contain the full history of the assets in the same format as the competition, so you can input that into your model too.

Baseline Example Notebooks:

• Neural Network Starter
• LightGBM Starter
• Catboost Starter
• XGBoost Starter
• TabNet Starter
• Reinforcement Learning (PPO) Starter

These notebooks follow the ideas presented in my "Initial Thoughts" here. Some code sections have been reused from Chris' great (great) notebook series on SIIM ISIC melanoma detection competition here

Loose-ends:

This is a work in progress and will be updated constantly throughout the competition. At the moment, there are some known issues that still needed to be addressed:

• VWAP: - At the moment VWAP calculation formula is still unclear. Currently the dataset uses an approximation calculated from the Open, High, Low, Close, Volume candlesticks. [Waiting for competition hosts input]
• Target Labeling: There exist some mismatches to the original target provided by the hosts at some time intervals. On all the others - it is the same. The labeling code can be seen here. [Waiting for competition hosts] input]
• Filtering: No filtration of 0 volume data is taken place.

Example Visualisations

Opening price with an added indicator (MA50): https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2234678%2Fb8664e6f26dc84e9a40d5a3d915c9640%2Fdownload.png?generation=1582053879538546&alt=media" alt="">

Volume and number of trades: https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2234678%2Fcd04ed586b08c1576a7b67d163ad9889%2Fdownload-1.png?generation=1582053899082078&alt=media" alt="">

License

This data is being collected automatically from the crypto exchange Binance.