Current issue 23/09/2020 Please note: Sensors 67, 68 and 69 are showing duplicate records. We are currently working on a fix to resolve this.

This dataset contains minute by minute directional pedestrian counts for the last hour from pedestrian sensor devices located across the city. The data is updated every 15 minutes and can be used to determine variations in pedestrian activity throughout the day. The sensor_id column can be used to merge the data with the Sensor Locations dataset which details the location, status and directional readings of sensors. Any changes to sensor locations are important to consider when analysing and interpreting historical pedestrian counting data. Note this dataset may not contain a reading for every sensor for every minute as sensor devices only create a record when one or more pedestrians have passed underneath the sensor. The Pedestrian Counting System helps us to understand how people use different city locations at different times of day to better inform decision-making and plan for the future. A representation of pedestrian volume which compares each location on any given day and time can be found in our Online Visualisation. Related datasets: Pedestrian Counting System – 2009 to Present (counts per hour).Pedestrian Counting System - Sensor Locations

This layer shows which parts of the United States and Puerto Rico fall within ten minutes' walk of one or more grocery stores. It is estimated that 20% of U.S. population live within a 10 minute walk of a grocery store, and 92% of the population live within a 10 minute drive of a grocery store. The layer is suitable for looking at access at a neighborhood scale.When you add this layer to your web map, along with the drivable access layer and the SafeGraph grocery store layer, it becomes easier to spot grocery stores that sit within a highly populated area, and grocery stores that sit in a shopping center far away from populated areas. Add the Census block points layer to show a popup with the count of stores within 10 minutes' walk and drive. This view of a city begins to hint at the question: how many people have each type of access to grocery stores? And, what if they are unable to walk a mile regularly, or don't own a car?How to Use This Layer in a Web MapUse this layer in a web map to introduce the concepts of access to grocery stores in your city or town. This is the kind of map where people will want to look up their home or work address to validate what the map is saying. See this example web map which you can use in your projects, storymaps, apps and dashboards.The layer was built with that use in mind. Many maps of access use straight-line, as-the-crow-flies distance, which ignores real-world barriers to walkability like rivers, lakes, interstates and other characteristics of the built environment. Block analysis using a network data set and Origin-Destination analysis factors these barriers in, resulting in a more realistic depiction of access.Lastly, this layer can serve as backdrop to other community resources, like food banks, farmers markets (example), and transit (example). Add a transit layer to immediately gauge its impact on the population's grocery access. You can also use this map to see how it relates to communities of concern. Add a layer of any block group or tract demographics, such as Percent Senior Population (examples), or Percent of Households with Access to 0 Vehicles (examples).The layer is a useful visual resource for helping community leaders, business and government leaders see their town from the perspective of its residents, and begin asking questions about how their community could be improved.Data sourcesPopulation data is from the 2010 U.S. Census blocks. Each census block has a count of stores within a 10 minute walk, and a count of stores within a ten minute drive. Census blocks known to be unpopulated are given a score of 0. The layer is available as a hosted feature layer.Grocery store locations are from SafeGraph, reflecting what was in the data as of October 2020. Access to the layer was obtained from the SafeGraph offering in ArcGIS Marketplace. For this project, ArcGIS StreetMap Premium was used for the street network in the origin-destination analysis work, because it already has the necessary attributes on each street segment to identify which streets are considered walkable, and supports a wide variety of driving parameters.The walkable access layer and drivable access layers are rasters, whose colors were chosen to allow the drivable access layer to serve as backdrop to the walkable access layer. Alternative versions of these layers are available. These pairs use different colors but are otherwise identical in content.Data PreparationArcGIS Network Analyst was used to set up a network street layer for analysis. ArcGIS StreetMap Premium was installed to a local hard drive and selected in the Origin-Destination workflow as the network data source. This allows the origins (Census block centroids) and destinations (SafeGraph grocery stores) to be connected to that network, to allow origin-destination analysis.The Census blocks layer contains the centroid of each Census block. The data allows a simple popup to be created. This layer's block figures can be summarized further, to tract, county and state levels.The SafeGraph grocery store locations were created by querying the SafeGraph source layer based on primary NAICS code. After connecting to the layer in ArcGIS Pro, a definition query was set to only show records with NAICS code 445110 as an initial screening. The layer was exported to a local disk drive for further definition query refinement, to eliminate any records that were obviously not grocery stores. The final layer used in the analysis had approximately 53,600 records. In this map, this layer is included as a vector tile layer.MethodologyEvery census block in the U.S. was assigned two access scores, whose numbers are simply how many grocery stores are within a 10 minute walk and a 10 minute drive of that census block. Every census block has a score of 0 (no stores), 1, 2 or more stores. The count of accessible stores was determined using Origin-Destination Analysis in ArcGIS Network Analyst, in ArcGIS Pro. A set of Tools in this ArcGIS Pro package allow a similar analysis to be conducted for any city or other area. The Tools step through the data prep and analysis steps. Download the Pro package, open it and substitute your own layers for Origins and Destinations. Parcel centroids are a suggested option for Origins, for example. Origin-Destination analysis was configured, using ArcGIS StreetMap Premium as the network data source. Census block centroids with population greater than zero were used as the Origins, and grocery store locations were used as the Destinations. A cutoff of 10 minutes was used with the Walk Time option. Only one restriction was applied to the street network: Walkable, which means Interstates and other non-walkable street segments were treated appropriately. You see the results in the map: wherever freeway overpasses and underpasses are present near a grocery store, the walkable area extends across/through that pass, but not along the freeway.A cutoff of 10 minutes was used with the Drive Time option. The default restrictions were applied to the street network, which means a typical vehicle's access to all types of roads was factored in.The results for each analysis were captured in the Lines layer, which shows which origins are within the cutoff of each destination over the street network, given the assumptions about that network (walking, or driving a vehicle).The Lines layer was then summarized by census block ID to capture the Maximum value of the Destination_Rank field. A census block within 10 minutes of 3 stores would have 3 records in the Lines layer, but only one value in the summarized table, with a MAX_Destination_Rank field value of 3. This is the number of stores accessible to that census block in the 10 minutes measured, for walking and driving. These data were joined to the block centroids layer and given unique names. At this point, all blocks with zero population or null values in the MAX_Destination_Rank fields were given a store count of 0, to help the next step.Walkable and Drivable areas are calculated into a raster layer, using Nearest Neighbor geoprocessing tool on the count of stores within a 10 minute walk, and a count of stores within a ten minute drive, respectively. This tool uses a 200 meter grid and interpolates the values between each census block. A census tracts layer containing all water polygons "erased" from the census tract boundaries was used as an environment setting, to help constrain interpolation into/across bodies of water. The same layer use used to "shoreline" the Nearest Neighbor results, to eliminate any interpolation into the ocean or Great Lakes. This helped but was not perfect.Notes and LimitationsThe map provides a baseline for discussing access to grocery stores in a city. It does not presume local population has the desire or means to walk or drive to obtain groceries. It does not take elevation gain or loss into account. It does not factor time of day nor weather, seasons, or other variables that affect a person's commute choices. Walking and driving are just two ways people get to a grocery store. Some people ride a bike, others take public transit, have groceries delivered, or rely on a friend with a vehicle. Thank you to Melinda Morang on the Network Analyst team for guidance and suggestions at key moments along the way; to Emily Meriam for reviewing the previous version of this map and creating new color palettes and marker symbols specific to this project. Additional ReadingThe methods by which access to food is measured and reported have improved in the past decade or so, as has the uses of such measurements. Some relevant papers and articles are provided below as a starting point.Measuring Food Insecurity Using the Food Abundance Index: Implications for Economic, Health and Social Well-BeingHow to Identify Food Deserts: Measuring Physical and Economic Access to Supermarkets in King County, WashingtonAccess to Affordable and Nutritious Food: Measuring and Understanding Food Deserts and Their ConsequencesDifferent Measures of Food Access Inform Different SolutionsThe time cost of access to food – Distance to the grocery store as measured in minutes

How much time do people spend on social media? As of 2024, the average daily social media usage of internet users worldwide amounted to 143 minutes per day, down from 151 minutes in the previous year. Currently, the country with the most time spent on social media per day is Brazil, with online users spending an average of three hours and 49 minutes on social media each day. In comparison, the daily time spent with social media in the U.S. was just two hours and 16 minutes. Global social media usageCurrently, the global social network penetration rate is 62.3 percent. Northern Europe had an 81.7 percent social media penetration rate, topping the ranking of global social media usage by region. Eastern and Middle Africa closed the ranking with 10.1 and 9.6 percent usage reach, respectively. People access social media for a variety of reasons. Users like to find funny or entertaining content and enjoy sharing photos and videos with friends, but mainly use social media to stay in touch with current events friends. Global impact of social mediaSocial media has a wide-reaching and significant impact on not only online activities but also offline behavior and life in general. During a global online user survey in February 2019, a significant share of respondents stated that social media had increased their access to information, ease of communication, and freedom of expression. On the flip side, respondents also felt that social media had worsened their personal privacy, increased a polarization in politics and heightened everyday distractions.

Context:

This dataset is an extra updating dataset for the G-Research Crypto Forecasting competition.

Introduction

This is a daily updated dataset, automaticlly collecting market data for G-Research crypto forecasting competition. The data is of the 1-minute resolution, collected for all competition assets and both retrieval and uploading are fully automated. see discussion topic.

The Data

For every asset in the competition, the following fields from Binance's official API endpoint for historical candlestick data are collected, saved, and processed.

1. **timestamp** - A timestamp for the minute covered by the row.
2. **Asset_ID** - An ID code for the cryptoasset.
3. **Count** - The number of trades that took place this minute.
4. **Open** - The USD price at the beginning of the minute.
5. **High** - The highest USD price during the minute.
6. **Low** - The lowest USD price during the minute.
7. **Close** - The USD price at the end of the minute.
8. **Volume** - The number of cryptoasset u units traded during the minute.
9. **VWAP** - The volume-weighted average price for the minute.
10. **Target** - 15 minute residualized returns. See the 'Prediction and Evaluation section of this notebook for details of how the target is calculated.
11. **Weight** - Weight, defined by the competition hosts [here](https://www.kaggle.com/cstein06/tutorial-to-the-g-research-crypto-competition)
12. **Asset_Name** - Human readable Asset name.

Indexing

The dataframe is indexed by timestamp and sorted from oldest to newest. The first row starts at the first timestamp available on the exchange, which is July 2017 for the longest-running pairs.

Usage Example

The following is a collection of simple starter notebooks for Kaggle's Crypto Comp showing PurgedTimeSeries in use with the collected dataset. Purged TimesSeries is explained here. There are many configuration variables below to allow you to experiment. Use either GPU or TPU. You can control which years are loaded, which neural networks are used, and whether to use feature engineering. You can experiment with different data preprocessing, model architecture, loss, optimizers, and learning rate schedules. The extra datasets contain the full history of the assets in the same format as the competition, so you can input that into your model too.

Baseline Example Notebooks:

• Neural Network Starter
• LightGBM Starter
• Catboost Starter
• XGBoost Starter
• TabNet Starter
• Reinforcement Learning (PPO) Starter

These notebooks follow the ideas presented in my "Initial Thoughts" here. Some code sections have been reused from Chris' great (great) notebook series on SIIM ISIC melanoma detection competition here

Loose-ends:

This is a work in progress and will be updated constantly throughout the competition. At the moment, there are some known issues that still needed to be addressed:

• VWAP: - At the moment VWAP calculation formula is still unclear. Currently the dataset uses an approximation calculated from the Open, High, Low, Close, Volume candlesticks. [Waiting for competition hosts input]
• Target Labeling: There exist some mismatches to the original target provided by the hosts at some time intervals. On all the others - it is the same. The labeling code can be seen here. [Waiting for competition hosts] input]
• Filtering: No filtration of 0 volume data is taken place.

Example Visualisations

Opening price with an added indicator (MA50): https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2234678%2Fb8664e6f26dc84e9a40d5a3d915c9640%2Fdownload.png?generation=1582053879538546&alt=media" alt="">

Volume and number of trades: https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2234678%2Fcd04ed586b08c1576a7b67d163ad9889%2Fdownload-1.png?generation=1582053899082078&alt=media" alt="">

License

This data is being collected automatically from the crypto exchange Binance.

This dataset contains hourly pedestrian counts since 2009 from pedestrian sensor devices located across the city. The data is updated on a monthly basis and can be used to determine variations in pedestrian activity throughout the day.The sensor_id column can be used to merge the data with the Pedestrian Counting System - Sensor Locations dataset which details the location, status and directional readings of sensors. Any changes to sensor locations are important to consider when analysing and interpreting pedestrian counts over time.Importants notes about this dataset:• Where no pedestrians have passed underneath a sensor during an hour, a count of zero will be shown for the sensor for that hour.• Directional readings are not included, though we hope to make this available later in the year. Directional readings are provided in the Pedestrian Counting System – Past Hour (counts per minute) dataset.The Pedestrian Counting System helps to understand how people use different city locations at different times of day to better inform decision-making and plan for the future. A representation of pedestrian volume which compares each location on any given day and time can be found in our Online Visualisation.Related datasets:Pedestrian Counting System – Past Hour (counts per minute)Pedestrian Counting System - Sensor Locations

This is NOT a raw population dataset. We use our proprietary stack to combine detailed 'WorldPop' UN-adjusted, sex and age structured population data with a spatiotemporal OD matrix.

The result is a dataset where each record indicates how many people can be reached in a fixed timeframe (60 Mins in this case) from that record's location.

The dataset is broken down into sex and age bands at 5 year intervals, e.g - male 25-29 (m_25) and also contains a set of features detailing the representative percentage of the total that the count represents.

The dataset provides 76174 records, one for each sampled location. These are labelled with a h3 index at resolution 7 - this allows easy plotting and filtering in Kepler.gl / Deck.gl / Mapbox, or easy conversion to a centroid (lat/lng) or the representative geometry of the hexagonal cell for integration with your geospatial applications and analyses.

A h3 resolution of 7, is a hexagonal cell area equivalent to: - ~1.9928 sq miles - ~5.1613 sq km

Higher resolutions or alternate geographies are available on request.

More information on the h3 system is available here: https://eng.uber.com/h3/

WorldPop data provides for a population count using a grid of 1 arc second intervals and is available for every geography.

More information on the WorldPop data is available here: https://www.worldpop.org/

One of the main use cases historically has been in prospecting for site selection, comparative analysis and network validation by asset investors and logistics companies. The data structure makes it very simple to filter out areas which do not meet requirements such as: - being able to access 70% of the German population within 4 hours by Truck and show only the areas which do exhibit this characteristic.

Clients often combine different datasets either for different timeframes of interest, or to understand different populations, such as that of the unemployed, or those with particular qualifications within areas reachable as a commute.

Cassava (Manihot esculenta) also known as manioc in South America, is grown world-wide in tropical and sub-tropical regions providing an important staple for the diet of over half a billion people. It is drought tolerant and grows well in marginal soils. More than half of the world's cassava production is from Africa and Nigeria is the world's largest producer. In Ghana, cassava accounts for roughly 30% of the calories eaten. The root of the cassava plant must be prepared to remove harmful compounds prior to eating. Dataset SummaryThis layer provides access to a 5 arc-minute (approximately 10 km at the equator) cell-sized raster of the 1999-2001 annual average area of cassava harvested in Africa. The data are in units of hectares/grid cell.The SPAM 2000 v3.0.6 data used to create this layer were produced by the International Food Policy Research Institute in 2012. This dataset was created by spatially disaggregating national and sub-national harvest data using the Spatial Production Allocation Model. Link to source metadataFor more information about this dataset and the importance of casava as a staple food see the Harvest Choice webpage.For data on other agricultural species in Africa see these layers:Groundnut (Peanut)Maize (Corn)MilletPotatoRiceSorghumSweet Potato and YamWheatData for important agricultural crops in South America are available here.What can you do with this layer?This layer is suitable for both visualization and analysis. It can be used in ArcGIS Online in web maps and applications and can be used in ArcGIS Desktop.This layer has query, identify, and export image services available. This layer is restricted to a maximum area of 24,000 x 24,000 pixels which allows access to the full dataset.The source data for this layer are available here.This layer is part of a larger collection of landscape layers that you can use to perform a wide variety of mapping and analysis tasks.The Living Atlas of the World provides an easy way to explore the landscape layers and many other beautiful and authoritative maps on hundreds of topics.

Seattle Parks and Recreation 2023 Walkability Gap Analysis. SPR’s intent is to gain a more accurate picture of access, by measuring how people walk to a park or recreation facility. We are calling this ‘walkability’.This map shows what a 5-minute and a 10-minute walking distance (or walkability area) looks like around park lands that are greater than 10,000 square feet in size.

Rice (Oryza sativa and O. glaberrima) is one of the world's most important staple food crops. Over half of the world's population relies on rice. The people in some parts of Africa have been cultivating rice for over 3,500 years.Dataset SummaryThis layer provides access to a 5 arc-minute (approximately 10 km at the equator) cell-sized raster of the 1999-2001 annual average area of rice harvested in Africa. The data are in units of hectares/grid cell.The SPAM 2000 v3.0.6 data used to create this layer were produced by the International Food Policy Research Institute in 2012. This dataset was created by spatially disaggregating national and sub-national harvest data using the Spatial Production Allocation Model. Link to source metadataFor more information about this dataset and the importance of rice as a staple food see the Harvest Choice webpage.For data on other agricultural species in Africa see these layers:CassavaGroundnut (Peanut)Maize (Corn)MilletPotatoSorghumSweet Potato and YamWheatData for important agricultural crops in South America are available here.What can you do with this layer?This layer is suitable for both visualization and analysis. It can be used in ArcGIS Online in web maps and applications and can be used in ArcGIS Desktop.This layer has query, identify, and export image services available. This layer is restricted to a maximum area of 24,000 x 24,000 pixels which allows access to the full dataset.The source data for this layer are available here.This layer is part of a larger collection of landscape layers that you can use to perform a wide variety of mapping and analysis tasks.The Living Atlas of the World provides an easy way to explore the landscape layers and many other beautiful and authoritative maps on hundreds of topics.Geonet is a good resource for learning more about landscape layers and the Living Atlas of the World. To get started follow these links:Landscape Layers - a reintroductionLiving Atlas Discussion Group

Meteorological and hydrological data monitored at stations in the Limpopo Basin in southern Africa. Most data are recorded every 15-minutes. The transboundary Limpopo River Basin crosses Botswana, Mozambique, South Africa, and Zimbabwe. At over 400,000 km2, the Limpopo River Basin is home to 18 million people living in both rural and urban areas. Industries in the Basin include businesses in the urban areas and water-intensive uses such as agriculture and mining; industrial water use is growing rapidly (LBPTC, 2010). In addition to the human residents, the Basin contains some of the most biodiverse natural areas on the planet (Kahinda et al., 2016). The rainfall in the Basin is heterogeneous with some sub-basins receiving less than 400 mm on average and other downstream sub-basins in Mozambique receiving over 750 mm annually (LBPTC, 2010). Even meteorological stations located in close proximity demonstrate substantial spatial variation within sub-basins. The Basin has experienced severe droughts in the last decade (LBPTC, 2010). In addition to the variation in the amount of rainfall, the timing, especially the start of the growing season, has varied significantly (Edokpayi et al., 2018). However, there remain many questions about the reliability of rainfall data and other water measurements due in part to the infrequent calibration and validation of field site measurements. The limited confidence in these data, combined with the substantial variation through time and space necessitates an integrated approach to improve data collection, validation, and overall Basin water resource management in the Basin. The goal of this project is to build resilience through the support of Basin stakeholders, including The Limpopo Watercourse Commission (LIMCOM), to improve governance around water resources management and water security in the Basin. A systems approach, such as integrated water resources management (IWRM) is needed to address such complex, large, and interrelated components of water resources. IRWM is recommended by the United States Agency for International Development (USAID) Water and Development Strategy Implementation Guide (2014). This context will be combined with data collection and validation, data sharing, and continuous evaluation of the interrelations that affect water resources. For example, surface water quality will affect ecosystem biodiversity and those who depend on those ecosystems for drinking water or fish as a food protein source. Another example is groundwater; Petrie et al. (2014) found that groundwater in the Limpopo Basin holds great potential for increased water access; however, groundwater measurement data are currently insufficient to calculate sustainable total withdrawal rates. With improved data collection, validation, and sharing of these data, managers will have a basis for scientifically-based decisions and may be able to utilize groundwater resources more effectively to increase (e.g., agricultural) productivity. These same resource managers need monitoring and surveillance tools to measure water consumption by users within the basin, how demand and use changes through time, and to understand if historic and current sparse monitoring is accurate. Since rural users withdraw water in informal schemes, measurements of water use currently do not exist. This project will support water resources monitoring, and the development of methods for water quality and quantity measurement based on in situ sensors and satellite measurements. These measurements will enable characterization of water resource dynamics at the whole Basin scale and form the foundation for hydrologic modeling that can help estimate hard-to-measure parameters and also provide holistic assessments of Basin scale stocks and flows. To support data sharing, the project will use cloud-based, automated data collection and web-based data sharing. All sensors and data will be shared. Increased monitoring and

The total amount of data created, captured, copied, and consumed globally is forecast to increase rapidly, reaching 149 zettabytes in 2024. Over the next five years up to 2028, global data creation is projected to grow to more than 394 zettabytes. In 2020, the amount of data created and replicated reached a new high. The growth was higher than previously expected, caused by the increased demand due to the COVID-19 pandemic, as more people worked and learned from home and used home entertainment options more often. Storage capacity also growing Only a small percentage of this newly created data is kept though, as just two percent of the data produced and consumed in 2020 was saved and retained into 2021. In line with the strong growth of the data volume, the installed base of storage capacity is forecast to increase, growing at a compound annual growth rate of 19.2 percent over the forecast period from 2020 to 2025. In 2020, the installed base of storage capacity reached 6.7 zettabytes.

In 2023, the number of data compromises in the United States stood at 3,205 cases. Meanwhile, over 353 million individuals were affected in the same year by data compromises, including data breaches, leakage, and exposure. While these are three different events, they have one thing in common. As a result of all three incidents, the sensitive data is accessed by an unauthorized threat actor. Industries most vulnerable to data breaches Some industry sectors usually see more significant cases of private data violations than others. This is determined by the type and volume of the personal information organizations of these sectors store. In 2022, healthcare, financial services, and manufacturing were the three industry sectors that recorded most data breaches. The number of healthcare data breaches in the United States has gradually increased within the past few years. In the financial sector, data compromises increased almost twice between 2020 and 2022, while manufacturing saw an increase of more than three times in data compromise incidents. Largest data exposures worldwide In 2020, an adult streaming website, CAM4, experienced a leakage of nearly 11 billion records. This, by far, is the most extensive reported data leakage. This case, though, is unique because cyber security researchers found the vulnerability before the cyber criminals. The second-largest data breach is the Yahoo data breach, dating back to 2013. The company first reported about one billion exposed records, then later, in 2017, came up with an updated number of leaked records, which was three billion. In March 2018, the third biggest data breach happened, involving India’s national identification database Aadhaar. As a result of this incident, over 1.1 billion records were exposed.