The total amount of data created, captured, copied, and consumed globally is forecast to increase rapidly, reaching *** zettabytes in 2024. Over the next five years up to 2028, global data creation is projected to grow to more than *** zettabytes. In 2020, the amount of data created and replicated reached a new high. The growth was higher than previously expected, caused by the increased demand due to the COVID-19 pandemic, as more people worked and learned from home and used home entertainment options more often. Storage capacity also growing Only a small percentage of this newly created data is kept though, as just * percent of the data produced and consumed in 2020 was saved and retained into 2021. In line with the strong growth of the data volume, the installed base of storage capacity is forecast to increase, growing at a compound annual growth rate of **** percent over the forecast period from 2020 to 2025. In 2020, the installed base of storage capacity reached *** zettabytes.

How many people use social media?

              Social media usage is one of the most popular online activities. In 2024, over five billion people were using social media worldwide, a number projected to increase to over six billion in 2028.

              Who uses social media?
              Social networking is one of the most popular digital activities worldwide and it is no surprise that social networking penetration across all regions is constantly increasing. As of January 2023, the global social media usage rate stood at 59 percent. This figure is anticipated to grow as lesser developed digital markets catch up with other regions
              when it comes to infrastructure development and the availability of cheap mobile devices. In fact, most of social media’s global growth is driven by the increasing usage of mobile devices. Mobile-first market Eastern Asia topped the global ranking of mobile social networking penetration, followed by established digital powerhouses such as the Americas and Northern Europe.

              How much time do people spend on social media?
              Social media is an integral part of daily internet usage. On average, internet users spend 151 minutes per day on social media and messaging apps, an increase of 40 minutes since 2015. On average, internet users in Latin America had the highest average time spent per day on social media.

              What are the most popular social media platforms?
              Market leader Facebook was the first social network to surpass one billion registered accounts and currently boasts approximately 2.9 billion monthly active users, making it the most popular social network worldwide. In June 2023, the top social media apps in the Apple App Store included mobile messaging apps WhatsApp and Telegram Messenger, as well as the ever-popular app version of Facebook.

As of February 2025, 5.56 billion individuals worldwide were internet users, which amounted to 67.9 percent of the global population. Of this total, 5.24 billion, or 63.9 percent of the world's population, were social media users. Global internet usage Connecting billions of people worldwide, the internet is a core pillar of the modern information society. Northern Europe ranked first among worldwide regions by the share of the population using the internet in 20254. In The Netherlands, Norway and Saudi Arabia, 99 percent of the population used the internet as of February 2025. North Korea was at the opposite end of the spectrum, with virtually no internet usage penetration among the general population, ranking last worldwide. Eastern Asia was home to the largest number of online users worldwide – over 1.34 billion at the latest count. Southern Asia ranked second, with around 1.2 billion internet users. China, India, and the United States rank ahead of other countries worldwide by the number of internet users. Worldwide internet user demographics As of 2024, the share of female internet users worldwide was 65 percent, five percent less than that of men. Gender disparity in internet usage was bigger in African countries, with around a ten percent difference. Worldwide regions, like the Commonwealth of Independent States and Europe, showed a smaller usage gap between these two genders. As of 2024, global internet usage was higher among individuals between 15 and 24 years old across all regions, with young people in Europe representing the most significant usage penetration, 98 percent. In comparison, the worldwide average for the age group 15–24 years was 79 percent. The income level of the countries was also an essential factor for internet access, as 93 percent of the population of the countries with high income reportedly used the internet, as opposed to only 27 percent of the low-income markets.

Facebook received 73,390 user data requests from federal agencies and courts in the United States during the second half of 2023. The social network produced some user data in 88.84 percent of requests from U.S. federal authorities. The United States accounts for the largest share of Facebook user data requests worldwide.

Autonomous Data Platform Market Outlook

In 2023, the global autonomous data platform market size was valued at approximately USD 2.5 billion, and it is forecasted to reach USD 10.5 billion by 2032, growing at a compound annual growth rate (CAGR) of 17.5% during this period. The growth of this market is primarily driven by the surge in demand for advanced data analytics and the increasing need for data-driven decision-making processes across various sectors. The widespread adoption of artificial intelligence (AI) and machine learning (ML) technologies to automate data management tasks is a significant growth factor, enabling businesses to harness data more efficiently and effectively.

One of the critical growth factors of the autonomous data platform market is the exponential increase in data generation and the complexity associated with data management. Organizations are overwhelmed with the amount of structured and unstructured data generated every day, which necessitates a robust platform that can autonomously manage, integrate, and analyze data without human intervention. The ability of autonomous data platforms to reduce operational costs by automating repetitive data management tasks, such as data cleaning, data preparation, and data integration, makes them highly appealing to enterprises seeking cost-effective solutions. Furthermore, these platforms enable businesses to derive actionable insights more rapidly, allowing for quicker response to market changes and improved decision-making capabilities.

Another significant growth driver is the increasing reliance on hybrid and multi-cloud environments. As organizations transition towards digital transformation, the use of cloud-based solutions is becoming more prevalent. Autonomous data platforms offer seamless integration with existing cloud infrastructures, providing flexibility and scalability while ensuring data security and compliance. The cloud-based deployment mode of these platforms supports remote data access, offering businesses the agility to operate across geographically dispersed locations. Moreover, the integration of AI and ML capabilities into autonomous data platforms enhances predictive analytics, allowing organizations to anticipate trends and make informed business decisions.

The growing need for enhanced data governance and regulatory compliance is also propelling the adoption of autonomous data platforms. As data privacy regulations such as the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA) become more stringent, organizations must ensure that their data management practices comply with these regulations. Autonomous data platforms provide robust data governance frameworks, enabling enterprises to maintain compliance while minimizing the risk of data breaches and ensuring data quality. This capability is especially critical for industries such as banking, financial services, and healthcare, where data integrity and security are paramount.

Regionally, North America holds the largest share of the autonomous data platform market, driven by the high concentration of technology companies and the rapid adoption of advanced analytics solutions. The presence of major market players and a strong focus on research and development are also contributing to the market's growth in this region. Moreover, Asia Pacific is anticipated to witness the highest growth rate during the forecast period, attributed to the increasing digitalization efforts and the growing adoption of cloud-based solutions in emerging economies like China and India. In Europe, the market is driven by the emphasis on data privacy and stringent regulatory frameworks, encouraging organizations to adopt autonomous data platforms to ensure compliance and data protection.

Component Analysis

The components of the autonomous data platform market are primarily segmented into platforms and services. The platform segment is the backbone of the entire market, providing the essential infrastructure for data management and analytics. Autonomous data platforms incorporate AI and ML algorithms to automate various data tasks, such as integration, preparation, and analysis. The ability to self-optimize and self-heal makes these platforms indispensable for organizations dealing with large volumes of data. The platform's role is to streamline data processes, reduce human intervention, and thereby lower operational costs. Organizations favor platforms that offer seamless integration with existing systems and provide scalability to handle dynamic data needs. As more companies aim to become data-driven, the demand for comprehensive platforms that c

The Everyday Itinerary Dataset is the first public-use dataset in the Dunham's Data series, a unique data collection created by Kate Elswit (Royal Central School of Speech and Drama, University of London) and Harmony Bench (The Ohio State University) to explore questions and problems that make the analysis and visualization of data meaningful for dance history through the case study of choreographer Katherine Dunham. It is a manually curated dataset of Katherine Dunham's touring from 1937-1962, encompassing Dunham's daily locations, travel, and performances. This dataset tracks geographic location and, less comprehensively, the accommodation in which Dunham stayed each night; the theatres, nightclubs, television studios, and other places where she and the company performed; the modes of transportation used when travel occurred; additional transit cities through which she passed; and whether or not Dunham was likely to be in rehearsals or giving public performances. Dunham's Data: Digital Methods for Dance Historical Inquiry is funded by the United Kingdom Arts and Humanities Research Council (AHRC AH/R012989/1, 2018-2022) and is part of a larger suite of ongoing digital collaborations by Bench and Elswit, Movement on the Move. The Dunham's Data team also includes digital humanities postdoctoral research assistant Antonio Jiménez-Mavillard and dance history postdoctoral research assistants Takiyah Nur Amin and Tia-Monique Uzor.

As of the third quarter of 2024, internet users in South Africa spent more than **** hours and ** minutes online per day, ranking first among the regions worldwide. Brazil followed, with roughly **** hours of daily online usage. As of the examined period, Japan registered the lowest number of daily hours spent online, with users in the country spending an average of over **** hours per day using the internet. The data includes the daily time spent online on any device. Social media usage In recent years, social media has become integral to internet users' daily lives, with users spending an average of *** minutes daily on social media activities. In April 2024, global social network penetration reached **** percent, highlighting its widespread adoption. Among the various platforms, YouTube stands out, with over *** billion monthly active users, making it one of the most popular social media platforms. YouTube’s global popularity In 2023, the keyword "YouTube" ranked among the most popular search queries on Google, highlighting the platform's immense popularity. YouTube generated most of its traffic through mobile devices, with about 98 billion visits. This popularity was particularly evident in the United Arab Emirates, where YouTube penetration reached approximately **** percent, the highest in the world.

App Download Key StatisticsApp and Game DownloadsiOS App and Game DownloadsGoogle Play App and Game DownloadsGame DownloadsiOS Game DownloadsGoogle Play Game DownloadsApp DownloadsiOS App...

According to Cognitive Market Research, the global Data Protection as a Service DPAAS market size will be USD 28241.8 million in 2025. It will expand at a compound annual growth rate (CAGR) of 20.80% from 2025 to 2033.

North America held the major market share for more than 40% of the global revenue with a market size of USD 10449.47 million in 2025 and will grow at a compound annual growth rate (CAGR) of 18.6% from 2025 to 2033.
Europe accounted for a market share of over 30% of the global revenue with a market size of USD 8190.12 million.
APAC held a market share of around 23% of the global revenue with a market size of USD 6778.03 million in 2025 and will grow at a compound annual growth rate (CAGR) of 22.8% from 2025 to 2033.
South America has a market share of more than 5% of the global revenue with a market size of USD 1073.19 million in 2025 and will grow at a compound annual growth rate (CAGR) of 19.8% from 2025 to 2033.
Middle East had a market share of around 2% of the global revenue and was estimated at a market size of USD 1129.67 million in 2025 and will grow at a compound annual growth rate (CAGR) of 20.1% from 2025 to 2033.
Africa had a market share of around 1% of the global revenue and was estimated at a market size of USD 621.32 million in 2025 and will grow at a compound annual growth rate (CAGR) of 20.5% from 2025 to 2033.
Payment Processing category is the fastest growing segment of the Data Protection as a Service DPAAS industry

Market Dynamics of Data Protection as a Service DPAAS Market

Key Drivers for Data Protection as a Service DPAAS Market

Escalating Cybersecurity Threats and Data Breaches to Boost Market Growth

The rising frequency and complexity of cyberattacks have significantly intensified concerns around data security. Organizations are increasingly grappling with threats such as ransomware, data breaches, and phishing attacks, which can result in severe financial losses and reputational harm. For example, in 2023, the U.S. reported 2,365 data breaches impacting approximately 343.3 million individuals—a staggering 72% increase compared to 2021. In the UK, half of all businesses (50%) and nearly a third of charities (32%) reported experiencing some form of cybersecurity breach or attack in the past year. The figures are even higher among medium-sized businesses (70%), large enterprises (74%), and high-income charities with annual revenues over £500,000 (66%). Phishing remains the most prevalent type of attack, affecting 84% of businesses and 83% of charities. This is followed by impersonation attacks via email or online platforms (35% of businesses and 37% of charities) and malware infections (17% of businesses and 14% of charities). This escalating threat landscape highlights the critical need for robust data protection strategies, driving demand for Data Protection as a Service (DPaaS) solution. These services offer advanced security features such as data encryption, multi-factor authentication, and real-time monitoring to help organizations safeguard their sensitive information.

https://www.gov.uk/government/statistics/cyber-security-breaches-survey-2024/cyber-security-breaches-survey-2024//./

Increasing Data Volumes from Digital Transformation and IoT to Boost Market Growth

The rapid surge in data generation—driven by digital transformation initiatives and the widespread adoption of Internet of Things (IoT) devices—has created an urgent need for efficient storage, backup, and recovery solutions. Global data volume skyrocketed from 2 zettabytes (ZB) in 2010 to an astounding 64.2 ZB by 2020, surpassing even the number of observable stars in the universe. This figure is projected to reach 181 ZB by 2025. Despite this explosive growth, only about 2% of the data created in 2020 was actually saved and stored by 2021. On a daily basis, the world produces around 2.5 quintillion bytes of data, with 90% of all existing data generated in just the past two years. Additionally, over 40% of internet data in 2020 was generated by machines. In this context, Data Protection as a Service (DPaaS) emerges as a vital solution, offering scalable, secure, and cost-effective means to protect this ever-expanding volume of data. DPaaS ensures data availability, security, and compliance with increasingly stringent regulatory requirements.

https://spacelift.io/blog/how-much-data-is-generated-every-day./

Restraint Factor for the Da...

The size of the Data Monetization market was valued at USD XXX Million in 2023 and is projected to reach USD XXX Million by 2032, with an expected CAGR of 19.94% during the forecast period.Data monetization refers to the processes of making data a source of value, and it has been one of the key drivers of innovation, coupled with being among the sources of revenues. Using vast quantities of data generated every day, big firms can unlock new opportunities and gain competitive advantage. There are various means of data monetization including direct sales, licensing, partnerships in the form of sharing, and data-based product developments. Business operations hence enable the extraction of actionable insights through interpretation of data-some of which can influence decisions, maximize operation performance, and develop new products and services. Data monetization has various applied uses that cut across most industries. It might be that they use data from anonymized patients to create sophisticated medical treatments and patient improvement within the health sector. The financial services sector might find fraud patterns using data analytics, which leads to optimal investments. The retail industry might then have these data-driven insights to better improve customer experience and personalize specific marketing campaigns. Actually, the only thing that seems to be limiting the extreme potential for data monetization is volume and complexity. In fact, data-driven strategies can open up for an organization the following new possibilities: new revenues, efficiency in operations, and sustainable growth. Recent developments include: April 2024: Carv, a data layer platform that lets web3 gaming apps, AI companies, and gamers control and monetize their data, raised a USD 10 million series A round led by Tribe Capital and IOSG Ventures. The company differentiates itself by empowering users with data ownership and monetization rights, which are expected to support the market growth during the forecast period., February 2024: Tecnotree, a digital platform and service leader for AI, 5G, and cloud-native technologies, partnered with BytePlus, the enterprise arm of Bytedance, to transform wholesale enterprise monetization through the Tecnotree Moments campaign management program for CSPs. This collaboration plans to work toward B2B2X digital ecosystem management, showcasing the growth opportunity of AI and API monetization strategies for CSPs across the world.. Key drivers for this market are: Rapid Adoption of Advanced Analytics and Visualization, Increasing Volume and Variety of Business Data. Potential restraints include: Interoperability With Existing Systems, Varying Structure of Regulatory Policies. Notable trends are: Large Enterprises to Hold Major Market Share.

Dataset contains basic changed data of RÚIAN, e.g. basic descriptive data of territorial elements and units of territorial registration, for which at least one of their attributes has changed on the selected day. Dataset contains no spatial location (polygons and definition lines) and centroids of RÚIAN elements. The file contains following elements (in case they have been changed): state, cohesion region, higher territorial self-governing entity (VÚSC), municipality with extended competence(ORP), authorized municipal office (POU), region (old ones – defined in 1960), county, municipality, municipality part, town district (MOMC), Prague city district (MOP), town district of Prague (SOP), cadastral units and basic urban units (ZSJ), streets, building objects and address point. Up-to-date data is specified for each element: code, centroid (if exists) and all available descriptive attributes including the code of superior element. Dataset is provided as Open Data (licence CC-BY 4.0). Data is based on RÚIAN (Register of Territorial Identification, Addresses and Real Estates). Files are created every day (in case any change of any element occurred). Data is provided in RÚIAN exchange format (VFR), which is based on XML language and fulfils the GML 3.2.1 standard (according to ISO 19136:2007). Dataset is compressed (ZIP) for downloading. More in the Act No. 111/2009 Coll., on the Basic Registers, in Decree No. 359/2011 Coll., on the Basic Register of Territorial Identification, Addresses and Real Estates.

Dataset contains full descriptive data and spatial location of territorial elements and units of territorial registration, for which at least one of their attributes has changed on the selected day. Data is generated in 3 files for the whole Czech Republic. In all files all available descriptive attributes are listed for each element including centroid (if it exists). File ST_ZKSG contains following elements: state, cohesion region, higher territorial self-governing entity (VÚSC), municipality with extended competence(ORP), authorized municipal office (POU), region (old ones – defined in 1960), county, municipality, municipality part, municipality district/city part (MOMC - for territorialy structured statutory cities), Prague city district (MOP), town district of Prague (SOP) for Prague, cadastral units and basic urban units (ZSJ), futhermore generalised boundaries for all elements (if it does not exist, then original boundaries). File ST_ZKSH for the whole state contains following elements: state, cohesion region, VÚSC, ORP, POU, region (old – defined in 1960), and county, municipality, part of municipality, MOMC, MOP, SOP, cadastral unit, ZSJ, parcels (including their polygons), building objects , streets (inluding their definition lines) and address points, futhermore original boundaries for all higher elements. File ST_ZKSO contains changes of pictures of flags and emblems of municipalities and MOMC. Dataset is provided as Open Data (licence CC-BY 4.0). Data is based on RÚIAN (Register of Territorial Identification, Addresses and Real Estates). Files are created every day (in case any change of any element occurred). Data is provided in RÚIAN exchange format (VFR), which is based on XML language and fulfils the GML 3.2.1 standard (according to ISO 19136:2007). Dataset is compressed (ZIP) for downloading. More in the Act No. 111/2009 Coll., on the Basic Registers, in Decree No. 359/2011 Coll., on the Basic Register of Territorial Identification, Addresses and Real Estates.

This dataset contains all data and code necessary to reproduce the analysis presented in the manuscript: Winzeler, H.E., Owens, P.R., Read Q.D.., Libohova, Z., Ashworth, A., Sauer, T. 2022. 2022. Topographic wetness index as a proxy for soil moisture in a hillslope catena: flow algorithms and map generalization. Land 11:2018. DOI: 10.3390/land11112018. There are several steps to this analysis. The relevant scripts for each are listed below. The first step is to use the raw digital elevation data (DEM) to produce different versions of the topographic wetness index (TWI) for the study region (Calculating TWI). Then, these TWI output files are processed, along with soil moisture (volumetric water content or VWC) time series data from a number of sensors located within the study region, to create analysis-ready data objects (Processing TWI and VWC). Next, models are fit relating TWI to soil moisture (Model fitting) and results are plotted (Visualizing main results). A number of additional analyses were also done (Additional analyses). Input data The DEM of the study region is archived in this dataset as SourceDem.zip. This contains the DEM of the study region (DEM1.sgrd) and associated auxiliary files all called DEM1.* with different extensions. In addition, the DEM is provided as a .tif file called USGS_one_meter_x39y400_AR_R6_WashingtonCO_2015.tif. The remaining data and code files are archived in the repository created with a GitHub release on 2022-10-11, twi-moisture-0.1.zip. The data are found in a subfolder called data. 2017_LoggerData_HEW.csv through 2021_HEW.csv: Soil moisture (VWC) logger data for each year 2017-2021 (5 files total). 2882174.csv: weather data from a nearby station. DryPeriods2017-2021.csv: starting and ending days for dry periods 2017-2021. LoggerLocations.csv: Geographic locations and metadata for each VWC logger. Logger_Locations_TWI_2017-2021.xlsx: 546 topographic wetness indexes calculated at each VWC logger _location. note: This is intermediate input created in the first step of the pipeline. Code pipeline To reproduce the analysis in the manuscript run these scripts in the following order. The scripts are all found in the root directory of the repository. See the manuscript for more details on the methods. Calculating TWI TerrainAnalysis.R: Taking the DEM file as input, calculates 546 different topgraphic wetness indexes using a variety of different algorithms. Each algorithm is run multiple times with different input parameters, as described in more detail in the manuscript. After performing this step, it is necessary to use the SAGA-GIS GUI to extract the TWI values for each of the sensor locations. The output generated in this way is included in this repository as Logger_Locations_TWI_2017-2021.xlsx. Therefore it is not necessary to rerun this step of the analysis but the code is provided for completeness. Processing TWI and VWC read_process_data.R: Takes raw TWI and moisture data files and processes them into analysis-ready format, saving the results as CSV. qc_avg_moisture.R: Does additional quality control on the moisture data and averages it across different time periods. Model fitting Models were fit regressing soil moisture (average VWC for a certain time period) against a TWI index, with and without soil depth as a covariate. In each case, for both the model without depth and the model with depth, prediction performance was calculated with and without spatially-blocked cross-validation. Where cross validation wasn't used, we simply used the predictions from the model fit to all the data. fit_combos.R: Models were fit to each combination of soil moisture averaged over 57 months (all months from April 2017-December 2021) and 546 TWI indexes. In addition models were fit to soil moisture averaged over years, and to the grand mean across the full study period. fit_dryperiods.R: Models were fit to soil moisture averaged over previously identified dry periods within the study period (each 1 or 2 weeks in length), again for each of the 546 indexes. fit_summer.R: Models were fit to the soil moisture average for the months of June-September for each of the five years, again for each of the 546 indexes. Visualizing main results Preliminary visualization of results was done in a series of RMarkdown notebooks. All the notebooks follow the same general format, plotting model performance (observed-predicted correlation) across different combinations of time period and characteristics of the TWI indexes being compared. The indexes are grouped by SWI versus TWI, DEM filter used, flow algorithm, and any other parameters that varied. The notebooks show the model performance metrics with and without the soil depth covariate, and with and without spatially-blocked cross-validation. Crossing those two factors, there are four values for model performance for each combination of time period and TWI index presented. performance_plots_bymonth.Rmd: Using the results from the models fit to each month of data separately, prediction performance was averaged by month across the five years of data to show within-year trends. performance_plots_byyear.Rmd: Using the results from the models fit to each month of data separately, prediction performance was averaged by year to show trends across multiple years. performance_plots_dry_periods.Rmd: Prediction performance was presented for the models fit to the previously identified dry periods. performance_plots_summer.Rmd: Prediction performance was presented for the models fit to the June-September moisture averages. Additional analyses Some additional analyses were done that may not be published in the final manuscript but which are included here for completeness. 2019dryperiod.Rmd: analysis, done separately for each day, of a specific dry period in 2019. alldryperiodsbyday.Rmd: analysis, done separately for each day, of the same dry periods discussed above. best_indices.R: after fitting models, this script was used to quickly identify some of the best-performing indexes for closer scrutiny. wateryearfigs.R: exploratory figures showing median and quantile interval of VWC for sensors in low and high TWI locations for each water year. Resources in this dataset:Resource Title: Digital elevation model of study region. File Name: SourceDEM.zipResource Description: .zip archive containing digital elevation model files for the study region. See dataset description for more details.Resource Title: twi-moisture-0.1: Archived git repository containing all other necessary data and code . File Name: twi-moisture-0.1.zipResource Description: .zip archive containing all data and code, other than the digital elevation model archived as a separate file. This file was generated by a GitHub release made on 2022-10-11 of the git repository hosted at https://github.com/qdread/twi-moisture (private repository). See dataset description and README file contained within this archive for more details.

Global Surface Summary of the Day is derived from The Integrated Surface Hourly (ISH) dataset. The ISH dataset includes global data obtained from the USAF Climatology Center, located in the Federal Climate Complex with NCDC. The latest daily summary data are normally available 1-2 days after the date-time of the observations used in the daily summaries. The online data files begin with 1929 and are at the time of this writing at the Version 8 software level. Over 9000 stations' data are typically available. The daily elements included in the dataset (as available from each station) are: Mean temperature (.1 Fahrenheit) Mean dew point (.1 Fahrenheit) Mean sea level pressure (.1 mb) Mean station pressure (.1 mb) Mean visibility (.1 miles) Mean wind speed (.1 knots) Maximum sustained wind speed (.1 knots) Maximum wind gust (.1 knots) Maximum temperature (.1 Fahrenheit) Minimum temperature (.1 Fahrenheit) Precipitation amount (.01 inches) Snow depth (.1 inches) Indicator for occurrence of: Fog, Rain or Drizzle, Snow or Ice Pellets, Hail, Thunder, Tornado/Funnel Cloud Global summary of day data for 18 surface meteorological elements are derived from the synoptic/hourly observations contained in USAF DATSAV3 Surface data and Federal Climate Complex Integrated Surface Hourly (ISH). Historical data are generally available for 1929 to the present, with data from 1973 to the present being the most complete. For some periods, one or more countries' data may not be available due to data restrictions or communications problems. In deriving the summary of day data, a minimum of 4 observations for the day must be present (allows for stations which report 4 synoptic observations/day). Since the data are converted to constant units (e.g, knots), slight rounding error from the originally reported values may occur (e.g, 9.9 instead of 10.0). The mean daily values described below are based on the hours of operation for the station. For some stations/countries, the visibility will sometimes 'cluster' around a value (such as 10 miles) due to the practice of not reporting visibilities greater than certain distances. The daily extremes and totals--maximum wind gust, precipitation amount, and snow depth--will only appear if the station reports the data sufficiently to provide a valid value. Therefore, these three elements will appear less frequently than other values. Also, these elements are derived from the stations' reports during the day, and may comprise a 24-hour period which includes a portion of the previous day. The data are reported and summarized based on Greenwich Mean Time (GMT, 0000Z - 2359Z) since the original synoptic/hourly data are reported and based on GMT.

About Cyclistic

Cyclistic is a bike-share program that features more than 5,800 bicycles and 600 docking stations. They offer making bike-share more inclusive to people with disabilities and riders who can’t use a standard two-wheeled bike. In 2016, Cyclistic launched a successful bike-share offering. Since then, the program has grown to a fleet of 5,824 bicycles that are geotracked and locked into a network of 692 stations across Chicago. The bikes can be unlocked from one station and returned to any other station in the system anytime.

Problem Statement

The target/aim of marketing team is to convert casual riders into annual riders. In order to convert the causal riders into annual members need to understand the behavior of the users that how the annual members are using this service differently than causal riders. Need to understand how often this service is being used by annual members and casual riders.

Solution

For the analysis of this project, we picked/chose Excel with the mutual consent of our team to show our work. To help with our analysis we started with Ask, then we prepare our data according to what client was asking to provide then we process the data to make it clean, organize and easy to accessible and at the end we analyze that data to get the results.

As per the requirement of our client, they wanted to increase the number of their annual members. To increase their annual members, they wanted to know How do annual members and casual riders use Cyclistic bike differently?

After having company’s requirement now, it was the time to Prepare and Process the data. For this analysis we been told to use only previous 12 months of Cyclistic trip data. The data has been made available online by Motivational International Inc. we checked the integrity and credibility of data by making sure that online source is safe and secure through which the data is available to use.

While preparing the data, we started with downloading the files on our machine. We saved the files and unzip them. Then we created the subfolders for the .csv and the .xls sheets. Before further analysis we cleaned the data. We used Filter option on our required columns to see if there are any NULLS or any data that it supposed to be not here.

While cleaning the data in some of the monthly files we found that start_at and end_at columns had the custom format of mm: ss.0. For consistency with all other spreadsheets we changed the custom format to m/d/yy h:mm. We also found that some spreadsheets had the data from other months but after further analysis we figured it out that the ride was starting in that month and ending in the next month so that data supposed to belong from that worksheet.

After cleaning the data, we created 2 new columns in each worksheet to perform our calculations. To perform our calculations, we made 2 new columns and named them: a) ride_length
b) day_of _week

To create ride_length column we used Subtraction Formula by choosing stareted_at and ended_at columns. That gave us the ride length of each ride for everyday of the month. To create day_of_week we used WEEKDAY command. After cleaning the data on monthly basis, it was the time to merge all 12 months into a single spreadsheet. After merging the whole data into a new sheet, it was time to Analyze! Before analyzing our team made sure one more time that the data is properly organize, formatted and there is no error or bug in our data to get the correct results. We made sure on more time that all the Formatting are correct.To analyze the data we ran few calculations to get a better sense of the data layout that we were using. We calculated: a) mean of ride_length b) max of ride_length c) mode of day_of_week

To find out mean of ride_length, we used Average Formula, to get an estimate/ overview of how long rides usually last. By doing Max calculation we found out the longest ride length. Last but not the least mode function we calculate the most frequent day of the week when riders were using that service.

To Support the requirement/ question that been asked by our client to identify the trends and relationship we made a Pivot Table in Excel so that we can show/ present our work/ insights/ results in an easy way to the client. By using Pivot Table its clearer to see the trend that annual members are using this service more than the casual riders and it’s also giving the good picture of the relation that how often annual members are using this service. By using the Pivot Table, we analyzed that total number of rides for annual members are more than the causal riders. On the basis of our analysis, we found out that the average length of ride is more for casual riders than the annual members, it means that casual members are riding for longer period of time than the annual members. But annual members are using more often than casual ri...

Provides fare premiums for airports in the top 1,000 city pairs, and demonstrates the impact of low-fare service and hub domination on fare levels. All records are aggregated as directionless city pair markets. Air traffic in each direction is combined. https://www.transportation.gov/policy/aviation-policy/competition-data-analysis/research-reports

Abstract

The harmonized data set on health, created and published by the ERF, is a subset of Iraq Household Socio Economic Survey (IHSES) 2012. It was derived from the household, individual and health modules, collected in the context of the above mentioned survey. The sample was then used to create a harmonized health survey, comparable with the Iraq Household Socio Economic Survey (IHSES) 2007 micro data set.

----> Overview of the Iraq Household Socio Economic Survey (IHSES) 2012:

Iraq is considered a leader in household expenditure and income surveys where the first was conducted in 1946 followed by surveys in 1954 and 1961. After the establishment of Central Statistical Organization, household expenditure and income surveys were carried out every 3-5 years in (1971/ 1972, 1976, 1979, 1984/ 1985, 1988, 1993, 2002 / 2007). Implementing the cooperation between CSO and WB, Central Statistical Organization (CSO) and Kurdistan Region Statistics Office (KRSO) launched fieldwork on IHSES on 1/1/2012. The survey was carried out over a full year covering all governorates including those in Kurdistan Region.

The survey has six main objectives. These objectives are:

1. Provide data for poverty analysis and measurement and monitor, evaluate and update the implementation Poverty Reduction National Strategy issued in 2009.
2. Provide comprehensive data system to assess household social and economic conditions and prepare the indicators related to the human development.
3. Provide data that meet the needs and requirements of national accounts.
4. Provide detailed indicators on consumption expenditure that serve making decision related to production, consumption, export and import.
5. Provide detailed indicators on the sources of households and individuals income.
6. Provide data necessary for formulation of a new consumer price index number.

The raw survey data provided by the Statistical Office were then harmonized by the Economic Research Forum, to create a comparable version with the 2006/2007 Household Socio Economic Survey in Iraq. Harmonization at this stage only included unifying variables' names, labels and some definitions. See: Iraq 2007 & 2012- Variables Mapping & Availability Matrix.pdf provided in the external resources for further information on the mapping of the original variables on the harmonized ones, in addition to more indications on the variables' availability in both survey years and relevant comments.

Geographic coverage

National coverage: Covering a sample of urban, rural and metropolitan areas in all the governorates including those in Kurdistan Region.

Analysis unit

1- Household/family. 2- Individual/person.

Universe

The survey was carried out over a full year covering all governorates including those in Kurdistan Region.

Kind of data

Sample survey data [ssd]

Sampling procedure

----> Design:

Sample size was (25488) household for the whole Iraq, 216 households for each district of 118 districts, 2832 clusters each of which includes 9 households distributed on districts and governorates for rural and urban.

----> Sample frame:

Listing and numbering results of 2009-2010 Population and Housing Survey were adopted in all the governorates including Kurdistan Region as a frame to select households, the sample was selected in two stages: Stage 1: Primary sampling unit (blocks) within each stratum (district) for urban and rural were systematically selected with probability proportional to size to reach 2832 units (cluster). Stage two: 9 households from each primary sampling unit were selected to create a cluster, thus the sample size of total survey clusters was 25488 households distributed on the governorates, 216 households in each district.

----> Sampling Stages:

In each district, the sample was selected in two stages: Stage 1: based on 2010 listing and numbering frame 24 sample points were selected within each stratum through systematic sampling with probability proportional to size, in addition to the implicit breakdown urban and rural and geographic breakdown (sub-district, quarter, street, county, village and block). Stage 2: Using households as secondary sampling units, 9 households were selected from each sample point using systematic equal probability sampling. Sampling frames of each stages can be developed based on 2010 building listing and numbering without updating household lists. In some small districts, random selection processes of primary sampling may lead to select less than 24 units therefore a sampling unit is selected more than once , the selection may reach two cluster or more from the same enumeration unit when it is necessary.

Mode of data collection

Face-to-face [f2f]

Research instrument

----> Preparation:

The questionnaire of 2006 survey was adopted in designing the questionnaire of 2012 survey on which many revisions were made. Two rounds of pre-test were carried out. Revision were made based on the feedback of field work team, World Bank consultants and others, other revisions were made before final version was implemented in a pilot survey in September 2011. After the pilot survey implemented, other revisions were made in based on the challenges and feedbacks emerged during the implementation to implement the final version in the actual survey.

----> Questionnaire Parts:

The questionnaire consists of four parts each with several sections: Part 1: Socio – Economic Data: - Section 1: Household Roster - Section 2: Emigration - Section 3: Food Rations - Section 4: housing - Section 5: education - Section 6: health - Section 7: Physical measurements - Section 8: job seeking and previous job

Part 2: Monthly, Quarterly and Annual Expenditures: - Section 9: Expenditures on Non – Food Commodities and Services (past 30 days). - Section 10 : Expenditures on Non – Food Commodities and Services (past 90 days). - Section 11: Expenditures on Non – Food Commodities and Services (past 12 months). - Section 12: Expenditures on Non-food Frequent Food Stuff and Commodities (7 days). - Section 12, Table 1: Meals Had Within the Residential Unit. - Section 12, table 2: Number of Persons Participate in the Meals within Household Expenditure Other Than its Members.

Part 3: Income and Other Data: - Section 13: Job - Section 14: paid jobs - Section 15: Agriculture, forestry and fishing - Section 16: Household non – agricultural projects - Section 17: Income from ownership and transfers - Section 18: Durable goods - Section 19: Loans, advances and subsidies - Section 20: Shocks and strategy of dealing in the households - Section 21: Time use - Section 22: Justice - Section 23: Satisfaction in life - Section 24: Food consumption during past 7 days

Part 4: Diary of Daily Expenditures: Diary of expenditure is an essential component of this survey. It is left at the household to record all the daily purchases such as expenditures on food and frequent non-food items such as gasoline, newspapers…etc. during 7 days. Two pages were allocated for recording the expenditures of each day, thus the roster will be consists of 14 pages.

Cleaning operations

----> Raw Data:

Data Editing and Processing: To ensure accuracy and consistency, the data were edited at the following stages: 1. Interviewer: Checks all answers on the household questionnaire, confirming that they are clear and correct. 2. Local Supervisor: Checks to make sure that questions has been correctly completed. 3. Statistical analysis: After exporting data files from excel to SPSS, the Statistical Analysis Unit uses program commands to identify irregular or non-logical values in addition to auditing some variables. 4. World Bank consultants in coordination with the CSO data management team: the World Bank technical consultants use additional programs in SPSS and STAT to examine and correct remaining inconsistencies within the data files. The software detects errors by analyzing questionnaire items according to the expected parameter for each variable.

----> Harmonized Data:

• The SPSS package is used to harmonize the Iraq Household Socio Economic Survey (IHSES) 2007 with Iraq Household Socio Economic Survey (IHSES) 2012.
• The harmonization process starts with raw data files received from the Statistical Office.
• A program is generated for each dataset to create harmonized variables.
• Data is saved on the household and individual level, in SPSS and then converted to STATA, to be disseminated.

Response rate

Iraq Household Socio Economic Survey (IHSES) reached a total of 25488 households. Number of households refused to response was 305, response rate was 98.6%. The highest interview rates were in Ninevah and Muthanna (100%) while the lowest rates were in Sulaimaniya (92%).

The Fermi GBM Daily Data database table contains entries for each day for which GBM data has been processed. The daily data products consist of GBM data that are produced continuously regardless of whether a burst occurred. Thus these products are the count rates from all detectors, the monitoring of the detector calibrations (e.g., the position of the 511 keV line), and the spacecraft position and orientation. Some days may also have event lists known as time-tagged event (TTE) files associated with them. These TTE files have the same format as those produced for bursts. Due to the large data volume associated with TTE files, only certain portions of the day considered of scientific interest to the instrument team will have TTE data. The underlying Level 0 data arrive continuously with each Ku band downlink. However, the GBM Instrument Operations Center (GIOC) will form FITS files of the resulting Level 1 data covering an entire calendar day (UTC); these daily files are then sent to the FSSC. Consequently, the data latency is about one day: the first bit from the beginning of a calendar day may arrive a few hours after the day began while the last bit will be processed and added to the data product file a few hours after the day ended. These data products may be sent to the FSSC file by file as they are produced, not necessarily in one package for a given day. Note that the data may include events from slightly before and slightly after the day official boundaries, which will be reflected in the start and stop times in the table. Consequently, some events may be listed in files for two consecutive days (e.g., at the end of one and the beginning of the next). Due to the continuous nature of GBM processing, new data files may arrive after the day has been included in Browse and reprocessed version may also arrive at any time. The reprocessed data will have the version number incremented (see file name conventions below). Browse will automatically download the latest versions of the data files. This database table was created by and is updated by the HEASARC based on information supplied by the Fermi Project. It is updated on a daily basis. The tte_flag parameter was added to the table in July 2010. This is a service provided by NASA HEASARC .

COVID-19 Data for Analysis and Machine Learning

There are lots of datasets online, more growing every day, to help us all get a handle on this pandemic. Here are just a few links to data we've found that students in ECE 657A, and anyone else who finds their way here, can play with and practice their machine learning skills. The main dataset is the COVID-19 dataset from John Hopkins university. This data is perfect for time series analysis and Recurrent Neural Networks, the final topic in the course. This dataset will be left public so anyone can see it but to join you must request the link from Prof. Crowley or be in the ECE 657A W20 course at the University of Waterloo.

For ECE 657A W20 Students

Your bonus grade for assignment 4 comes from creating a kernel from this dataset and writing up some useful analysis and publishing that notebook. You can do any kind of analysis you like but some good places to start are - Analysis: feature extraction and analysis of the data to look for patterns that aren't evident from the original features (this is hard for the simple spread/infection/death data since there aren't that many features) - Other Data: utilize any other datasets in your kernels by loading data about the countries themselves (population, density, wealthy etc.) or their responses to the situation. Tip: If you open a New Notebook related to this dataset you can easily add new data available on Kaggle and link that to you analysis. - HOW'S MY FLATTENING COVID19 DATASET - This dataset has a lot more files and includes a lot of what I was talking about, so if you produce good kernels there you can also count them for your asg4 grade. https://www.kaggle.com/howsmyflattening/covid19-challenges - Predict: make predictions about confirmed cases, deaths, recoveries or other metrics for the future. You can test you models by training on the past and predicting on the following days, then post a prediction for tomorrow or the next few days given ALL the data up to this point. Hopefully the datasets we've linked here will updated automatically so your kernels would update as well. - Create Tasks: you can make your own "Tasks" as part of this kaggle and propose your own solution to it. Then others can try solving it as well. - Groups: students can do this assignment either in the same groups they had for assignment 3 or individually.

Suggest other datasets

We're happy to add other relevant data to this Kaggle, in particular it would be great to integrate live data on the following: - Progression of each country/region/city in "days since X Level" such as Days since 100 confirmed cases, see the link for a great example such a dataset being plotted. I haven't see a live link to a csv of that data, but we could generate. - Mitigation Policies enacted by local governments in each city/region/country. These are dates when that region first enacted Level 1, 2, 3, 4 containment, or started encouraging social distancing or the date when they closed different levels of schools, pubs, restaurants etc. - The hidden positives: this would be a dataset, or method for estimating, as described by Emtiyaz Khan in this twitter thread. The idea is, how many unreported or unconfirmed cases are there in any region, and can we build an estimate of that number using other regions with widespread testing as a baseline and the death rates which are like an observation of a process with a hidden variable or true infection rate. - Paper discussing one way to compute this : https://cmmid.github.io/topics/covid19/severity/global_cfr_estimates.html

The global number of Facebook users was forecast to continuously increase between 2023 and 2027 by in total 391 million users (+14.36 percent). After the fourth consecutive increasing year, the Facebook user base is estimated to reach 3.1 billion users and therefore a new peak in 2027. Notably, the number of Facebook users was continuously increasing over the past years. User figures, shown here regarding the platform Facebook, have been estimated by taking into account company filings or press material, secondary research, app downloads and traffic data. They refer to the average monthly active users over the period and count multiple accounts by persons only once.The shown data are an excerpt of Statista's Key Market Indicators (KMI). The KMI are a collection of primary and secondary indicators on the macro-economic, demographic and technological environment in up to 150 countries and regions worldwide. All indicators are sourced from international and national statistical offices, trade associations and the trade press and they are processed to generate comparable data sets (see supplementary notes under details for more information).