This dataset contains two tables: creative_stats and removed_creative_stats. The creative_stats table contains information about advertisers that served ads in the European Economic Area or Turkey: their legal name, verification status, disclosed name, and location. It also includes ad specific information: impression ranges per region (including aggregate impressions for the European Economic Area), first shown and last shown dates, which criteria were used in audience selection, the format of the ad, the ad topic and whether the ad is funded by Google Ad Grants program. A link to the ad in the Google Ads Transparency Center is also provided. The removed_creative_stats table contains information about ads that served in the European Economic Area that Google removed: where and why they were removed and per-region information on when they served. The removed_creative_stats table also contains a link to the Google Ads Transparency Center for the removed ad. Data for both tables updates periodically and may be delayed from what appears on the Google Ads Transparency Center website. About BigQuery This data is hosted in Google BigQuery for users to easily query using SQL. Note that to use BigQuery, users must have a Google account and create a GCP project. This public dataset is included in BigQuery's 1TB/mo of free tier processing. Each user receives 1TB of free BigQuery processing every month, which can be used to run queries on this public dataset. Watch this short video to learn how to get started quickly using BigQuery to access public datasets. What is BigQuery . Download Dataset This public dataset is also hosted in Google Cloud Storage here and available free to use. Use this quick start guide to quickly learn how to access public datasets on Google Cloud Storage. We provide the raw data in JSON format, sharded across multiple files to support easier download of the large dataset. A README file which describes the data structure and our Terms of Service (also listed below) is included with the dataset. You can also download the results from a custom query. See here for options and instructions. Signed out users can download the full dataset by using the gCloud CLI. Follow the instructions here to download and install the gCloud CLI. To remove the login requirement, run "$ gcloud config set auth/disable_credentials True" To download the dataset, run "$ gcloud storage cp gs://ads-transparency-center/* . -R" This public dataset is hosted in Google BigQuery and is included in BigQuery's 1TB/mo of free tier processing. This means that each user receives 1TB of free BigQuery processing every month, which can be used to run queries on this public dataset. Watch this short video to learn how to get started quickly using BigQuery to access public datasets. What is BigQuery .

You can check the fields description in the documentation: current Keyword database: https://docs.dataforseo.com/v3/databases/google/keywords/?bash; Historical Keyword database: https://docs.dataforseo.com/v3/databases/google/history/keywords/?bash. You don’t have to download fresh data dumps in JSON or CSV – we can deliver data straight to your storage or database. We send terrabytes of data to dozens of customers every month using Amazon S3, Google Cloud Storage, Microsoft Azure Blob, Eleasticsearch, and Google Big Query. Let us know if you’d like to get your data to any other storage or database.

The AlphaFold Protein Structure Database is a collection of protein structure predictions made using the machine learning model AlphaFold. AlphaFold was developed by DeepMind , and this database was created in partnership with EMBL-EBI . For information on how to interpret, download and query the data, as well as on which proteins are included / excluded, and change log, please see our main dataset guide and FAQs . To interactively view individual entries or to download proteomes / Swiss-Prot please visit https://alphafold.ebi.ac.uk/ . The current release aims to cover most of the over 200M sequences in UniProt (a commonly used reference set of annotated proteins). The files provided for each entry include the structure plus two model confidence metrics (pLDDT and PAE). The files can be found in the Google Cloud Storage bucket gs://public-datasets-deepmind-alphafold-v4 with metadata in the BigQuery table bigquery-public-data.deepmind_alphafold.metadata . If you use this data, please cite: Jumper, J et al. Highly accurate protein structure prediction with AlphaFold. Nature (2021) Varadi, M et al. AlphaFold Protein Structure Database: massively expanding the structural coverage of protein-sequence space with high-accuracy models. Nucleic Acids Research (2021) This public dataset is hosted in Google Cloud Storage and is available free to use. Use this quick start guide to quickly learn how to access public datasets on Google Cloud Storage.

You can check the fields description in the documentation: regular SERP: https://docs.dataforseo.com/v3/databases/google/serp_regular/?bash; Advanced SERP: https://docs.dataforseo.com/v3/databases/google/serp_advanced/?bash; Historical SERP: https://docs.dataforseo.com/v3/databases/google/history/serp_advanced/?bash You don’t have to download fresh data dumps in JSON or CSV – we can deliver data straight to your storage or database. We send terrabytes of data to dozens of customers every month using Amazon S3, Google Cloud Storage, Microsoft Azure Blob, Eleasticsearch, and Google Big Query. Let us know if you’d like to get your data to any other storage or database.

The Structured Query Language (SQL) server transformation market is experiencing robust growth, projected to reach $15 million in 2025 and maintain a Compound Annual Growth Rate (CAGR) of 9.4% from 2025 to 2033. This expansion is fueled by several key drivers. The increasing adoption of cloud-based solutions and the rising demand for real-time data analytics are significantly impacting the market. Businesses are increasingly migrating their on-premise SQL servers to cloud platforms like AWS, Azure, and Google Cloud, driven by scalability, cost efficiency, and enhanced accessibility. Furthermore, the growing need for faster data processing and improved database performance is pushing organizations to adopt advanced SQL server technologies, including in-memory databases and distributed SQL solutions. The market is segmented by deployment model (cloud, on-premise), database type (relational, NoSQL), and industry vertical (finance, healthcare, retail). Major players like Oracle, IBM, Microsoft, and Amazon Web Services are actively investing in research and development, launching new products and services to solidify their market positions. Competitive pressures are driving innovation and pushing the market towards more efficient, scalable, and secure solutions. The restraining factors impacting the market include the complexities associated with migrating existing SQL servers to new platforms, the high initial investment required for cloud-based solutions, and security concerns related to data breaches. However, the long-term benefits of improved efficiency, scalability, and cost optimization are outweighing these challenges, leading to sustained market growth. The ongoing trend of big data adoption and the demand for advanced analytics are creating new opportunities for vendors. We anticipate that the market will see increased adoption of serverless SQL databases and the development of more sophisticated tools for data integration and management in the coming years. This will likely reshape the competitive landscape and accelerate the transformation of the SQL server market.

The Enterprise Search Platform market is booming, projected to reach $772.8 million in 2025 with an 11.6% CAGR. Discover key drivers, trends, and top companies shaping this rapidly evolving sector. Learn more about AI-powered search, cloud solutions, and the future of enterprise search.

Data Lake Query Engine Market Outlook

According to our latest research, the global Data Lake Query Engine market size reached USD 1.82 billion in 2024, reflecting robust momentum and increased enterprise adoption across various sectors. The industry is advancing at a CAGR of 23.7% from 2025 to 2033, with the market expected to hit USD 13.48 billion by 2033. This remarkable growth trajectory is primarily driven by the escalating need for real-time analytics, the exponential rise in data volumes, and the ongoing digital transformation initiatives across major verticals.

The primary growth factor fueling the Data Lake Query Engine market is the unprecedented surge in data generation from both structured and unstructured sources. Enterprises are increasingly seeking scalable, cost-effective solutions to store, manage, and analyze massive datasets, which traditional databases struggle to handle efficiently. Data lake query engines are emerging as the backbone for big data analytics, enabling organizations to derive actionable insights without the need for complex data movement or transformation. Furthermore, the integration of advanced technologies such as artificial intelligence, machine learning, and real-time analytics into data lake architectures is propelling the demand for sophisticated query engines that can seamlessly process diverse data types and formats.

Another significant driver is the growing adoption of cloud-based data lake solutions. As organizations migrate their workloads to the cloud for enhanced agility, scalability, and cost optimization, the demand for cloud-native query engines is witnessing exponential growth. Cloud deployment not only reduces infrastructure overheads but also accelerates time-to-insight, making it particularly attractive for enterprises with dynamic, fast-changing data requirements. The proliferation of multi-cloud and hybrid cloud strategies further amplifies the need for flexible query engines that can operate across disparate environments while maintaining data governance and security.

Additionally, the increasing emphasis on business intelligence and data-driven decision-making is shaping the evolution of the Data Lake Query Engine market. Companies across BFSI, healthcare, retail, and manufacturing are leveraging data lake architectures to democratize access to analytics, empowering business users and data scientists alike. The ability to perform ad-hoc queries, interactive analytics, and advanced reporting on petabyte-scale datasets is transforming how organizations extract value from their data assets. This trend is further reinforced by the emergence of self-service analytics platforms and the growing ecosystem of data integration and visualization tools.

From a regional perspective, North America continues to dominate the market, accounting for the largest revenue share due to the presence of leading technology providers, early cloud adoption, and a mature analytics landscape. However, Asia Pacific is emerging as the fastest-growing region, fueled by rapid digitalization, expanding IT infrastructure, and increasing investments in big data technologies by enterprises and governments. Europe is also witnessing substantial growth, particularly in sectors such as finance, healthcare, and manufacturing, where data compliance and regulatory requirements drive innovation in data management and analytics.

Component Analysis

The Data Lake Query Engine market is segmented by component into software and services, each playing a pivotal role in the overall ecosystem. The software segment constitutes the core of the market, encompassing query engines designed to provide high-performance, low-latency access to large-scale data lakes. These solutions are continuously evolving to support a wider range of data formats, advanced analytics capabilities, and seamless integration with popular data lake platforms such as Amazon S3, Azure Data Lake, and Google Cloud Storage. The growing demand for open-source and commercial query engines, including Presto,

You can check the fields description in the documentation: current Full database: https://docs.dataforseo.com/v3/databases/google/full/?bash; Historical Full database: https://docs.dataforseo.com/v3/databases/google/history/full/?bash.

Full Google Database is a combination of the Advanced Google SERP Database and Google Keyword Database.

Google SERP Database offers millions of SERPs collected in 67 regions with most of Google’s advanced SERP features, including featured snippets, knowledge graphs, people also ask sections, top stories, and more.

Google Keyword Database encompasses billions of search terms enriched with related Google Ads data: search volume trends, CPC, competition, and more.

This database is available in JSON format only.

You don’t have to download fresh data dumps in JSON – we can deliver data straight to your storage or database. We send terrabytes of data to dozens of customers every month using Amazon S3, Google Cloud Storage, Microsoft Azure Blob, Eleasticsearch, and Google Big Query. Let us know if you’d like to get your data to any other storage or database.

In 2023, Google Cloud revenue amounted to 43.22 billion U.S. dollars, accounting for over 10 percent of Google's total revenues. The company's Cloud segment primarily generates revenue through the Google Cloud Platform (GCP), which offers a suite of cloud computing services running on Google infrastructure. Competition on the cloud In recent years, GCP has seen tremendous growth with increasing demand for cloud computing to keep pace with digital transformation. The development is fueled by the increasing demand for cloud-based services, dependency on cloud infrastructure for scalability, and the growing popularity of microservices. Tough competition from the largest cloud providers Microsoft Azure and Amazon Web Services has forced all cloud providers to continually innovate and offer new services to gain or retain existing customers. AI on the cloud The growing demand for new technologies like artificial intelligence (AI) will further fuel the demand for cloud infrastructure. AI development, deployment, and management for various applications are more straightforward on the cloud, with the availability of tools for data storage, data processing, and easy integration of machine learning in AI models. One of the critical factors that would drive the development of AI on the cloud is the providers' offer of a pay-as-you-go pricing model.

Query Performance Optimization Market Outlook

According to our latest research, the global Query Performance Optimization market size reached USD 3.42 billion in 2024, underpinned by growing digital transformation initiatives and the proliferation of data-driven business models across industries. The market is expected to expand at a robust CAGR of 13.6% from 2025 to 2033, reaching a projected value of USD 10.61 billion by 2033. The primary growth factor driving this surge is the exponential increase in data volumes and the corresponding need for real-time, high-performance data querying to support analytics, business intelligence, and mission-critical applications.

A key growth factor for the Query Performance Optimization market is the relentless expansion of big data and advanced analytics across multiple sectors. As enterprises accumulate massive volumes of structured and unstructured data, the demand for efficient query performance optimization solutions has become paramount. Organizations are increasingly leveraging complex analytical queries to extract actionable insights, necessitating advanced optimization tools that can minimize latency, maximize throughput, and ensure data consistency. The shift towards data-driven decision-making is compelling businesses to invest in robust query optimization technologies to maintain a competitive edge, streamline operations, and enhance customer experiences.

Another significant driver is the rapid adoption of cloud-based infrastructure and services. The migration to cloud platforms such as AWS, Microsoft Azure, and Google Cloud has introduced new complexities in data management, including distributed architectures and multi-cloud environments. These trends have heightened the need for sophisticated query optimization solutions that can seamlessly operate across hybrid and cloud-native ecosystems. Cloud deployment models offer scalability, flexibility, and cost-effectiveness, enabling organizations to optimize query performance dynamically as workloads fluctuate. This has led to a surge in demand for solutions that can deliver high performance and reliability in increasingly complex cloud environments.

Furthermore, the growing emphasis on digital transformation within industries such as BFSI, healthcare, retail, and manufacturing is fueling market expansion. Enterprises in these sectors are adopting advanced business intelligence and analytics platforms to drive innovation, improve operational efficiency, and deliver personalized services. Query performance optimization is critical to ensuring that these platforms deliver timely and accurate insights. Additionally, regulatory requirements around data governance and compliance are prompting organizations to invest in solutions that guarantee data integrity and optimize query execution, further propelling market growth.

Regionally, North America holds the largest share of the Query Performance Optimization market in 2024, owing to the early adoption of advanced IT solutions, a robust presence of key market players, and significant investments in digital infrastructure. However, the Asia Pacific region is poised for the fastest growth over the forecast period, driven by rapid digitalization, increasing cloud adoption, and a burgeoning startup ecosystem. Europe and Latin America are also experiencing steady growth, supported by expanding enterprise IT spending and the rising importance of data analytics in business operations. The Middle East & Africa region is gradually catching up, with digital transformation initiatives gaining momentum across key sectors.

Component Analysis

The Component segment of the Query Performance Optimization market is bifurcated into software and services. Software solutions dominate the market, accounting for a substantial portion of the overall revenue in 2024. This dominance is attributed to the continuous advancements in query optimization algorithms, integration with modern database platforms, and the growing need for automation in d

Discover the explosive growth of the serverless cloud computing market! Our in-depth analysis reveals a 23.17% CAGR, driven by increased adoption across industries like BFSI and retail. Learn about key trends, leading companies (AWS, Microsoft, Google), and future projections for this transformative technology. Recent developments include: September 2022 - Launch of Log Analytics powered by Big Query by Google Cloud Logging. Users of the feature can perform analytics on logs using the power of BQ within Cloud Logging. To begin using Log Analytics, you can change your current Log Buckets. Also, to consume data, simple data pipeline designs are optional., November 2022 - ModelScope, an open-source Model-as-a-Service (MaaS) platform from Alibaba Cloud, was introduced with big pre-trained models and hundreds of AI models for researchers and developers worldwide. To further assist clients in achieving business innovation through cloud technologies, the cloud provider provided various serverless database products and enhanced its integrated data analytics and intelligent computing platform.. Key drivers for this market are: Growth in Enhanced Scalability, Decreased in Time-To-Market Along with Reduced Operational Cost, Proliferation of the Microservices Architecture Across Organization's Business Model; Increase in demand of Professional services globally to drive the market. Potential restraints include: Growth in Enhanced Scalability, Decreased in Time-To-Market Along with Reduced Operational Cost, Proliferation of the Microservices Architecture Across Organization's Business Model; Increase in demand of Professional services globally to drive the market. Notable trends are: Professional Services are Expected to Grow at a Significant Rate.

The Metropolitan Museum of Art, better known as the Met, provides a public domain dataset with over 200,000 objects including metadata and images. In early 2017, the Met debuted their Open Access policy to make part of their collection freely available for unrestricted use under the Creative Commons Zero designation and their own terms and conditions. This dataset provides a new view to one of the world’s premier collections of fine art. The data includes both image in Google Cloud Storage, and associated structured data in two BigQuery two tables, objects and images (1:N). Locations to images on both The Met’s website and in Google Cloud Storage are available in the BigQuery table. The meta data for this public dataset is hosted in Google BigQuery and is included in BigQuery's 1TB/mo of free tier processing. This means that each user receives 1TB of free BigQuery processing every month, which can be used to run queries on this public dataset. Watch this short video to learn how to get started quickly using BigQuery to access public datasets. What is BigQuery . The image data for this public dataset is hosted in Google Cloud Storage and available free to use. Use this quick start guide to quickly learn how to access public datasets on Google Cloud Storage.

The Analytics Query Accelerator (AQA) market is experiencing robust growth, driven by the increasing demand for real-time insights from massive datasets across various industries. The market, estimated at $15 billion in 2025, is projected to achieve a Compound Annual Growth Rate (CAGR) of 20% from 2025 to 2033, reaching an estimated $70 billion by 2033. This expansion is fueled by several key factors. Firstly, the proliferation of big data and the need for rapid data analysis across sectors like finance, healthcare, and e-commerce are creating significant demand. Secondly, advancements in cloud computing and distributed database technologies are enabling faster query processing and improved performance of AQAs. Finally, the rising adoption of advanced analytics techniques such as machine learning and artificial intelligence is further driving the need for efficient query acceleration solutions. Key players like Google, Amazon, Snowflake, Microsoft, Databricks, Teradata, and Cloudera are actively competing in this rapidly evolving landscape, investing heavily in R&D and strategic partnerships to maintain market leadership. The growth trajectory of the AQA market is further shaped by emerging trends such as the increasing adoption of serverless computing and the expansion of edge analytics. However, challenges remain, including the complexity of implementing and managing AQA solutions, the need for skilled professionals, and concerns related to data security and privacy. Despite these restraints, the long-term outlook for the AQA market remains exceptionally positive, fueled by continuous technological innovations and the ever-increasing reliance on data-driven decision-making across all industries. The market segmentation is likely diversified across various deployment models (cloud, on-premise), data types (structured, unstructured), and industry verticals. This diverse landscape presents numerous opportunities for both established players and emerging companies to capture market share.

Note: the below description is as-is from the original dataset New York City Taxi Trip Duration - thank you, credits to the original author for creating it. The data preparatory kernel can be found here: ChaiEDA: NYC Taxi Trip Duration (data-prep).

The competition dataset is based on the 2016 NYC Yellow Cab trip record data made available in Big Query on Google Cloud Platform. The data was originally published by the NYC Taxi and Limousine Commission (TLC). The data was sampled and cleaned for the purposes of this playground competition. Based on individual trip attributes, participants should predict the duration of each trip in the test set.

File descriptions

• train.csv - the training set (contains 1458644 trip records)
• test.csv - the testing set (contains 625134 trip records)
• sample_submission.csv - a sample submission file in the correct format
• nyc_additional_info.csv - additional New York related location details from https://www.kaggle.com/dgomonov/new-york-city-airbnb-open-data
• train_extended.csv - above training set extended with new fields (see below for further details on additional fields)
• test_extended.csv - above test set extended with new fields (see below for further details on additional fields)
• train_test_extended.csv - above training and test sets (combined) extended with new fields (see below for further details on additional fields)

Data fields

• id - a unique identifier for each trip
• vendor_id - a code indicating the provider associated with the trip record
• pickup_datetime - date and time when the meter was engaged
• dropoff_datetime - date and time when the meter was disengaged
• passenger_count - the number of passengers in the vehicle (driver entered value)
• pickup_longitude - the longitude where the meter was engaged
• pickup_latitude - the latitude where the meter was engaged
• dropoff_longitude - the longitude where the meter was disengaged
• dropoff_latitude - the latitude where the meter was disengaged
• store_and_fwd_flag - This flag indicates whether the trip record was held in the vehicle's memory before sending to the vendor because the vehicle did not have a connection to the server - Y=store and forward; N=not a store and forward trip
• trip_duration - duration of the trip in seconds

Extended fields (additional fields to the above) - pickup_district - the name of the NY district corresponding to the latitude and longitude of the pickup location - pickup_neighbourhood - the name of the NY neighbourhood corresponding to the latitude and longitude of the pickup location - dropoff_district - the name of the NY district corresponding to the latitude and longitude of the dropoff location - dropoff_neighbourhood - the name of the NY neighbourhood corresponding to the latitude and longitude of the dropoff location - pickup_geonumber - latitude and longitude of the pickup location combined into a single number - dropoff_geonumber - latitude and longitude of the dropoff location combined into a single number - pickup_hour - the hour of the day when a pickup was made for a trip/ride - day_period - the name of the period of the day i.e. morning, evening, etc.. based on the pickup date/time - day_name - day name i.e. Monday, Tuesday, etc... - weekday_or_weekend - if the day is a weekday or weekend based on the pickup date/time - regular_day_or_holiday - if the day is a regular day or holiday based on the pickup date/time - month - calendar month based on the pickup date/time - year - calendar year based on the pickup date/time - season - season based on the pickup date/time

Fields in nyc_additional_info.csv

• name - name of the area or building complex
• district - NY district in which the area or building complex is situated
• neighbourhood - NY neighbourhood in which the area or building complex is situated
• latitude - latitude of the area or building complex
• longitude - longitude of the area or building complex

Disclaimer: The decision was made to not remove dropoff coordinates from the dataset order to provide an expanded set of variables to use in Kernel

The global Cloud-Based Data Analytics Platform market is poised for significant expansion, projected to reach a substantial market size of $150 billion by 2025, exhibiting a robust Compound Annual Growth Rate (CAGR) of 18% throughout the forecast period of 2025-2033. This impressive growth trajectory is fueled by an increasing reliance on data-driven decision-making across all industries. Key drivers include the escalating volume and complexity of data, the growing demand for real-time insights to gain a competitive edge, and the inherent scalability and cost-effectiveness offered by cloud platforms compared to on-premise solutions. Businesses are increasingly leveraging these platforms to extract actionable intelligence from their data, enabling them to optimize operations, enhance customer experiences, and identify new revenue streams. The democratization of data analytics tools, with user-friendly interfaces and advanced AI/ML capabilities, is further accelerating adoption among small and medium-sized enterprises, broadening the market's reach and impact. The market landscape is characterized by a dynamic interplay of technological advancements and evolving business needs. Major trends include the proliferation of hybrid and multi-cloud strategies, offering organizations greater flexibility and control over their data. Advancements in AI and machine learning are deeply integrated into these platforms, enabling more sophisticated predictive analytics, natural language processing for query simplification, and automated insights. The emphasis on data governance, security, and compliance in cloud environments is also a critical consideration, with vendors investing heavily in robust security features. While the market experiences immense growth, potential restraints such as data privacy concerns, vendor lock-in anxieties, and the need for skilled personnel to manage and interpret complex data sets present challenges. However, the overwhelming benefits of enhanced agility, improved collaboration, and reduced IT infrastructure costs continue to drive strong market momentum, with platforms like those offered by industry leaders such as Amazon, Google, Microsoft, and Snowflake dominating the competitive arena. This comprehensive report provides an in-depth analysis of the global Cloud-Based Data Analytics Platform market, forecasting its trajectory from 2019 to 2033, with a base year of 2025. The study delves into the market's intricate dynamics, exploring its growth drivers, challenges, and emerging trends, while also providing valuable insights into its competitive landscape and key regional contributions. The estimated market size is expected to reach $XX million by 2025, with significant growth projected during the forecast period.

IoTeX is a decentralized crypto system, a new generation of blockchain platform for the development of the Internet of things (IoT). The project team is sure that the users do not have such an application that would motivate to implement the technology of the Internet of things in life. And while this will not be created, people will not have the desire to spend money and time on IoT. The developers of IoTeX decided to implement not the application itself, but the platform for creation. It is through the platform that innovative steps in the space of the Internet of things will be encouraged. Learn more... This dataset is one of many crypto datasets that are available within the Google Cloud Public Datasets . As with other Google Cloud public datasets, you can query this dataset for free, up to 1TB/month of free processing, every month. Watch this short video to learn how to get started with the public datasets. Want to know how the data from these blockchains were brought into BigQuery, and learn how to analyze the data? Saiba mais

This is a Google Cloud Platform image with data from the Distributed Observatory Search project. This dataset is made available to University Staff and Students only due to rights and technical issues.

In fiscal year 2024, Alphabet's revenue by segment (products & services) are as follows: Google Cloud: $43.23 B, Google Network: $30.36 B, Google Search & Other: $198.08 B, Google Subscriptions, Platforms, And Devices: $40.34 B, Other Bets: $1.65 B, YouTube Ads: $36.15 B.

In the fourth quarter of 2024, the most popular vendor in the cloud infrastructure services market, Amazon Web Services (AWS), controlled ** percent of the entire market. Microsoft Azure takes second place with ** percent market share, followed by Google Cloud with ** percent market share. Together, these three cloud vendors account for ** percent of total spend in the fourth quarter of 2024. Organizations use cloud services from these vendors for machine learning, data analytics, cloud native development, application migration, and other services. AWS Services Amazon Web Services is used by many organizations because it offers a wide variety of services and products to its customers that improve business agility while being secure and reliable. One of AWS’s most used services is Amazon EC2, which lets customers create virtual machines for their strategic projects while spending less time on maintaining servers. Another important service is Amazon Simple Storage Service (S3), which offers a secure file storage service. In addition, Amazon also offers security, website infrastructure management, and identity and access management solutions. Cloud infrastructure services Vendors offering cloud services to a global customer base do so through different types of cloud computing, which include infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS). Further, there are different cloud computing deployment models available for customers, namely private cloud and public cloud, as well as community cloud and hybrid cloud. A cloud deployment model is defined based on the location where the deployment resides, and who has access to and control over the infrastructure.

This resource contains Jupyter Python notebooks which are intended to be used to learn about the U.S. National Water Model (NWM). These notebooks explore NWM forecasts in various ways. NWM Notebooks 1, 2, and 3, access NWM forecasts directly from the NOAA NOMADS file sharing system. Notebook 4 accesses NWM forecasts from Google Cloud Platform (GCP) storage in addition to NOMADS. A brief summary of what each notebook does is included below:

Notebook 1 (NWM1_Visualization) focuses on visualization. It includes functions for downloading and extracting time series forecasts for any of the 2.7 million stream reaches of the U.S. NWM. It also demonstrates ways to visualize forecasts using Python packages like matplotlib.

Notebook 2 (NWM2_Xarray) explores methods for slicing and dicing NWM NetCDF files using the python library, XArray.

Notebook 3 (NWM3_Subsetting) is focused on subsetting NWM forecasts and NetCDF files for specified reaches and exporting NWM forecast data to CSV files.

Notebook 4 (NWM4_Hydrotools) uses Hydrotools, a new suite of tools for evaluating NWM data, to retrieve NWM forecasts both from NOMADS and from Google Cloud Platform storage where older NWM forecasts are cached. This notebook also briefly covers visualizing, subsetting, and exporting forecasts retrieved with Hydrotools.

NOTE: Notebook 4 Requires a newer version of NumPy that is not available on the default CUAHSI JupyterHub instance. Please use the instance "HydroLearn - Intelligent Earth" and ensure to run !pip install hydrotools.nwm_client[gcp].

The notebooks are part of a NWM learning module on HydroLearn.org. When the associated learning module is complete, the link to it will be added here. It is recommended that these notebooks be opened through the CUAHSI JupyterHub App on Hydroshare. This can be done via the 'Open With' button at the top of this resource page.