About San Francisco San Francisco is a vibrant and dynamic city located on the west coast of the United States, in the state of California. Known for its hilly terrain, diverse neighborhoods, and iconic landmarks such as the Golden Gate Bridge and Alcatraz Island, San Francisco is a hub of culture, creativity, and innovation. The city is renowned for its world-class restaurants, thriving arts scene, and historic architecture, and is home to many tech companies and startups. With its mild climate, stunning views, and rich history, San Francisco is a must-visit destination for travelers from around the world.

About Dataset This dataset contains daily weather observations for San Francisco, USA from January 1, 1993 to January 1, 2023. The data is collected from Meteostat. The dataset contains 10 columns with 10958 rows.

For an urban heat island map during an average summer see this dataset.

A heatwave refers to a prolonged period of unusually hot weather. While there is no standard definition of a heatwave in England, the Met Office uses the World Meteorological Organization definition of a heatwave, which is "when the daily maximum temperature of more than five consecutive days exceeds the average maximum temperature by 5°C, the normal period being 1961-1990". They are common in the northern and southern hemisphere during summer have historically been associated with health problems and an increase in mortality.

The urban heat island (UHI) is the phenomenon where temperatures are relatively higher in cities compared to surrounding rural areas due to, for example, the urban surfaces and anthropogenic heat sources.

This urban heat island map was produced using LondUM, a specific set-up of the Met Office Unified Model version 6.1 for London. It uses the Met Office Reading Surface Exchange Scheme (MORUSES), as well as urban morphology data derived from Virtual London. The model was run from May until September 2006 and December 2006. This map shows average surface temperatures over the summer period of 2006 at a 1km by 1km resolution. To find out more about LondUM, see the University of Reading’s website.

The hourly outputs from LondUM have been aggregated and mapped by Jonathon Taylor, UCL Institute for Environmental Design and Engineering. Variables include:

• WSAVGMAX= the average of the maximum daily temperatures across the summer period (May 26th-August 31st)
• WSAVG=the average temperature across the summer period
• WSAVGMIN = the average minimum daily temperature across the summer period
• HWAVGMAX= the average of the maximum daily temperatures across the 2006 heatwave (July 16th-19th)
• HWAVG=the average temperature across the across the 2006 heatwave
• HWAVGMIN = the average minimum daily temperature across 2006 heatwave period

The maps are also available as one combined PDF.

The gif below maps the temperatures across London during the four-day period of 16-19th July, which was considered a heatwave.

If you make use of the LondUM data, please use the following citation to acknowledge the data and reference the publication below for model description:

LondUM (2011). Model data generated by Sylvia I. Bohnenstengel (*), Department of Meteorology, University of Reading and data retrieved from http://www.met.reading.ac.uk/~sws07sib/home/LondUM.html.

(*) Now at Metoffice@Reading, Email: sylvia.bohnenstengel@metoffice.gov.uk

Bohnenstengel SI, Evans S, Clark P and Belcher SeE (2011) Simulations of the London Urban Heat island. Quarterly journal of the Royal Meteorological Society, 137(659). pp. 1625-1640. ISSN 1477-870X doi 10.1002/qj.855. LondUM data (2013).

For an urban heat island map during an average summer see this dataset. A heatwave refers to a prolonged period of unusually hot weather. While there is no standard definition of a heatwave in England, the Met Office uses the World Meteorological Organization definition of a heatwave, which is "when the daily maximum temperature of more than five consecutive days exceeds the average maximum temperature by 5°C, the normal period being 1961-1990". They are common in the northern and southern hemisphere during summer have historically been associated with health problems and an increase in mortality. The urban heat island (UHI) is the phenomenon where temperatures are relatively higher in cities compared to surrounding rural areas due to, for example, the urban surfaces and anthropogenic heat sources. This urban heat island map was produced using LondUM, a specific set-up of the Met Office Unified Model version 6.1 for London. It uses the Met Office Reading Surface Exchange Scheme (MORUSES), as well as urban morphology data derived from Virtual London. The model was run from May until September 2006 and December 2006. This map shows average surface temperatures over the summer period of 2006 at a 1km by 1km resolution. To find out more about LondUM, see the University of Reading’s website. The hourly outputs from LondUM have been aggregated and mapped by Jonathon Taylor, UCL Institute for Environmental Design and Engineering. Variables include: WSAVGMAX= the average of the maximum daily temperatures across the summer period (May 26th-August 31st) WSAVG=the average temperature across the summer period WSAVGMIN = the average minimum daily temperature across the summer period HWAVGMAX= the average of the maximum daily temperatures across the 2006 heatwave (July 16th-19th) HWAVG=the average temperature across the across the 2006 heatwave HWAVGMIN = the average minimum daily temperature across 2006 heatwave period The maps are also available as one combined PDF. The gif below maps the temperatures across London during the four-day period of 16-19th July, which was considered a heatwave. If you make use of the LondUM data, please use the following citation to acknowledge the data and reference the publication below for model description: LondUM (2011). Model data generated by Sylvia I. Bohnenstengel (), Department of Meteorology, University of Reading and data retrieved from http://www.met.reading.ac.uk/~sws07sib/home/LondUM.html. () Now at Metoffice@Reading, Email: sylvia.bohnenstengel@metoffice.gov.uk Bohnenstengel SI, Evans S, Clark P and Belcher SeE (2011) Simulations of the London Urban Heat island. Quarterly journal of the Royal Meteorological Society, 137(659). pp. 1625-1640. ISSN 1477-870X doi 10.1002/qj.855. LondUM data (2013).

Context

Simple time series data for weather prediction time series projects.

Content

The data contains the following information from the UK Met Office location at London Heathrow Airport. The data runs from Jan 1948 to Oct 2020 and includes the following monthly data fields:

• yyyy = Year
• mm = Month
• tmax = Maximum temperature (Celsius)
• tmin = Minimum temperature (Celsius)
• af = Count of Air Frost days in the given month
• rain = Total rainfall (mm)
• sun = Sunshine duration (hrs)

Acknowledgements

Provided by the UK Met Office: https://www.metoffice.gov.uk/research/climate/maps-and-data/historic-station-data Available under Open Government Licence: http://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/

Example code

The following Python code will load into a Pandas DataFrame:

colspecs = [(3, 7), (9,11),(14,18),(22,26),(32,34),(37,42),(45,50)] data = pd.read_fwf('../input/heathrow-weather-data/heathrowdata.txt',colspecs=colspecs)

The following will remove the first few lines of text

data = data[3:].reset_index(drop=True) data.columns = data.iloc[1] data = data[3:].reset_index(drop=True)

Observational datasets from two days (2^nd May 2016, day of year (DOY) 123, and 5^th May 2016, DOY 126) used in Saunders et al. 2024. Day cases are separated by folder named in the format year + DOY.

AWS processed:

Processed and quality-control checked data from a Davis Vantage Pro Plus located in the city of London (Saunders et al. 2024).

CNR4 processed:

Processed and quality-control checked data from a Kipp & Zonen CNR4 radiometer in the city of London (Saunders et al. 2024).

LAS raw:

Raw data from a single beam Kipp and Zonen MKII LAS (850 nm wavelength) sensor defined as the scintillometer path ‘BCT-IMU’ (Saunders et al. 2024) located between the City of London and Islington.

LAS source areas:

Source areas for the scintillometer path ‘BCT-IMU’. Data are derived using the methodology as described by Saunders et al. 2024 (see Fig. 2), and were written using the ‘scintools’ python code (10.5281/zenodo.7434074). Digital surface models used for these cases are also included as part of the ‘scintools’ zenodo repository.

Source areas for the specific case study days used in Saunders et al. 2024 are processed using the python code ‘scint_fp’ (included in 10.5281/zenodo.7434153) using automatic weather station data included here (in the directory AWS processed). csv files included in each subdirectory of ‘LAS_Source_Areas’ are the inputs given to ‘scint_fp’.

Files in the ‘10_mins’ subdirectory are source areas every at 10 minutes. Files in the ‘hourly’ subdirectory are source areas calculated every 60 minutes. The end of each file name indicates the time period (time ending) of which the source area applies.

Files in the ‘SA_constant’ subdirectory are source areas with roughness parameters calculated and held constant along the path (only used for sensitivity tests, see Fig.5 and Fig. SM.5 of Saunders et al. 2024).

LAS processed:

Processed data from the scintillometer path ‘BCT-IMU’. Data were derived using the methodology as described by Saunders et al. 2024 (see Fig. 3), and written using the ‘scint_flux’ python code (10.5281/zenodo.7434143) and using scintillometer source areas derived from the ‘scintools’ python code (10.5281/zenodo.7434074).

Files ending with PERIOD_VAR_## refer to the averaging period performed on the LAS raw data as number in minutes (where ## is 1, 2, 3, 5, 10, 15, 20, 30, and 60 minutes). Files with a name including ‘sa10min’ use source areas calculated every 10 minutes. Otherwise, data are calculated source areas calculated every 60 minutes.

A heatwave refers to a prolonged period of unusually hot weather. While there is no standard definition of a heatwave in England, the Met Office generally uses the World Meteorological Organization definition of a heatwave, which is "when the daily maximum temperature of more than five consecutive days exceeds the average maximum temperature by 5°C, the normal period being 1961-1990". They are common in the northern and southern hemisphere during summer, and have historically been associated with health problems and an increase in mortality. The urban heat island (UHI) is the phenomenon where temperatures are relatively higher in cities compared to surrounding rural areas due to, for example, the urban surfaces and anthropogenic heat sources. For an example of an urban heat island map during an average summer, see this dataset. For an example of an urban heat island map during a warm summer, see this dataset. As well as outdoor temperature, an individual’s heat exposure may also depend on the type of building they are inside, if indoors. Indoor temperature exposure may depend on a number of characteristics, such as the building geometry, construction materials, window sizes, and the ability to add extra ventilation. It is also known that people have different vulnerabilities to heat, with some more prone to negative health issues when exposed to high temperatures. This Triple Jeopardy dataset combines: Urban Heat Island information for London, based on the 55 days between May 26th -July 19th 2006, where the last four days were considered a heatwave An estimate of the indoor temperatures for individual dwellings in London across this time period Population age, as a proxy for heat vulnerability, and distribution From this, local levels of heat-related mortality were estimated using a mortality model derived from epidemiological data. The dataset comprises four layers: Ind_Temp_A – indoor Temperature Anomaly is the difference in degrees Celsius between the estimated indoor temperatures for dwellings and the average indoor temperature estimate for the whole of London, averaged by ward. Positive numbers show dwellings with a greater tendency to overheat in comparison with the London average HeatMortpM – total estimated mortality due to heat (outdoor and indoor) per million population over the entire 55 day period, inclusive of age effects HeatMorUHI – estimated mortality per million population due to increased outdoor temperature exposure caused by the UHI over the 55 day period (excluding the effect of overheating housing), inclusive of age effects HeatMorInd - estimated mortality per million population due to increased temperature exposure caused by heat-vulnerable dwellings (excluding the effect of the UHI) over the 55 day period, inclusive of age effects More information is on this website and in the Triple Jeopardy leaflet. The maps are also available as one combined PDF. More information is on this website and in the Triple Jeopardy leaflet.

AWS processed:

Processed and quality-control checked data from a Davis Vantage Pro Plus located in the city of London (Saunders et al. 2024).

CNR4 processed:

Processed and quality-control checked data from a Kipp & Zonen CNR4 radiometer either in the city of London (‘KSSW’, Saunders et al. 2024) or on BT Tower (‘BTT’).

LAS raw:

Raw data from four scintillometer paths in London; ‘BCT-IMU’, ‘IMU-BTT’, ‘BTT-BCT’ and ‘SCT-SWT’. The experimental setup of ‘BCT-IMU’ is described by Saunders et al. 2024. Three of the four paths (‘BCT-IMU’, ‘BTT-BCT’, ‘SCT-SWT’) consist of a single beam Kipp and Zonen MKII LAS (850 nm wavelength) sensor. ‘IMU-BTT’ is a Scintec BLS900 sensor.

LAS source areas:

Source areas for four scintillometer paths. Data are derived using the methodology as described by Saunders et al. 2024 (see Fig. 2), and were written using the ‘scintools’ python code (10.5281/zenodo.7434074). Digital surface models used for these cases are also included as part of the ‘scintools’ zenodo repository.

Source areas for the specific case study days used are processed using the python code ‘scint_fp’ (included in 10.5281/zenodo.7434153) using automatic weather station data included here (in the directory AWS processed). csv files included in each subdirectory of ‘LAS_Source_Areas’ are the inputs given to ‘scint_fp’.

Source areas are calculated every hour using inputs averaged over the last 10 minutes of that our (time-ending). For example, a 1200 source area is calculated using meteorological conditions averaged over 1150-1200. The end of each file name indicates the time period (time ending) of which the source area applies.

LAS processed:

Processed data for four scintillometer paths. Data were derived using the methodology as described by Saunders et al. 2024 (see Fig. 3), and written using the ‘scint_flux’ python code (10.5281/zenodo.7434143) and using scintillometer source areas derived from the ‘scintools’ python code (10.5281/zenodo.7434074).

Files ending with PERIOD_VAR_## refer to the averaging period performed on the LAS raw data as number in minutes (where ## is 1, 10, and 60 minutes). Files with a name including ‘sa10min_ending’ use source areas calculated every hour with input meteorological conditions averaged over 10 minutes.

These data are used in Chapter 4, “Impact of horizontal resolution on numerical weather predicted sensible heat fluxes in London” in the University of Reading thesis "Urban surface-atmosphere exchanges: scintillometry observations and NWP evaluation" submitted by Beth Saunders, 2024.

The NWP (Met Office Unified Model, UM) dataset consists of a one-day case study (13^th May 2016, day-of-year 134). Some of the experimental setup (observations) is also described in Saunders et. al 2024: “Methodology to evaluate numerical weather predictions using large aperture scintillometry sensible heat fluxes: demonstration in London”, DOI: https://doi.org/10.1002/qj.4837" href="https://doi.org/10.1002/qj.4837" target="_blank" rel="noopener">https://doi.org/10.1002/qj.4837. File names and subdirectories are named in the format year + day of year (DOY).

These data correspond to the 300 m configuration of the UM, which are run offline over London. 100 m and 1.5 km configurations of this model run are also available (10.5281/zenodo.15169388 and 10.5281/zenodo.15241730 respectively).

Analysis of ‘London bike sharing dataset’ provided by Analyst-2 (analyst-2.ai), based on source dataset retrieved from https://www.kaggle.com/hmavrodiev/london-bike-sharing-dataset on 12 November 2021.

--- Dataset description provided by original source is as follows ---

License

These licence terms and conditions apply to TfL's free transport data service and are based on version 2.0 of the Open Government Licence with specific amendments for Transport for London (the "Licence"). TfL may at any time revise this Licence without notice. It is up to you ("You") to regularly review the Licence, which will be available on this website, in case there are any changes. Your continued use of the transport data feeds You have opted to receive ("Information") after a change has been made to the Licence will be treated as Your acceptance of that change.

Using Information under this Licence TfL grants You a worldwide, royalty-free, perpetual, non-exclusive Licence to use the Information subject to the conditions below (as varied from time to time).

This Licence does not affect Your freedom under fair dealing or fair use or any other copyright or database right exceptions and limitations.

This Licence shall apply from the date of registration and shall continue for the period the Information is provided to You or You breach the Licence.

Rights You are free to:

Copy, publish, distribute and transmit the Information Adapt the Information and Exploit the Information commercially and non-commercially for example, by combining it with other Information, or by including it in Your own product or application Requirements You must, where You do any of the above:

Acknowledge TfL as the source of the Information by including the following attribution statement 'Powered by TfL Open Data' Acknowledge that this Information contains Ordnance Survey derived data by including the following attribution statement: 'Contains OS data © Crown copyright and database rights 2016' and Geomni UK Map data © and database rights [2019] Ensure our intellectual property rights, including all logos, design rights, patents and trademarks, are protected by following our design and branding guidelines Limit traffic requests up to a maximum of 300 calls per minute per data feed. TfL reserves the right to throttle or limit access to feeds when it is believed the overall service is being degraded by excessive use and Ensure the information You provide on registration is accurate These are important conditions of this Licence and if You fail to comply with them the rights granted to You under this Licence, or any similar licence granted by TfL, will end automatically.

Exemptions This Licence does not:

Transfer any intellectual property rights in the Information to You or any third party Include personal data in the Information Provide any rights to use the Information after this Licence has ended Provide any rights to use any other intellectual property rights, including patents, trade marks, and design rights or permit You to: Use data from the Oyster, Congestion Charging and Santander Cycles websites to populate or update any other software or database or Use any automated system, software or process to extract content and/or data, including trawling, data mining and screen scraping in relation to the Oyster, Congestion Charging and Santander Cycles websites, except where expressly permitted under a written licence agreement with TfL. These are important conditions of this Licence and, if You fail to comply with them, the rights granted to You under this Licence, or any similar licence granted by TfL, will end automatically.

Non-endorsement This Licence does not grant You any right to use the Information in a way that suggests any official status or that TfL endorses You or Your use of the Information.

Context

The purpose is to try predict the future bike shares.

Content

The data is acquired from 3 sources:
- Https://cycling.data.tfl.gov.uk/ 'Contains OS data © Crown copyright and database rights 2016' and Geomni UK Map data © and database rights [2019] 'Powered by TfL Open Data'
- freemeteo.com - weather data
- https://www.gov.uk/bank-holidays
From 1/1/2015 to 31/12/2016

The data from cycling dataset is grouped by "Start time", this represent the count of new bike shares grouped by hour. The long duration shares are not taken in the count.

Metadata:

"timestamp" - timestamp field for grouping the data
"cnt" - the count of a new bike shares
"t1" - real temperature in C
"t2" - temperature in C "feels like"
"hum" - humidity in percentage
"wind_speed" - wind speed in km/h
"weather_code" - category of the weather
"is_holiday" - boolean field - 1 holiday / 0 non holiday
"is_weekend" - boolean field - 1 if the day is weekend
"season" - category field meteorological seasons: 0-spring ; 1-summer; 2-fall; 3-winter.

"weathe_code" category description:
1 = Clear ; mostly clear but have some values with haze/fog/patches of fog/ fog in vicinity
2 = scattered clouds / few clouds
3 = Broken clouds
4 = Cloudy
7 = Rain/ light Rain shower/ Light rain
10 = rain with thunderstorm
26 = snowfall
94 = Freezing Fog

--- Original source retains full ownership of the source dataset ---

Surface synoptic weather reports for the entire globe for the 10-year period from December 1981 through November 1991 have been processed, edited, and rewritten to provide a data set designed for use in cloud analyses. The information in these reports relating to clouds, including the present weather information, was extracted and put through a series of quality control checks. Reports not meeting certain quality control standards were rejected, as were reports from buoys and automatic weather stations. Correctable inconsistencies within reports were edited for consistency, so that the "edited cloud report" can be used for cloud analysis without further quality checking. Cases of "sky obscured" were interpreted by reference to the present weather code as to whether they indicated fog, rain or snow and were given appropriate cloud type designations. Nimbostratus clouds, which are not specifically coded for in the standard synoptic code, were also given a special designation. Changes made to an original report are indicated in the edited report so that the original report can be reconstructed if desired. While low cloud amount is normally given directly in the synoptic report, the edited cloud report also includes the amounts, either directly reported or inferred, of middle and high clouds, both the non-overlapped amounts and the "actual" amounts (which may be overlapped). Since illumination from the moon is important for the adequate detection of clouds at night, both the relative lunar illuminance and the solar altitude are given, as well as a parameter that indicates whether our recommended illuminance criterion was satisfied. This data set contains 124 million reports from land stations and 15 million reports from ships. Each report is 56 characters in length. The archive consists of 240 files, one file for each month of data for land and ocean separately. With this data set a user can develop a climatology for any particular cloud type or group of types, for any geographical region and any spatial and temporal resolution desired. For access to the data files, click this link to the CDIAC data transition website: http://cdiac.ess-dive.lbl.gov/epubs/ndp/ndp026b/ndp026b.html

The WEA dataset is derived from the WeatherBench repository and designed for medium-range weather forecasting at five geographically diverse cities: London (UK), New York (US), Hong Kong (China), Cape Town (South Africa), and Singapore. It spans the period from 1979 to 2018, with a temporal resolution of 6 hours and a spatial resolution of 5.625° in both latitude and longitude. Each city is matched to its nearest grid point on the WeatherBench grid using minimal absolute distance in both axes.

AWS processed:

Processed and quality-control checked data from a Davis Vantage Pro Plus located in the city of London (Saunders et al. 2024).

CNR4 processed:

Processed and quality-control checked data from a Kipp & Zonen CNR4 radiometer in the city of London (Saunders et al. 2024).

LAS raw:

Raw data from a single beam Kipp and Zonen MKII LAS (850 nm wavelength) sensor defined as the scintillometer path ‘BCT-IMU’ (Saunders et al. 2024) located between the City of London and Islington.

LAS source areas:

Source areas for the scintillometer path ‘BCT-IMU’. Data are derived using the methodology as described by Saunders et al. 2024 (see Fig. 2), and were written using the ‘scintools’ python code (10.5281/zenodo.7434074). Digital surface models used for these cases are also included as part of the ‘scintools’ zenodo repository.

Source areas for the specific case study day are processed using the python code ‘scint_fp’ (included in 10.5281/zenodo.7434153) using automatic weather station data included here (in the directory AWS processed). csv files included in each subdirectory of ‘LAS_Source_Areas’ are the inputs given to ‘scint_fp’.

Source areas are calculated every hour using inputs averaged over the last 10 minutes of that our (time-ending). For example, a 1200 source area is calculated using meteorological conditions averaged over 1150-1200. The end of each file name indicates the time period (time ending) of which the source area applies.

LAS processed:

Processed data from the scintillometer path ‘BCT-IMU’. Data were derived using the methodology as described by Saunders et al. 2024 (see Fig. 3), and written using the ‘scint_flux’ python code (10.5281/zenodo.7434143) and using scintillometer source areas derived from the ‘scintools’ python code (10.5281/zenodo.7434074).

Files ending with PERIOD_VAR_## refer to the averaging period performed on the LAS raw data as number in minutes (where ## is 1, 10, and 60 minutes). Files with a name including ‘sa10min_ending’ use source areas calculated every hour with input meteorological conditions averaged over 10 minutes.

This dataset is a supplement to the publication in the 2016 Applied Energy conference in Beijing, China. The data reveals that the adaptive comfort temperature in indoors in buildings isincreased when indoor and outdoor temperatures increased. The neutral comfort temperature will change from around 24 C to 26 C from present day to 2080s for occupants in London. When people get used to the increasing high temperatures in future, the most unstable factor which will severely harm occupant health is the more and more frequent sudden peaks of high temperatures.