VITAL SIGNS INDICATOR Migration (EQ4)

FULL MEASURE NAME Migration flows

LAST UPDATED December 2018

DESCRIPTION Migration refers to the movement of people from one location to another, typically crossing a county or regional boundary. Migration captures both voluntary relocation – for example, moving to another region for a better job or lower home prices – and involuntary relocation as a result of displacement. The dataset includes metropolitan area, regional, and county tables.

DATA SOURCE American Community Survey County-to-County Migration Flows 2012-2015 5-year rolling average http://www.census.gov/topics/population/migration/data/tables.All.html

CONTACT INFORMATION vitalsigns.info@bayareametro.gov

METHODOLOGY NOTES (across all datasets for this indicator) Data for migration comes from the American Community Survey; county-to-county flow datasets experience a longer lag time than other standard datasets available in FactFinder. 5-year rolling average data was used for migration for all geographies, as the Census Bureau does not release 1-year annual data. Data is not available at any geography below the county level; note that flows that are relatively small on the county level are often within the margin of error. The metropolitan area comparison was performed for the nine-county San Francisco Bay Area, in addition to the primary MSAs for the nine other major metropolitan areas, by aggregating county data based on current metropolitan area boundaries. Data prior to 2011 is not available on Vital Signs due to inconsistent Census formats and a lack of net migration statistics for prior years. Only counties with a non-negligible flow are shown in the data; all other pairs can be assumed to have zero migration.

Given that the vast majority of migration out of the region was to other counties in California, California counties were bundled into the following regions for simplicity: Bay Area: Alameda, Contra Costa, Marin, Napa, San Francisco, San Mateo, Santa Clara, Solano, Sonoma Central Coast: Monterey, San Benito, San Luis Obispo, Santa Barbara, Santa Cruz Central Valley: Fresno, Kern, Kings, Madera, Merced, Tulare Los Angeles + Inland Empire: Imperial, Los Angeles, Orange, Riverside, San Bernardino, Ventura Sacramento: El Dorado, Placer, Sacramento, Sutter, Yolo, Yuba San Diego: San Diego San Joaquin Valley: San Joaquin, Stanislaus Rural: all other counties (23)

One key limitation of the American Community Survey migration data is that it is not able to track emigration (movement of current U.S. residents to other countries). This is despite the fact that it is able to quantify immigration (movement of foreign residents to the U.S.), generally by continent of origin. Thus the Vital Signs analysis focuses primarily on net domestic migration, while still specifically citing in-migration flows from countries abroad based on data availability.

This dataset contains non-seasonally adjusted California Unemployment Rate by age groups, from the Current Population Survey (CPS). The age group ranges are as follows; 16-19 ; 20 - 24; 25 - 34; 35 - 44; 45 - 54; 55 -64; 65+. This data is based on a 12-month moving average.

This dataset is invaluable for data science applications due to its granularity and the historical depth it offers. With detailed monthly data on unemployment rates by age groups, data scientists can perform a myriad of analyses:

• Time Series Analysis: Analyzing trends over time to forecast future unemployment rates or identify cyclic patterns.
• Demographic Analysis: Understanding which age groups are most affected by unemployment and how this changes through economic conditions.
• Geographic Analysis: Although the dataset currently focuses on California, if similar data is available for other regions, comparisons can be made to understand regional economic differences.

The dataset can also be merged with other socioeconomic indicators like GDP, education levels, and industry growth metrics to examine broader economic narratives or policy impacts.

Note: In these datasets, a person is defined as up to date if they have received at least one dose of an updated COVID-19 vaccine. The Centers for Disease Control and Prevention (CDC) recommends that certain groups, including adults ages 65 years and older, receive additional doses.

On 6/16/2023 CDPH replaced the booster measures with a new “Up to Date” measure based on CDC’s new recommendations, replacing the primary series, boosted, and bivalent booster metrics The definition of “primary series complete” has not changed and is based on previous recommendations that CDC has since simplified. A person cannot complete their primary series with a single dose of an updated vaccine. Whereas the booster measures were calculated using the eligible population as the denominator, the new up to date measure uses the total estimated population. Please note that the rates for some groups may change since the up to date measure is calculated differently than the previous booster and bivalent measures.

This data is from the same source as the Vaccine Progress Dashboard at https://covid19.ca.gov/vaccination-progress-data/ which summarizes vaccination data at the county level by county of residence. Where county of residence was not reported in a vaccination record, the county of provider that vaccinated the resident is included. This applies to less than 1% of vaccination records. The sum of county-level vaccinations does not equal statewide total vaccinations due to out-of-state residents vaccinated in California.

These data do not include doses administered by the following federal agencies who received vaccine allocated directly from CDC: Indian Health Service, Veterans Health Administration, Department of Defense, and the Federal Bureau of Prisons.

Totals for the Vaccine Progress Dashboard and this dataset may not match, as the Dashboard totals doses by Report Date and this dataset totals doses by Administration Date. Dose numbers may also change for a particular Administration Date as data is updated.

Previous updates:

• On March 3, 2023, with the release of HPI 3.0 in 2022, the previous equity scores have been updated to reflect more recent community survey information. This change represents an improvement to the way CDPH monitors health equity by using the latest and most accurate community data available. The HPI uses a collection of data sources and indicators to calculate a measure of community conditions ranging from the most to the least healthy based on economic, housing, and environmental measures.

• Starting on July 13, 2022, the denominator for calculating vaccine coverage has been changed from age 5+ to all ages to reflect new vaccine eligibility criteria. Previously the denominator was changed from age 16+ to age 12+ on May 18, 2021, then changed from age 12+ to age 5+ on November 10, 2021, to reflect previous changes in vaccine eligibility criteria. The previous datasets based on age 16+ and age 5+ denominators have been uploaded as archived tables.

• Starting on May 29, 2021 the methodology for calculating on-hand inventory in the shipped/delivered/on-hand dataset has changed. Please see the accompanying data dictionary for details. In addition, this dataset is now down to the ZIP code level.

The Likely Tables herd contains migrants, but this herd does not migrate between traditional summer and winter seasonal ranges. Instead, much of the herd displays a nomadic tendency, slowly migrating north for the summer using various high use areas as they move. Therefore, annual ranges were modeled using year-round data to demarcate high use areas in lieu modeling specific winter ranges. A high use area being used during winter by many of the collared animals is west of the Warner Mountains, east of U.S. Highway 395, and north of Moon Lake. Some animals live in the agricultural fields west of U.S. Highway 395. There appears to be little if any movement across the highway, which is fenced on both sides in this area. Summer ranges are spread out, with some individuals moving as far north as Goose Lake. A few outliers in the herd moved long distances south toward the Lassen herd or east to Nevada. Drought, increasing fire frequency, invasive annual grasses, and juniper encroachment negatively affect pronghorn habitat. Recent population surveys indicate a declining population (Trausch and others, 2020). Juniper removal on public and private lands have potential to improve habitat quality and potentially reduce predation (Ewanyk, 2020). Fences on public and private lands affect movement corridors and increase crossing and/or migration times. Recent fence modifications on BLM lands have shown potential to ease pronghorn movements (Hudgens, 2022). These mapping layers show the location of the migration corridors for pronghorn (Antilocapra americana) in the Likely Tables population in California. They were developed from 29 migration sequences collected from a sample size of 17 animals comprising GPS locations collected every 1-4 hours.

Note: This dataset is no longer being updated due to the end of the COVID-19 Public Health Emergency.

The California Department of Public Health (CDPH) is identifying vaccination status of COVID-19 cases, hospitalizations, and deaths by analyzing the state immunization registry and registry of confirmed COVID-19 cases. Post-vaccination cases are individuals who have a positive SARS-Cov-2 molecular test (e.g. PCR) at least 14 days after they have completed their primary vaccination series.

Tracking cases of COVID-19 that occur after vaccination is important for monitoring the impact of immunization campaigns. While COVID-19 vaccines are safe and effective, some cases are still expected in persons who have been vaccinated, as no vaccine is 100% effective. For more information, please see https://www.cdph.ca.gov/Programs/CID/DCDC/Pages/COVID-19/Post-Vaccine-COVID19-Cases.aspx

Post-vaccination infection data is updated monthly and includes data on cases, hospitalizations, and deaths among the unvaccinated and the vaccinated. Partially vaccinated individuals are excluded. To account for reporting and processing delays, there is at least a one-month lag in provided data (for example data published on 9/9/22 will include data through 7/31/22).

Notes:

• On September 9, 2022, the post-vaccination data has been changed to compare unvaccinated with those with at least a primary series completed for persons age 5+. These data will be updated monthly (first Thursday of the month) and include at least a one month lag.

• On February 2, 2022, the post-vaccination data has been changed to distinguish between vaccination with a primary series only versus vaccinated and boosted. The previous dataset has been uploaded as an archived table. Additionally, the lag on this data has been extended to 14 days.

• On November 29, 2021, the denominator for calculating vaccine coverage has been changed from age 16+ to age 12+ to reflect new vaccine eligibility criteria. The previous dataset based on age 16+ denominators has been uploaded as an archived table.

The Clear Lake herd contains migrants, but this herd does not migrate between traditional summer and winter seasonal ranges. Instead, much of the herd displays a nomadic tendency, slowly migrating north, east, or south for the summer using various high use areas as they move. Therefore, annual ranges were modeled using year-round data to demarcate high use areas in lieu of modeling specific winter ranges. The areas adjacent to Clear Lake Reservoir were heavily used during winter by many of the collared animals. A few collared individuals persisted west of State Route 139 year-round, seemingly separated from the rest of the herd due to this highway barrier. However, some pronghorn cross this road near Cornell and join this subgroup. Summer ranges are spread out, with many individuals moving southeast through protected forests or over the state border into Oregon. A few outliers in the herd moved long distances south, crossing State Route 139 to Oak Ridge, or east into Likely Tables pronghorn herd areas. Drought, increasing fire frequency, invasive annual grasses, and juniper encroachment negatively affect pronghorn habitat. Recent population surveys indicate a declining population (Trausch and others, 2020). Juniper removal on public and private lands has potential to improve habitat quality and potentially reduce predation (Ewanyk, 2020). These mapping layers show the _location of the migration stopovers for pronghorn (Antilocapra americana) in the Clear Lake population in California. They were developed from 72 migration sequences collected from a sample size of 23 animals comprising GPS locations collected every 1-6 hours.

The project lead for the collection of this data was Richard Shinn. Pronghorn (30 adult females and 1 adult male) were captured and equipped with GPS collars (Sirtrack, Havelock North, NZ) transmitting data from 2014-2020. The Likely Tables herd contains migrants, but this herd does not migrate between traditional summer and winter seasonal ranges. Instead, much of the herd displays a somewhat nomadic migratory tendency, slowly migrating north for the summer using various high use areas as they move. Therefore, annual home ranges were modeled using year-round data to demarcate high use areas in lieu of modeling the specific winter ranges commonly seen in other ungulate analyses in California. A high use area being used during winter by many of the collared animals is west of the Warner Mountains, east of Highway 395, and north of the Modoc County line. Additionally, a few individuals persist east of Highway 395, seemingly separated from the rest of the herd. Summer ranges are spread out, with some individuals moving into the Modoc National Forest and as far north as Goose Lake. A few outliers in the herd moved long distances south or east. GPS locations were fixed between 1-4 hour intervals in the dataset. To improve the quality of the data set as per Bjørneraas et al. (2010), the GPS data were filtered prior to analysis to remove locations which were: i) further from either the previous point or subsequent point than an individual pronghorn is able to travel in the elapsed time, ii) forming spikes in the movement trajectory based on outgoing and incoming speeds and turning angles sharper than a predefined threshold , or iii) fixed in 2D space and visually assessed as a bad fix by the analyst. The methodology used for this migration analysis allowed for the mapping of the herd’s home range and the identification and prioritization of migration corridors. Brownian Bridge Movement Models (BBMMs; Sawyer et al. 2009) were constructed with GPS collar data from 17 migrating pronghorn, including 29 migration sequences, location, date, time, and average location error as inputs in Migration Mapper. The average migration time and average migration distance for pronghorn was 15.42 days and 38.02 km, respectively. Corridors and stopovers were prioritized based on the number of animals moving through a particular area. BBMMs were produced at a spatial resolution of 50 m using a sequential fix interval of less than 27 hours. Due to varying fix rates in the data, separate models using Brownian bridge movement models (BMMM), with an adaptable variance rate, and fixed motion variances of 1000 were produced per migration sequence and visually compared for the entire dataset, with best models being combined prior to population-level analyses (72% of sequences selected with BBMM). In general, fixed motion variances were used when BBMM variances exceeded 8000. Home range analyses were based on data from 20 pronghorn and 25 year-round sequences using a combination of BBMMs and fixed motion variances of 1000 (84% of sequences selected with BBMM). Home range designations for this herd may expand with a larger sample, filling in some of the gaps between home range polygons in the map. Large water bodies were clipped from the final outputs.Corridors are visualized based on pronghorn use per cell, with greater than or equal to 1 pronghorn and greater than or equal to 3 pronghorn (20% of the sample) representing migration corridors and high use corridors, respectively. Stopovers were calculated as the top 10 percent of the population level utilization distribution during migrations and can be interpreted as high use areas. Stopover polygon areas less than 20,000 m2 were removed, but remaining small stopovers may be interpreted as short-term resting sites, likely based on a small concentration of points from an individual animal. Home range is visualized as the 50th percentile contour of the home range utilization distribution.

Reasons for moving and location of previous dwelling for households that moved in the past five years, and intentions to move in less than five years for all households, Canada, provinces and territories.

UPDATE 1/7/2025: On June 28th 2023, the San Francisco Police Department (SFPD) changed its Stops Data Collection System (SDCS). As a result of this change, record identifiers have changed from the Department of Justice (DOJ) identifier to an internal record numbering system (referred to as "LEA Record ID"). The data that SFPD uploads to the DOJ system will contain the internal record number which can be used for joins with the data available on DataSF.

A. SUMMARY The San Francisco Police Department (SFPD) Stop Data was designed to capture information to comply with the Racial and Identity Profiling Act (RIPA), or California Assembly Bill (AB)953. SFPD officers collect specific information on each stop, including elements of the stop, circumstances and the perceived identity characteristics of the individual(s) stopped. The information obtained by officers is reported to the California Department of Justice. This dataset includes data on stops starting on July 1st, 2018, which is when the data collection program went into effect. Read the detailed overview for this dataset here.

B. HOW THE DATASET IS CREATED By the end of each shift, officers enter all stop data into the Stop Data Collection System, which is automatically submitted to the California Department of Justice (CA DOJ). Once a quarter the Department receives a stops data file from CA DOJ. The SFPD conducts several transformations of this data to ensure privacy, accuracy and compliance with State law and regulation. For increased usability, text descriptions have also been added for several data fields which include numeric codes (including traffic, suspicion, citation, and custodial arrest offense codes, and actions taken as a result of a stop). See the data dictionaries below for explanations of all coded data fields. Read more about the data collection, and transformation, including geocoding and PII cleaning processes, in the detailed overview of this dataset.

C. UPDATE PROCESS Information is updated on a quarterly basis.

D. HOW TO USE THIS DATASET This dataset includes information about police stops that occurred, including some details about the person(s) stopped, and what happened during the stop. Each row is a person stopped with a record identifier for the stop and a unique identifier for the person. A single stop may involve multiple people and may produce more than one associated unique identifier for the same record identifier. A certain percentage of stops have stop information that can’t be geocoded. This may be due to errors in data input at the officer level (typos in entry or providing an address that doesn't exist). More often, this is due to officers providing a level of detail that isn't codable to a geographic coordinate - most often at the Airport (ie: Terminal 3, door 22.) In these cases, the location of the stops is coded as unknown.

E. DATA DICTIONARIES CJIS Offense Codes data look up table

Look up table for other coded data fields

Note: In these datasets, a person is defined as up to date if they have received at least one dose of an updated COVID-19 vaccine. The Centers for Disease Control and Prevention (CDC) recommends that certain groups, including adults ages 65 years and older, receive additional doses.

Starting on July 13, 2022, the denominator for calculating vaccine coverage has been changed from age 5+ to all ages to reflect new vaccine eligibility criteria. Previously the denominator was changed from age 16+ to age 12+ on May 18, 2021, then changed from age 12+ to age 5+ on November 10, 2021, to reflect previous changes in vaccine eligibility criteria. The previous datasets based on age 12+ and age 5+ denominators have been uploaded as archived tables.

Starting June 30, 2021, the dataset has been reconfigured so that all updates are appended to one dataset to make it easier for API and other interfaces. In addition, historical data has been extended back to January 5, 2021.

This dataset shows full, partial, and at least 1 dose coverage rates by zip code tabulation area (ZCTA) for the state of California. Data sources include the California Immunization Registry and the American Community Survey’s 2015-2019 5-Year data.

This is the data table for the LHJ Vaccine Equity Performance dashboard. However, this data table also includes ZTCAs that do not have a VEM score.

This dataset also includes Vaccine Equity Metric score quartiles (when applicable), which combine the Public Health Alliance of Southern California’s Healthy Places Index (HPI) measure with CDPH-derived scores to estimate factors that impact health, like income, education, and access to health care. ZTCAs range from less healthy community conditions in Quartile 1 to more healthy community conditions in Quartile 4.

The Vaccine Equity Metric is for weekly vaccination allocation and reporting purposes only. CDPH-derived quartiles should not be considered as indicative of the HPI score for these zip codes. CDPH-derived quartiles were assigned to zip codes excluded from the HPI score produced by the Public Health Alliance of Southern California due to concerns with statistical reliability and validity in populations smaller than 1,500 or where more than 50% of the population resides in a group setting.

These data do not include doses administered by the following federal agencies who received vaccine allocated directly from CDC: Indian Health Service, Veterans Health Administration, Department of Defense, and the Federal Bureau of Prisons.

For some ZTCAs, vaccination coverage may exceed 100%. This may be a result of many people from outside the county coming to that ZTCA to get their vaccine and providers reporting the county of administration as the county of residence, and/or the DOF estimates of the population in that ZTCA are too low. Please note that population numbers provided by DOF are projections and so may not be accurate, especially given unprecedented shifts in population as a result of the pandemic.

Background
The Labour Force Survey (LFS) is a unique source of information using international definitions of employment and unemployment and economic inactivity, together with a wide range of related topics such as occupation, training, hours of work and personal characteristics of household members aged 16 years and over. It is used to inform social, economic and employment policy. The LFS was first conducted biennially from 1973-1983. Between 1984 and 1991 the survey was carried out annually and consisted of a quarterly survey conducted throughout the year and a 'boost' survey in the spring quarter (data were then collected seasonally). From 1992 quarterly data were made available, with a quarterly sample size approximately equivalent to that of the previous annual data. The survey then became known as the Quarterly Labour Force Survey (QLFS). From December 1994, data gathering for Northern Ireland moved to a full quarterly cycle to match the rest of the country, so the QLFS then covered the whole of the UK (though some additional annual Northern Ireland LFS datasets are also held at the UK Data Archive). Further information on the background to the QLFS may be found in the documentation.

Longitudinal data
The LFS retains each sample household for five consecutive quarters, with a fifth of the sample replaced each quarter. The main survey was designed to produce cross-sectional data, but the data on each individual have now been linked together to provide longitudinal information. The longitudinal data comprise two types of linked datasets, created using the weighting method to adjust for non-response bias. The two-quarter datasets link data from two consecutive waves, while the five-quarter datasets link across a whole year (for example January 2010 to March 2011 inclusive) and contain data from all five waves. A full series of longitudinal data has been produced, going back to winter 1992. Linking together records to create a longitudinal dimension can, for example, provide information on gross flows over time between different labour force categories (employed, unemployed and economically inactive). This will provide detail about people who have moved between the categories. Also, longitudinal information is useful in monitoring the effects of government policies and can be used to follow the subsequent activities and circumstances of people affected by specific policy initiatives, and to compare them with other groups in the population. There are however methodological problems which could distort the data resulting from this longitudinal linking. The ONS continues to research these issues and advises that the presentation of results should be carefully considered, and warnings should be included with outputs where necessary.

New reweighting policy
Following the new reweighting policy ONS has reviewed the latest population estimates made available during 2019 and have decided not to carry out a 2019 LFS and APS reweighting exercise. Therefore, the next reweighting exercise will take place in 2020. These will incorporate the 2019 Sub-National Population Projection data (published in May 2020) and 2019 Mid-Year Estimates (published in June 2020). It is expected that reweighted Labour Market aggregates and microdata will be published towards the end of 2020/early 2021.

LFS Documentation
The documentation available from the Archive to accompany LFS datasets largely consists of the latest version of each user guide volume alongside the appropriate questionnaire for the year concerned. However, volumes are updated periodically by ONS, so users are advised to check the latest documents on the ONS Labour Force Survey - User Guidance pages before commencing analysis. This is especially important for users of older QLFS studies, where information and guidance in the user guide documents may have changed over time.

Additional data derived from the QLFS
The Archive also holds further QLFS series: End User Licence (EUL) quarterly data; Secure Access datasets; household datasets; quarterly, annual and ad hoc module datasets compiled for Eurostat; and some additional annual Northern Ireland datasets.

Variables DISEA and LNGLST
Dataset A08 (Labour market status of disabled people) which ONS suspended due to an apparent discontinuity between April to June 2017 and July to September 2017 is now available. As a result of this apparent discontinuity and the inconclusive investigations at this stage, comparisons should be made with caution between April to June 2017 and subsequent time periods. However users should note that the estimates are not seasonally adjusted, so some of the change between quarters could be due to seasonality. Further recommendations on historical comparisons of the estimates will be given in November 2018 when ONS are due to publish estimates for July to September 2018.

An article explaining the quality assurance investigations that have been conducted so far is available on the ONS Methodology webpage. For any queries about Dataset A08 please email Labour.Market@ons.gov.uk.

Occupation data for 2021 and 2022 data files

The ONS has identified an issue with the collection of some occupational data in 2021 and 2022 data files in a number of their surveys. While they estimate any impacts will be small overall, this will affect the accuracy of the breakdowns of some detailed (four-digit Standard Occupational Classification (SOC)) occupations, and data derived from them. Further information can be found in the ONS article published on 11 July 2023: https://www.ons.gov.uk/employmentandlabourmarket/peopleinwork/employmentandemployeetypes/articles/revisionofmiscodedoccupationaldataintheonslabourforcesurveyuk/january2021toseptember2022" style="background-color: rgb(255, 255, 255);">Revision of miscoded occupational data in the ONS Labour Force Survey, UK: January 2021 to September 2022.

2022 Weighting

The population totals used for the latest LFS estimates use projected growth rates from Real Time Information (RTI) data for UK, EU and non-EU populations based on 2021 patterns. The total population used for the LFS therefore does not take into account any changes in migration, birth rates, death rates, and so on since June 2021, and hence levels estimates may be under- or over-estimating the true values and should be used with caution. Estimates of rates will, however, be robust.

This annual study provides migration pattern data for the United States by State or by county and are available for inflows (the number of new residents who moved to a State or county and where they migrated from) and outflows (the number of residents who left a State or county and where they moved to). The data include the number of returns filed, number of personal exemptions claimed, total adjusted gross income, and aggregate migration flows at the State level, by the size of adjusted gross income (AGI) and by age of the primary taxpayer. Data are collected and based on year-to-year address changes reported on U.S. Individual Income Tax Returns (Form 1040) filed with the IRS. SOI collects these data as part of its Individual Income Tax Return (Form 1040) Statistics program, Data by Geographic Areas, U.S. Population Migration Data.

Abstract copyright UK Data Service and data collection copyright owner.

Background
The Labour Force Survey (LFS) is a unique source of information using international definitions of employment and unemployment and economic inactivity, together with a wide range of related topics such as occupation, training, hours of work and personal characteristics of household members aged 16 years and over. It is used to inform social, economic and employment policy. The LFS was first conducted biennially from 1973-1983. Between 1984 and 1991 the survey was carried out annually and consisted of a quarterly survey conducted throughout the year and a 'boost' survey in the spring quarter (data were then collected seasonally). From 1992 quarterly data were made available, with a quarterly sample size approximately equivalent to that of the previous annual data. The survey then became known as the Quarterly Labour Force Survey (QLFS). From December 1994, data gathering for Northern Ireland moved to a full quarterly cycle to match the rest of the country, so the QLFS then covered the whole of the UK (though some additional annual Northern Ireland LFS datasets are also held at the UK Data Archive). Further information on the background to the QLFS may be found in the documentation.

Longitudinal data
The LFS retains each sample household for five consecutive quarters, with a fifth of the sample replaced each quarter. The main survey was designed to produce cross-sectional data, but the data on each individual have now been linked together to provide longitudinal information. The longitudinal data comprise two types of linked datasets, created using the weighting method to adjust for non-response bias. The two-quarter datasets link data from two consecutive waves, while the five-quarter datasets link across a whole year (for example January 2010 to March 2011 inclusive) and contain data from all five waves. A full series of longitudinal data has been produced, going back to winter 1992. Linking together records to create a longitudinal dimension can, for example, provide information on gross flows over time between different labour force categories (employed, unemployed and economically inactive). This will provide detail about people who have moved between the categories. Also, longitudinal information is useful in monitoring the effects of government policies and can be used to follow the subsequent activities and circumstances of people affected by specific policy initiatives, and to compare them with other groups in the population. There are however methodological problems which could distort the data resulting from this longitudinal linking. The ONS continues to research these issues and advises that the presentation of results should be carefully considered, and warnings should be included with outputs where necessary.

New reweighting policy
Following the new reweighting policy ONS has reviewed the latest population estimates made available during 2019 and have decided not to carry out a 2019 LFS and APS reweighting exercise. Therefore, the next reweighting exercise will take place in 2020. These will incorporate the 2019 Sub-National Population Projection data (published in May 2020) and 2019 Mid-Year Estimates (published in June 2020). It is expected that reweighted Labour Market aggregates and microdata will be published towards the end of 2020/early 2021.

LFS Documentation
The documentation available from the Archive to accompany LFS datasets largely consists of the latest version of each user guide volume alongside the appropriate questionnaire for the year concerned. However, volumes are updated periodically by ONS, so users are advised to check the latest documents on the ONS Labour Force Survey - User Guidance pages before commencing analysis. This is especially important for users of older QLFS studies, where information and guidance in the user guide documents may have changed over time.

Additional data derived from the QLFS
The Archive also holds further QLFS series: End User Licence (EUL) quarterly data; Secure Access datasets; household datasets; quarterly, annual and ad hoc module datasets compiled for Eurostat; and some additional annual Northern Ireland datasets.

Variables DISEA and LNGLST
Dataset A08 (Labour market status of disabled people) which ONS suspended due to an apparent discontinuity between April to June 2017 and July to September 2017 is now available. As a result of this apparent discontinuity and the inconclusive...

Description

This comprehensive dataset contains detailed information about sleep health and lifestyle metrics across a diverse range of individuals. With over 850 records for the reduced, and 1500 records for the extended. It offers rich insights into the relationships between sleep patterns, physical activity, stress levels, and cardiovascular health indicators.

Dataset Features

• Person ID: Unique identifier for each individual
• Gender: Gender of the person (Male/Female)
• Age: Age of the person in years (18-80)
• Occupation: Profession of the person (15 different occupations)
• Sleep Duration: Number of hours of sleep per day (4.5-10 hours)
• Quality of Sleep: Subjective rating of sleep quality (scale: 1-10)
• Physical Activity Level: Minutes of physical activity per day (0-180 minutes)
• Stress Level: Subjective rating of stress (scale: 1-10)
• BMI Category: Body Mass Index category (Underweight, Normal, Overweight, Obese)
• Blood Pressure: Blood pressure measurement (systolic/diastolic)
• Heart Rate: Resting heart rate in beats per minute (40-120 bpm)
• Daily Steps: Number of steps walked per day (1,000-20,000)
• Sleep Disorder: Presence of sleep disorders (None, Insomnia, Sleep Apnea, Restless Leg Syndrome, Narcolepsy)

Potential Uses

This dataset is valuable for: - Analyzing correlations between physical activity and sleep quality - Exploring relationships between occupation types and stress levels - Investigating how sleep disorders relate to cardiovascular metrics - Building predictive models for sleep quality or disorder risk - Creating visualizations to understand health patterns across different demographics

Data Generation

The dataset was created by augmenting an original sample with statistically consistent synthetic data to enhance diversity and research potential. Special attention was given to creating realistic correlations between variables (e.g., higher physical activity tends to correlate with lower resting heart rates and better sleep quality).

Notes

• Sleep disorders follow realistic distributions (70% None, 10% Insomnia, 10% Sleep Apnea, 5% Restless Leg Syndrome, 5% Narcolepsy)
• Physical activity levels have been expanded to show greater variation (0-180 minutes) to enable more robust visualization and analysis
• Demographic diversity has been enhanced to reflect a wider range of ages, occupations, and gender identities

Acknowledgements

This dataset was developed for educational and research purposes. While based on patterns from real health data, individual records have been generated synthetically to ensure privacy while maintaining realistic health patterns and relationships.

Some say climate change is the biggest threat of our age while others say it’s a myth based on dodgy science. We are turning some of the data over to you so you can form your own view.

Even more than with other data sets that Kaggle has featured, there’s a huge amount of data cleaning and preparation that goes into putting together a long-time study of climate trends. Early data was collected by technicians using mercury thermometers, where any variation in the visit time impacted measurements. In the 1940s, the construction of airports caused many weather stations to be moved. In the 1980s, there was a move to electronic thermometers that are said to have a cooling bias.

Given this complexity, there are a range of organizations that collate climate trends data. The three most cited land and ocean temperature data sets are NOAA’s MLOST, NASA’s GISTEMP and the UK’s HadCrut.

We have repackaged the data from a newer compilation put together by the Berkeley Earth, which is affiliated with Lawrence Berkeley National Laboratory. The Berkeley Earth Surface Temperature Study combines 1.6 billion temperature reports from 16 pre-existing archives. It is nicely packaged and allows for slicing into interesting subsets (for example by country). They publish the source data and the code for the transformations they applied. They also use methods that allow weather observations from shorter time series to be included, meaning fewer observations need to be thrown away.

In this dataset, we have include several files:

Global Land and Ocean-and-Land Temperatures (GlobalTemperatures.csv):

• Date: starts in 1750 for average land temperature and 1850 for max and min land temperatures and global ocean and land temperatures
• LandAverageTemperature: global average land temperature in celsius
• LandAverageTemperatureUncertainty: the 95% confidence interval around the average
• LandMaxTemperature: global average maximum land temperature in celsius
• LandMaxTemperatureUncertainty: the 95% confidence interval around the maximum land temperature
• LandMinTemperature: global average minimum land temperature in celsius
• LandMinTemperatureUncertainty: the 95% confidence interval around the minimum land temperature
• LandAndOceanAverageTemperature: global average land and ocean temperature in celsius
• LandAndOceanAverageTemperatureUncertainty: the 95% confidence interval around the global average land and ocean temperature

Other files include:

• Global Average Land Temperature by Country (GlobalLandTemperaturesByCountry.csv)
• Global Average Land Temperature by State (GlobalLandTemperaturesByState.csv)
• Global Land Temperatures By Major City (GlobalLandTemperaturesByMajorCity.csv)
• Global Land Temperatures By City (GlobalLandTemperaturesByCity.csv)

The raw data comes from the Berkeley Earth data page.

The IRS publishes migration data for the US population based upon the individual tax returns filed with the IRS, where they track on a year-by-year basis

• where people were coming from - the prior state of residency
• where people moving to - the new state of residency
• number of returns filed - approximate number of households that migrated
• number of exemptions - approximate number of individuals
• the adjusted gross income (AGI) - recorded in thousands of dollars

The raw data published on the IRS website clearly shows patterns of evolution - changing patterns of what is recorded, how it is record, and naming conventions used - making it a challenge to track changes in the underlying data over time. The current dataset attempts to address these shortcomings by normalizing the record layout, standardizing the conventions, and collecting the annual into a single, coherent dataset.

An individual record is laid out with 9 fields

Y1 Y1_STATE_FIPS Y1_STATE_ABBR Y1_STATE_NAME Y2 Y2_STATE_FIPS Y2_STATE_ABBR Y2_STATE_NAME NUM_RETURNS NUM_EXEMPTIONS AGI Here, Y1 refers to the first year (from where the people are migrating) while Y2 refers to the second year (to where the people are migrating). As this is annual data, Y2 should always be the next year after Y1. Associated with each year are three different ways of identifying a state - the name of the state, it's two-letter abbreviaion, and it's FIPS code. Granted, carrying around three IDs per state is redundant; however, the various IDs are useful in different contexts. One thing to note - the IRS data represents migration into and out of the country via the introduction of a fake state, identified by STATE_NAME=FOREIGN, STATE_ABBR=FR, and STATE_FIPS=57.

From any given state, the dataset records migration to 52 destinations

• either not moving, or staying in the same state
• migrating to one of the other 49 states
• migrating to Washington DC
• migrating overseas (i.e., to the FOREIGN state)

Similarly, the dataset represents the migation into any given state as being from one of 52 destinations. Typically, the numbers associated with "staying put" constitute, by far, the largest contingent of tax payers for the given state. The one exception to this description is the FOREIGN state. The dataset does not record "staying put" outside of the country; there is no record for FOREIGN-to-FOREIGN migration. As such, there are 51, not 52, destinations paired with migration to-and-from the FOREIGN state.

Inflow and Outflow are two of the most important measures for monitoring progress towards our goal of ending homelessness. The goal is to see outflow exceed inflow. When this happens, it means that more people are exiting from homelessness than are entering the homeless-serving system.Inflow represents: the number of individuals newly identified as homeless, returned to homelessness from housing, returned to homelessness from inactive status (they have accessed the homeless-serving system within the last 90 days). Outflow represents: the number of individuals who have moved from housing to housed and those who have moved to inactive status (they have not interacted with the homeless-serving system for 90+ days). Data includes individuals who have had a shelter stay since October 2019 (the date which our homeless-serving system started using the newly released HIFIS 4.0). Data depicted is collected by homeless-serving shelters through the Homeless Individuals and Families Information System (HIFIS).

These data describe differences in percent cover of microhabitats and vegetation types at giant gartersnake locations and random points paired with giant gartersnake locations for use in case-control logistic regression. The data also include information on the individual snake, the year of observation, and whether the individual had moved from its previous location to allow for correlation of the data within individuals and locations

Annual number of interprovincial migrants by province of origin and destination, Canada, provinces and territories.