Abstract

The Consumer Expenditure Survey (CE) program provides a continuous and comprehensive flow of data on the buying habits of American consumers. These data are used widely in economic research and analysis, and in support of revisions of the Consumer Price Index. To meet the needs of users, the Bureau of Labor Statistics (BLS) produces population estimates for consumer units (CUs) of average expenditures in news releases, reports, issues, and articles in the Monthly Labor Review. Tabulated CE data are also available on the Internet and by facsimile transmission (See Section XV. APPENDIX 4). The microdata are available online at http://www/bls.gov/cex/pumdhome.htm. These microdata files present detailed expenditure and income data for the Diary component of the CE for 2002. They include weekly expenditure (EXPD) and annual income (DTBD) files. The data in EXPD and DTBD files are categorized by a Universal Classification Code (UCC). The advantage of the EXPD and DTBD files is that with the data classified in a standardized format, the user may perform comparative expenditure (income) analysis with relative ease. The FMLD and MEMD files present data on the characteristics and demographics of CUs and CU members. The summary level expenditure and income information on the FMLD files permits the data user to link consumer spending, by general expenditure category, and household characteristics and demographics on one set of files. Estimates of average expenditures in 2002 from the Diary survey, integrated with data from the Interview survey, are published in Consumer Expenditures in 2002. A list of recent publications containing data from the CE appears at the end of this documentation. The microdata files are in the public domain and with appropriate credit, may be reproduced without permission. A suggested citation is: "U.S. Department of Labor, Bureau of Labor Statistics, Consumer Expenditure Survey, Diary Survey, 2002".

Analysis unit

Consumer Units

Kind of data

Sample survey data [ssd]

Sampling procedure

Samples for the CE are national probability samples of households designed to be representative of the total U. S. civilian population. Eligible population includes all civilian noninstitutional persons. The first step in sampling is the selection of primary sampling units (PSUs), which consist of counties (or parts thereof) or groups of counties. The set of sample PSUs used for the 2002 sample is composed of 105 areas. The design classifies the PSUs into four categories: • 31 "A" certainty PSUs are Metropolitan Statistical Areas (MSA's) with a population greater than 1.5 million. • 46 "B" PSUs, are medium-sized MSA's. • 10 "C" PSUs are nonmetropolitan areas that are included in the CPI. • 18 "D" PSUs are nonmetropolitan areas where only the urban population data will be included in the CPI.

The sampling frame (that is, the list from which housing units were chosen) for the 2002 survey is generated from the 1990 Population Census 100-percent-detail file. The sampling frame is augmented by new construction permits and by techniques used to eliminate recognized deficiencies in census coverage. All Enumeration Districts (ED's) from the Census that fail to meet the criterion for good addresses for new construction, and all ED's in nonpermit-issuing areas are grouped into the area segment frame. To the extent possible, an unclustered sample of units is selected within each PSU. This lack of clustering is desirable because the sample size of the Diary Survey is small relative to other surveys, while the intraclass correlations for expenditure characteristics are relatively large. This suggests that any clustering of the sample units could result in an unacceptable increase in the within-PSU variance and, as a result, the total variance. Each selected sample unit is requested to keep two 1-week diaries of expenditures over consecutive weeks. The earliest possible day for placing a diary with a household is predesignated with each day of the week having an equal chance to be the first of the reference week. The diaries are evenly spaced throughout the year. During the last 6 weeks of the year, however, the Diary Survey sample is supplemented to twice its normal size to increase the reporting of types of expenditures unique to the holidays.

STATE IDENTIFIER Since the CE is not designed to produce state-level estimates, summing the consumer unit weights by state will not yield state population totals. A CU's basic weight reflects its probability of selection among a group of primary sampling units of similar characteristics. For example, sample units in an urban nonmetropolitan area in California may represent similar areas in Wyoming and Nevada. Among other adjustments, CUs are post-stratified nationally by sex-age-race. For example, the weights of consumer units containing a black male, age 16-24 in Alabama, Colorado, or New York, are all adjusted equivalently. Therefore, weighted population state totals will not match population totals calculated from other surveys that are designed to represent state data. To summarize, the CE sample was not designed to produce precise estimates for individual states. Although state-level estimates that are unbiased in a repeated sampling sense can be calculated for various statistical measures, such as means and aggregates, their estimates will generally be subject to large variances. Additionally, a particular state-population estimate from the CE sample may be far from the true state-population estimate.

INTERPRETING THE DATA Several factors should be considered when interpreting the expenditure data. The average expenditure for an item may be considerably lower than the expenditure by those CUs that purchased the item. The less frequently an item is purchased, the greater the difference between the average for all consumer units and the average of those purchasing. (See Section V.B. for ESTIMATION OF TOTAL AND MEAN EXPENDITURES). Also, an individual CU may spend more or less than the average, depending on its particular characteristics. Factors such as income, age of family members, geographic location, taste and personal preference also influence expenditures. Furthermore, even within groups with similar characteristics, the distribution of expenditures varies substantially. Expenditures reported are the direct out-of-pocket expenditures. Indirect expenditures, which may be significant, may be reflected elsewhere. For example, rental contracts often include utilities. Renters with such contracts would record no direct expense for utilities, and therefore, appear to have no utility expenses. Employers or insurance companies frequently pay other costs. CUs with members whose employers pay for all or part of their health insurance or life insurance would have lower direct expenses for these items than those who pay the entire amount themselves. These points should be considered when relating reported averages to individual circumstances.

Mode of data collection

Computer Assisted Personal Interview [capi]

The Consumer Expenditure Survey (CE) program consists of two surveys: the quarterly Interview survey and the annual Diary survey. Combined, these two surveys provide information on the buying habits of American consumers, including data on their expenditures, income, and consumer unit (families and single consumers) characteristics. The survey data are collected for the U.S. Bureau of Labor Statistics (BLS) by the U.S. Census Bureau. The CE collects all on all spending components including food, housing, apparel and services, transportation, entertainment, and out-of-pocket health care costs. The CE tables are an easy-to-use tool for obtaining arts-related spending estimates. They feature several arts-related spending categories, including the following items: Spending on Admissions Plays, theater, opera, and concerts Movies, parks, and museums Spending on Reading Newspapers and magazines Books Digital book readers Spending on Other Arts-Related Items Musical instruments Photographic equipment Audio-visual equipment Toys, games, arts and crafts The CE is important because it is the only Federal survey to provide information on the complete range of consumers' expenditures and incomes, as well as the characteristics of those consumers. It is used by economic policymakers examining the impact of policy changes on economic groups, by the Census Bureau as the source of thresholds for the Supplemental Poverty Measure, by businesses and academic researchers studying consumers' spending habits and trends, by other Federal agencies, and, perhaps most importantly, to regularly revise the Consumer Price Index market basket of goods and services and their relative importance. The most recent data tables are for 2023 and include: 1) Detailed tables with the most granular level of expenditure data available, along with variances and percent reporting for each expenditure item, for all consumer units (listed as "Other" in the Download menu); and 2) Tables with calendar year aggregate shares by demographic characteristics that provide annual aggregate expenditures and shares across demographic groups (listed as "Excel" in the Download menu). Also, see Featured CE Tables and Economic News Releases sections on the CE home page for current data tables and news release. The 1980 through 2023 CE public-use microdata, including Interview Survey data, Diary Survey data, and paradata (information about the data collection process), are available on the CE website.

blockgroupspending Opportunity US Consumers express their behavior in a number of ways, but critically in their spending decisions. The US Bureau of Labor Statistics is charged with publishing spending activity and provides its Consumer Expenditure Survey (CEX) annually with US totals, with selected states (40) and cities (23). Limited to aggregates, the survey only needs 10s of thousands of observations in the original collection. While this is sufficient for macroeconomic use, the volume gives a weak basis for estimating lower levels of geography. In addition, the CEX includes demographic measurements that are similar, but not directly related, to Census variables. So, the CEX does not integtate well with the American Commuity Survey or other Census publications. This blockgroupspending publication by Open Environments attempts to address this problem by using the BLS' Public Microdata (PUMD) sample to allocate CEX spending categories across 220,000 US Census block group geographies. For each block group, the effort applies two models to estimate: total consumer spending (regression) distribution of spending across spending categories (penetration) including Food, Transportation, Housing and Health costs. Ultimately, these project spending on block groups that can be joined to US Census publications for additional demographics. Understanding the results requires awareness of the BLS' CEX data structures. This is available in the markdown file named oe_bls_cex_EDA.md The publication is made together with the source python code and notebooks used for repeatability. The materials are maintained under version control at https://github.com/OpenEnvironments/blockgroupspending. All feedback and development requests are welcome. Model details -- The CEX publication includes many files reflecting detailed 'diary' surveys capturing spend on thousands of items every two weeks family 'interviews' collecting household spending over the previous 3 months The models are trained upon the latter, 'FMLI' files. The regression model uses extreme gradient boosting, or XGBoost methods that apply many decision trees to iteratively correct prediction error. The subcategory models also use tree based methods, trained upon a the family interview details. The spending variables are named, following the BLS' CEX convention: |Variable|Definition|2023|pct| |---|---|---|---| |TOTEXP|Average annual expenditures|77280|| |FOOD|Food|9985|0.129| |ALCBEV|Alcoholic beverages|637|0.008| |HOUS|Housing|25436|0.329| |APPAR|Apparel and services|2041|0.026| |TRANS|Transportation|13174|0.17| |HEALTH|Healthcare|6159|0.08| |ENTERT|Entertainment|3635|0.047| |PERSCA|Personal care products and services|950|0.012| |READ|Reading|117|0.002| |EDUCA|Education|1656|0.021| |TOBACC|Tobacco products and smoking supplies|370|0.005| |MISC|Miscellaneous|1184|0.015| |CASHCO|Cash contributions|2378|0.031| |RETPEN|Personal insurance and pensions|9556|0.124| During the exploratory phase of this effort, ensemble modelling was evaluated finding that different groupings of income did not appreciably change model estimates while racial and ethnic categories did. As a result, the models are case for major races (White, African American, Asian, Other) and Hispanic. The ACS is collected by API at the block group level. Block group geographies are the lowest level of Census ACS detail and consolidate into Census tracts which in turn consolidate into counties. The FMLI responses are recorded in nominal dollars throughout the year, while total expenditure and ACS data represent year end states. As a result, the models' prediction for total expenditure is cast up using monthly inflation, weighted by monthly expenditure. Additional Caveats It is import to note, analytically, that the results are a stretch for credibility. CEX Consumer Units (people sharing financial decisions) are not exactly Census households (people in a housing unit) CEX demographics are not exactly Census demographics, with the CEX imputing incomes differenly than the Census medians. The CEX applies population weightings to the microdata while the Census primarily aggregates from respondents. The CEX observations are from 1 household (race is a 0/1 indicator) while Census demographics are many households (races are proportions) Models are trained upon repeated measures from a Consumer unit but not revised for ANOVA. Several of the CEX subcategories are very small, as spending has changed over the years. Reading, Alcohol and Tobacco use are still top level subcategories, for example as those have declined significantly since the CEX was first designed. So, this model is limited to the major subcategories of food, housing, transportation, health and retirement spending.* The model apply machine learning to large datasets so significance is not a consideration. However, in practice, those very small subcategories should be avoided. Difference in spending across racial categories also have different...

Abstract

The Consumer Expenditure Survey (CE) program provides a continuous and comprehensive flow of data on the buying habits of American consumers. These data are used widely in economic research and analysis, and in support of revisions of the Consumer Price Index. To meet the needs of users, the Bureau of Labor Statistics (BLS) produces population estimates (for consumer units or CUs) of average expenditures in news releases, reports, and articles in the Monthly Labor Review. Tabulated CE data are also available on the Internet and by facsimile transmission (see Section XVI. Appendix 5). These microdata files present detailed expenditure and income data for the Diary component of the CE for 2005. They include weekly expenditure (EXPD), annual income (DTBD) files, and imputed income files (DTID). The data in EXPD, DTBD, and DTID files are categorized by a Universal Classification Code (UCC). The advantage of the EXPD and DTBD files is that with the data classified in a standardized format, the user may perform comparative expenditure (income) analysis with relative ease. The FMLD and MEMD files present data on the characteristics and demographics of CUs and CU members. The summary level expenditure and income information on the FMLD files permits the data user to link consumer spending, by general expenditure category, and household characteristics and demographics on one set of files. Estimates of average expenditures in 2005 from the Diary survey, integrated with data from the Interview survey, are published in Consumer Expenditures in 2005. A list of recent publications containing data from the CE appears at the end of this documentation.

The microdata files are in the public domain and, with appropriate credit, may be reproduced without permission. A suggested citation is: “U.S. Department of Labor, Bureau of Labor Statistics, Consumer Expenditure Survey, Diary Survey, 2005”.

State Identifier Since the CE is not designed to produce state-level estimates, summing the consumer unit weights by state will not yield state population totals. A CU's basic weight reflects its probability of selection among a group of primary sampling units of similar characteristics. For example, sample units in an urban nonmetropolitan area in California may represent similar areas in Wyoming and Nevada. Among other adjustments, CUs are post-stratified nationally by sex-age-race. For example, the weights of consumer units containing a black male, age 16-24 in Alabama, Colorado, or New York, are all adjusted equivalently. Therefore, weighted population state totals will not match population totals calculated from other surveys that are designed to represent state data.

To summarize, the CE sample was not designed to produce precise estimates for individual states. Although state-level estimates that are unbiased in a repeated sampling sense can be calculated for various statistical measures, such as means and aggregates, their estimates will generally be subject to large variances. Additionally, a particular state-population estimate from the CE sample may be far from the true state-population estimate.

Interpreting the data Several factors should be considered when interpreting the expenditure data. The average expenditure for an item may be considerably lower than the expenditure by those CUs that purchased the item. The less frequently an item is purchased, the greater the difference between the average for all consumer units and the average of those purchasing. (See Section V.B. for ESTIMATION OF TOTAL AND MEAN EXPENDITURES). Also, an individual CU may spend more or less than the average, depending on its particular characteristics. Factors such as income, age of family Members, geographic location, taste and personal preference also influence expenditures. Furthermore, even within groups with similar characteristics, the distribution of expenditures varies substantially.

Expenditures reported are the direct out-of-pocket expenditures. Indirect expenditures, which may be significant, may be reflected elsewhere. For example, rental contracts often include utilities. Renters with such contracts would record no direct expense for utilities, and therefore, appear to have no utility expenses. Employers or insurance companies frequently pay other costs.CUs with Members whose employers pay for all or part of their health insurance or life insurance would have lower direct expenses for these items than those who pay the entire amount themselves. These points should be considered when relating reported averages to individual circumstances.

The Diary survey PUMD are organized into five major data files for each quarter: 1. FMLD - a file with characteristics, income, and summary level expenditures for the household 2. MEMD - a file with characteristics and income for each member in the household
3. EXPD - a detailed weekly expenditure file categorized by UCC 4. DTBD - a detailed annual income file categorized by UCC
5. DTID - a household imputed income file categorized by UCC

Analysis unit

Consumer Unit

Kind of data

Sample survey data [ssd]

Sampling procedure

A. SURVEY SAMPLE DESIGN

Samples for the CE are national probability samples of households designed to be representative of the total U. S. civilian population. Eligible population includes all civilian noninstitutional persons.

The first step in sampling is the selection of primary sampling units (PSUs), which consist of counties (or parts thereof) or groups of counties. The set of sample PSUs used for the 2005 sample is composed of 102 areas. The design classifies the PSUs into four categories:

• 28 "A" certainty PSUs are Metropolitan Statistical Areas (MSA's) with a population greater than 1.5 million. • 42 "B" PSUs, are medium-sized MSAs. • 16 "C" PSUs are nonmetropolitan areas that are included in the CPI. • 16 "D" PSUs are nonmetropolitan areas where only the urban population data will be included in the CPI.

The sampling frame (that is, the list from which housing units were chosen) for the 2005 survey is generated from the 2000 Population Census file. The sampling frame is augmented by new construction permits and by techniques used to eliminate recognized deficiencies in census coverage. All Enumeration Districts (EDs) from the Census that fail to meet the criterion for good addresses for new construction, and all EDs in nonpermit-issuing areas are grouped into the area segment frame.

To the extent possible, an unclustered sample of units is selected within each PSU. This lack of clustering is desirable because the sample size of the Diary Survey is small relative to other surveys, while the intraclass correlations for expenditure characteristics are relatively large. This suggests that any clustering of the sample units could result in an unacceptable increase in the within-PSU variance and, as a result, the total variance.

Each selected sample unit is requested to keep two 1-week diaries of expenditures over consecutive weeks. The earliest possible day for placing a diary with a household is predesignated with each day of the week having an equal chance to be the first of the reference week. The diaries are evenly spaced throughout the year.

B. COOPERATION LEVELS

The annual target sample size at the United States level for the Diary Survey is 7,800 participating sample units. To achieve this target the total estimated work load is 11,275 sample units. This allows for refusals, vacancies, or nonexistent sample unit addresses.

Each participating sample unit selected is asked to keep two 1-week diaries. Each diary is treated independently, so response rates are based on twice the number of housing units sampled.

Mode of data collection

Computer Assisted Personal Interview [capi]

Response rate

The response rate for the 2005 Diary Survey is 68.9%. This response rate refers to all diaries in the year.

These data represent Socioeconomic Projections for the MAG Region by municipal planning area (MPA)*,adopted June 28, 2023, by the MAG Regional Council. An official set of projections is required to be used in transportation, air quality, and water quality management plans, as well as providing the base for all other regional planning activities. Current projections, therefore, are integral for managing future growth. The development of socioeconomic projections requires the collection and merging of a substantial amount of data from varying sources with differing data quality and resolution. These data include the following:Population and Housing: American Community Survey 5-year data (2017-2021),MAG Residential Completions database, County Property Assessment data, MAG/Arizona Department of Administration (ADOA) Annual Population Estimates.Group Quarters (Institutional and Non-institutional): MAG group quarters inventory.Detailed Population Characteristics: American Community Survey (ACS) Public Use Microdata Sample (PUMS) - 5-year data (2017-2021).Employment: MAG Employer Database, county level control totals developed from the Quarterly Census of Employment and Wages and Bureau of Labor Statistics (QCEW/BLS) data.Residential Completions: Current through 2022Q4, submitted and reviewed by MAG member agencies.Existing Land Use: Land use current as of December 2022, reviewed by MAG Population Technical Advisory Committee (POPTAC).Built Space: Maricopa County Assessor’s data current as of July 2022.Future Plans: General Plans current as of December 2022 or later, reviewed by MAG POPTAC.Development Data: data current as of 2023Q2 or later, reviewed by MAG POPTAC.TAZ system: TAZ2021b supplied by MAG Transportation Division.Educational institutions: Inventory of schools from Arizona Department of Education and post high school institutions.Mobile Home and RV Parks: Inventory of mobile home and RV parks.Retirement Areas: Age restricted communities reviewed by MAG POPTAC.Hotels/Motels/Resorts: Inventory of hotels/motels.For full documentation on the model process, please consult the Socioeconomic Projections Documentation: Data, Models, Methods, and Assumptions in the MAG Socioeconomic Projections 2023 on the MAG website at https://www.azmag.gov.These projections were adopted by the MAG Regional Council on June 28, 2023 for the MAG planning area. Areas outside of the MAG planning area are not adopted by the MAG Regional Council, but are prepared on behalf of Central Arizona Governments (CAG) and adopted separately.*Municipal planning areas are determined by the MAG member agencies in consultation with MAG staff. The MPAs identify the anticipated future corporate limits of a city or town.