blockgroupspending Opportunity US Consumers express their behavior in a number of ways, but critically in their spending decisions. The US Bureau of Labor Statistics is charged with publishing spending activity and provides its Consumer Expenditure Survey (CEX) annually with US totals, with selected states (40) and cities (23). Limited to aggregates, the survey only needs 10s of thousands of observations in the original collection. While this is sufficient for macroeconomic use, the volume gives a weak basis for estimating lower levels of geography. In addition, the CEX includes demographic measurements that are similar, but not directly related, to Census variables. So, the CEX does not integtate well with the American Commuity Survey or other Census publications. This blockgroupspending publication by Open Environments attempts to address this problem by using the BLS' Public Microdata (PUMD) sample to allocate CEX spending categories across 220,000 US Census block group geographies. For each block group, the effort applies two models to estimate: total consumer spending (regression) distribution of spending across spending categories (penetration) including Food, Transportation, Housing and Health costs. Ultimately, these project spending on block groups that can be joined to US Census publications for additional demographics. Understanding the results requires awareness of the BLS' CEX data structures. This is available in the markdown file named oe_bls_cex_EDA.md The publication is made together with the source python code and notebooks used for repeatability. The materials are maintained under version control at https://github.com/OpenEnvironments/blockgroupspending. All feedback and development requests are welcome. Model details -- The CEX publication includes many files reflecting detailed 'diary' surveys capturing spend on thousands of items every two weeks family 'interviews' collecting household spending over the previous 3 months The models are trained upon the latter, 'FMLI' files. The regression model uses extreme gradient boosting, or XGBoost methods that apply many decision trees to iteratively correct prediction error. The subcategory models also use tree based methods, trained upon a the family interview details. The spending variables are named, following the BLS' CEX convention: |Variable|Definition|2023|pct| |---|---|---|---| |TOTEXP|Average annual expenditures|77280|| |FOOD|Food|9985|0.129| |ALCBEV|Alcoholic beverages|637|0.008| |HOUS|Housing|25436|0.329| |APPAR|Apparel and services|2041|0.026| |TRANS|Transportation|13174|0.17| |HEALTH|Healthcare|6159|0.08| |ENTERT|Entertainment|3635|0.047| |PERSCA|Personal care products and services|950|0.012| |READ|Reading|117|0.002| |EDUCA|Education|1656|0.021| |TOBACC|Tobacco products and smoking supplies|370|0.005| |MISC|Miscellaneous|1184|0.015| |CASHCO|Cash contributions|2378|0.031| |RETPEN|Personal insurance and pensions|9556|0.124| During the exploratory phase of this effort, ensemble modelling was evaluated finding that different groupings of income did not appreciably change model estimates while racial and ethnic categories did. As a result, the models are case for major races (White, African American, Asian, Other) and Hispanic. The ACS is collected by API at the block group level. Block group geographies are the lowest level of Census ACS detail and consolidate into Census tracts which in turn consolidate into counties. The FMLI responses are recorded in nominal dollars throughout the year, while total expenditure and ACS data represent year end states. As a result, the models' prediction for total expenditure is cast up using monthly inflation, weighted by monthly expenditure. Additional Caveats It is import to note, analytically, that the results are a stretch for credibility. CEX Consumer Units (people sharing financial decisions) are not exactly Census households (people in a housing unit) CEX demographics are not exactly Census demographics, with the CEX imputing incomes differenly than the Census medians. The CEX applies population weightings to the microdata while the Census primarily aggregates from respondents. The CEX observations are from 1 household (race is a 0/1 indicator) while Census demographics are many households (races are proportions) Models are trained upon repeated measures from a Consumer unit but not revised for ANOVA. Several of the CEX subcategories are very small, as spending has changed over the years. Reading, Alcohol and Tobacco use are still top level subcategories, for example as those have declined significantly since the CEX was first designed. So, this model is limited to the major subcategories of food, housing, transportation, health and retirement spending.* The model apply machine learning to large datasets so significance is not a consideration. However, in practice, those very small subcategories should be avoided. Difference in spending across racial categories also have different...

Survey of Household Spending (SHS), average household spending, Canada, regions and provinces.

The Measurable AI Rappi alternative Dataset is a leading source of email receipts and transaction data, offering data collected directly from users via Proprietary Consumer Apps, with millions of opt-in users.

We source our email receipt consumer data panel via two consumer apps which garner the express consent of our end-users (GDPR compliant). We then aggregate and anonymize all the transactional data to produce raw and aggregate datasets for our clients.

Use Cases Our clients leverage our alternative data to produce actionable consumer insights for use cases such as: - User overlap between players - Market share analysis - User behavioral traits (e.g. retention rates, spending patterns) - Average order values - Promotional strategies used by the key players - Items ordered (SKU level data) Several of our clients also use our datasets for forecasting and understanding industry trends better.

Coverage - LATAM (Brazil, Mexico, Argentina, Colombia, Chile)

Granular Data Itemized, high-definition data per transaction level with metrics such as - Order value - Items ordered - No. of orders per user - Delivery fee - Service fee - Promotions used - Geolocation data and more - MAIDs

Aggregate Data - Weekly/ monthly order volume - Revenue delivered in aggregate form, with historical data dating back to 2018. All the transactional e-receipts are sent from the Rappi food delivery app to users’ registered accounts.

Most of our clients are fast-growing Tech Companies, Financial Institutions, Buyside Firms, Market Research Agencies, Consultancies and Academia.

Our dataset is GDPR compliant, contains no PII information and is aggregated & anonymized with user consent. Contact michelle@measurable.ai for a data dictionary and to find out our volume in each country.

The Consumer Expenditure Survey (CE) program provides a continuous and comprehensive flow of data on the buying habits of American consumers, including data on their expenditures, income, and consumer unit (families and single consumers) characteristics. These data are used widely in economic research and analysis, and in support of revisions of the Consumer Price Index.

The CE program is comprised of two separate components (each with its own survey questionnaire and independent sample), the Diary Survey and the quarterly Interview Survey (ICPSR 36237). This data collection contains the Diary Survey component, which was designed to obtain data on frequently purchased smaller items, including food, housing, apparel and services, transportation, entertainment, and out-of-pocket health care costs. Each consumer unit (CU) recorded its expenditures in a diary for two consecutive 1-week periods. Although the diary was designed to collect information on expenditures that could not be easily recalled over time, respondents were asked to report all expenses (except overnight travel) that the CU incurred during the survey week.

The 2013 Diary Survey release contains five sets of data files (FMLD, MEMD, EXPD, DTBD, DTID), and one processing file (DSTUB). The FMLD, MEMD, EXPD, DTBD, and DTID files are organized by the quarter of the calendar year in which the data were collected. There are four quarterly datasets for each of these files.

The FMLD files contain CU characteristics, income, and summary level expenditures; the MEMD files contain member characteristics and income data; the EXPD files contain detailed weekly expenditures at the Universal Classification Code (UCC) level; the DTBD files contain the CU's reported annual income values or the mean of the five imputed income values in the multiple imputation method; and the DTID files contain the five imputed income values. Please note that the summary level expenditure and income information on the FMLD files permit the data user to link consumer spending, by general expenditure category, and household characteristics and demographics on one set of files.

The DSTUB file provides the aggregation scheme used in the published consumer expenditure tables. The DSTUB file is further explained in Section III.F.6. 'Processing Files' of the Diary Survey Users' Guide. A second documentation guide, the 'Users' Guide to Income Imputation,' includes information on how to appropriately use the imputed income data.

Demographic and family characteristics data include age, sex, race, marital status, and CU relationships for each CU member. Income information was also collected, such as wage, salary, unemployment compensation, child support, and alimony, as well as information on the employment of each CU member age 14 and over.

The unpublished integrated CE data tables produced by the BLS are available to download through NADAC (click on 'Other' in the Dataset(s) section). The tables show average and percentile expenditures for detailed items, as well as the standard error and coefficient of variation (CV) for each spending estimate. The BLS unpublished integrated CE data tables are provided as an easy-to-use tool for obtaining spending estimates. However, users are cautioned to read the BLS explanatory letter accompanying the tables. The letter explains that estimates of average expenditures on detailed spending items (such as leisure and art-related categories) may be unreliable due to so few reports of expenditures for those items.

Overview

The report presents updated estimates of household food expenditure trends and examines further issues relating to Australia's household food expenditure. The analysis builds on a June 2017 ABARES report that examined recent trends in food demand in Australia and a range of food security issues.

Key Issues

Between 2009-10 and 2016-17, the key drivers of Australia's household food demand growth were, in order of importance, population growth, changes in tastes and preferences (including lifestyle choices), lower real food prices and real income growth. While population growth is important, increasing the number of people seeking to meet their energy and nutrition requirements, there has also been a broadly-based shift toward spending on meals out and fast foods, with the share of meals out and fast foods in household food expenditure in Australia increasing from 31 per cent in 2009-10 to 34 per cent in 2015-16. This increases food expenditure per person, all else constant.

Domestic household consumption is still the most important market for food producers (based on value), but food exports have recovered strongly in recent years, from $25 billion in 2009-10 to $39 billion in 2016-17 (in 2015-16 prices); the share of exports in Australia's indicative food production increased from a recent low of 25 per cent in 2009-10 to 33 per cent in 2016-17.

Two key questions posed in the report relate to food security across population sub-groups and economic opportunities for farmers and other food product and service providers. • Food security-based on average outcomes in population sub-groups in 2015-16 using HES data, the Australian Government's transfer system is important in ensuring a high level of food security across households in Australia; some households, such as those highly reliant on family support payments, may require complementary support, for example, from non-government organisations.

• Economic opportunities in the domestic food supply chain-future food demand growth in Australia will be underpinned by population and income growth. For people living in higher income and/or net worth households, there is a demonstrated willingness to pay a premium for quality attributes of food products and services, including convenience factors. Food labelling is a key approach to inform consumers about quality attributes that may earn a price premium.

A key challenge in the long-term trend toward increased demand for meals out and fast foods is to ensure people have information about food attributes such as nutrition content. Reliable and well understood food product and service labelling may enhance nutrition security in Australia, and allow consumers to make food choices that are more closely aligned with their tastes and preferences (including in relation to nutrition and health), and wider circumstances, as well as contributing to reducing food waste.

Abstract

THE CLEANED AND HARMONIZED VERSION OF THE SURVEY DATA PRODUCED AND PUBLISHED BY THE ECONOMIC RESEARCH FORUM REPRESENTS 100% OF THE ORIGINAL SURVEY DATA COLLECTED BY THE PALESTINIAN CENTRAL BUREAU OF STATISTICS

The basic goal of the Household and Consumption Survey is to provide a necessary database for formulating national policies at various levels. This survey provides the contribution of the household sector to the Gross National Product (GNP). It determines the incidence of poverty, and provides weighted data which reflects the relative importance of the consumption items to be employed in determining the benchmark for rates and prices of items and services. Furthermore, this survey is a fundamental cornerstone in the process of studying the nutritional status in the Palestinian territory.

The raw survey data provided by the Statistical Agency were cleaned and harmonized by the Economic Research Forum, in the context of a major project that started in 2009. During which extensive efforts have been exerted to acquire, clean, harmonize, preserve and disseminate micro data of existing household surveys in several Arab countries.

Geographic coverage

The survey data covers urban, rural and camp areas in West Bank and Gaza Strip.

Analysis unit

1- Household/family. 2- Individual/person.

Universe

The survey covered all the Palestinian households who are a usual residence in the Palestinian Territory.

Kind of data

Sample survey data [ssd]

Sampling procedure

THE CLEANED AND HARMONIZED VERSION OF THE SURVEY DATA PRODUCED AND PUBLISHED BY THE ECONOMIC RESEARCH FORUM REPRESENTS 100% OF THE ORIGINAL SURVEY DATA COLLECTED BY THE PALESTINIAN CENTRAL BUREAU OF STATISTICS

Sample and Frame:

The sampling frame consists of all enumeration areas which were enumerated in 1997; the enumeration area consists of buildings and housing units and is composed of an average of 120 households. The enumeration areas were used as Primary Sampling Units (PSUs) in the first stage of the sampling selection. The enumeration areas of the master sample were updated in 2003.

Sample Design:

The sample is a stratified cluster systematic random sample with two stages: First stage: selection of a systematic random sample of 299 enumeration areas. Second stage: selection of a systematic random sample of 12-18 households from each enumeration area selected in the first stage. A person (18 years and more) was selected from each household in the second stage.

Sample strata:

The population was divided by: 1- Governorate 2- Type of Locality (urban, rural, refugee camps)

Sample Size:

The calculated sample size is 3,781 households.

Target cluster size:

The target cluster size or "sample-take" is the average number of households to be selected per PSU. In this survey, the sample take is around 12 households.

Detailed information/formulas on the sampling design are available in the user manual.

Mode of data collection

Face-to-face [f2f]

Research instrument

The PECS questionnaire consists of two main sections:

First section: Certain articles / provisions of the form filled at the beginning of the month,and the remainder filled out at the end of the month. The questionnaire includes the following provisions:

Cover sheet: It contains detailed and particulars of the family, date of visit, particular of the field/office work team, number/sex of the family members.

Statement of the family members: Contains social, economic and demographic particulars of the selected family.

Statement of the long-lasting commodities and income generation activities: Includes a number of basic and indispensable items (i.e, Livestock, or agricultural lands).

Housing Characteristics: Includes information and data pertaining to the housing conditions, including type of shelter, number of rooms, ownership, rent, water, electricity supply, connection to the sewer system, source of cooking and heating fuel, and remoteness/proximity of the house to education and health facilities.

Monthly and Annual Income: Data pertaining to the income of the family is collected from different sources at the end of the registration / recording period.

Second section: The second section of the questionnaire includes a list of 54 consumption and expenditure groups itemized and serially numbered according to its importance to the family. Each of these groups contains important commodities. The number of commodities items in each for all groups stood at 667 commodities and services items. Groups 1-21 include food, drink, and cigarettes. Group 22 includes homemade commodities. Groups 23-45 include all items except for food, drink and cigarettes. Groups 50-54 include all of the long-lasting commodities. Data on each of these groups was collected over different intervals of time so as to reflect expenditure over a period of one full year.

Cleaning operations

Raw Data

Both data entry and tabulation were performed using the ACCESS and SPSS software programs. The data entry process was organized in 6 files, corresponding to the main parts of the questionnaire. A data entry template was designed to reflect an exact image of the questionnaire, and included various electronic checks: logical check, range checks, consistency checks and cross-validation. Complete manual inspection was made of results after data entry was performed, and questionnaires containing field-related errors were sent back to the field for corrections.

Harmonized Data

• The Statistical Package for Social Science (SPSS) is used to clean and harmonize the datasets.
• The harmonization process starts with cleaning all raw data files received from the Statistical Agency.
• Cleaned data files are then all merged to produce one data file on the individual level containing all variables subject to harmonization.
• A country-specific program is generated for each dataset to generate/ compute/ recode/ rename/ format/ label harmonized variables.
• A post-harmonization cleaning process is then conducted on the data.
• Harmonized data is saved on the household as well as the individual level, in SPSS and converted to STATA format.

Response rate

The survey sample consists of about 3,781 households interviewed over a twelve-month period between January 2004 and January 2005. There were 3,098 households that completed the interview, of which 2,060 were in the West Bank and 1,038 households were in GazaStrip. The response rate was 82% in the Palestinian Territory.

Sampling error estimates

The calculations of standard errors for the main survey estimations enable the user to identify the accuracy of estimations and the survey reliability. Total errors of the survey can be divided into two kinds: statistical errors, and non-statistical errors. Non-statistical errors are related to the procedures of statistical work at different stages, such as the failure to explain questions in the questionnaire, unwillingness or inability to provide correct responses, bad statistical coverage, etc. These errors depend on the nature of the work, training, supervision, and conducting all various related activities. The work team spared no effort at different stages to minimize non-statistical errors; however, it is difficult to estimate numerically such errors due to absence of technical computation methods based on theoretical principles to tackle them. On the other hand, statistical errors can be measured. Frequently they are measured by the standard error, which is the positive square root of the variance. The variance of this survey has been computed by using the “programming package” CENVAR.