analyze the current population survey (cps) annual social and economic supplement (asec) with r the annual march cps-asec has been supplying the statistics for the census bureau's report on income, poverty, and health insurance coverage since 1948. wow. the us census bureau and the bureau of labor statistics ( bls) tag-team on this one. until the american community survey (acs) hit the scene in the early aughts (2000s), the current population survey had the largest sample size of all the annual general demographic data sets outside of the decennial census - about two hundred thousand respondents. this provides enough sample to conduct state- and a few large metro area-level analyses. your sample size will vanish if you start investigating subgroups b y state - consider pooling multiple years. county-level is a no-no. despite the american community survey's larger size, the cps-asec contains many more variables related to employment, sources of income, and insurance - and can be trended back to harry truman's presidency. aside from questions specifically asked about an annual experience (like income), many of the questions in this march data set should be t reated as point-in-time statistics. cps-asec generalizes to the united states non-institutional, non-active duty military population. the national bureau of economic research (nber) provides sas, spss, and stata importation scripts to create a rectangular file (rectangular data means only person-level records; household- and family-level information gets attached to each person). to import these files into r, the parse.SAScii function uses nber's sas code to determine how to import the fixed-width file, then RSQLite to put everything into a schnazzy database. you can try reading through the nber march 2012 sas importation code yourself, but it's a bit of a proc freak show. this new github repository contains three scripts: 2005-2012 asec - download all microdata.R down load the fixed-width file containing household, family, and person records import by separating this file into three tables, then merge 'em together at the person-level download the fixed-width file containing the person-level replicate weights merge the rectangular person-level file with the replicate weights, then store it in a sql database create a new variable - one - in the data table 2012 asec - analysis examples.R connect to the sql database created by the 'download all microdata' progr am create the complex sample survey object, using the replicate weights perform a boatload of analysis examples replicate census estimates - 2011.R connect to the sql database created by the 'download all microdata' program create the complex sample survey object, using the replicate weights match the sas output shown in the png file below 2011 asec replicate weight sas output.png statistic and standard error generated from the replicate-weighted example sas script contained in this census-provided person replicate weights usage instructions document. click here to view these three scripts for more detail about the current population survey - annual social and economic supplement (cps-asec), visit: the census bureau's current population survey page the bureau of labor statistics' current population survey page the current population survey's wikipedia article notes: interviews are conducted in march about experiences during the previous year. the file labeled 2012 includes information (income, work experience, health insurance) pertaining to 2011. when you use the current populat ion survey to talk about america, subract a year from the data file name. as of the 2010 file (the interview focusing on america during 2009), the cps-asec contains exciting new medical out-of-pocket spending variables most useful for supplemental (medical spending-adjusted) poverty research. confidential to sas, spss, stata, sudaan users: why are you still rubbing two sticks together after we've invented the butane lighter? time to transition to r. :D

Starting in mid-July of 2020, despite many delays due to Covid-19, census takers began interviewing households who had not yet responded online or via the mail to the U.S. 2020 Census. The federal census, required by the United States’ Constitution, happens once every 10 years and each time, there are new variations in enumeration (counting) techniques and what statistical data to collect. There are processes around “how” to count and then also “what” to count; the data collected needs to be useful for governance and allocation yet also respectful of privacy and remain fair and impartial for the entire U.S. population. In 2019 and 2020, hundreds of thousands of temporary workers from local communities were hired to go out into the field as census takers as well as staff offices and provide supervision. This 22nd federal census count began in January 2020 with remote portions of Alaska, where the territory was still frozen and traversable. These employed citizens are just one aspect of how the census is truly a community event. Let’s dive into the history of the U.S. Census and also learn why this count is so important.

This feature class includes States, Counties or Boroughs, Congressional Districts, Alaska Recording Districts, County Subdivisions, and Places boundaries that are derived from the latest official Census Bureau and Alaska Department of Natural Resources datasets. Features within Forest Service Administrative Forest boundaries may have been modified by the Forest Service for improved accuracy and spatial coincidence(vertical integration).

Abstract:The 50-hectare plot at Barro Colorado Island, Panama, is a 1000 meter by 500 meter rectangle of forest inside of which all woody trees and shrubs with stems at least 1 cm in stem diameter have been censused. Every individual tree in the 50 hectares was permanently numbered with an aluminum tag in 1982, and every individual has been revisited six times since (in 1985, 1990, 1995, 2000, 2005, and 2010). In each census, every tree was measured, mapped and identified to species. Details of the census method are presented in Condit (Tropical forest census plots: Methods and results from Barro Colorado Island, Panama and a comparison with other plots; Springer-Verlag, 1998), and a description of the seven-census results in Condit, Chisholm, and Hubbell (Thirty years of forest census at Barro Colorado and the Importance of Immigration in maintaining diversity; PLoS ONE, 7:e49826, 2012).Description:CITATION TO DATABASE: Condit, R., Lao, S., Pérez, R., Dolins, S.B., Foster, R.B. Hubbell, S.P. 2012. Barro Colorado Forest Census Plot Data, 2012 Version. DOI http://dx.doi.org/10.5479/data.bci.20130603 CO-AUTHORS: Stephen Hubbell and Richard Condit have been principal investigators of the project for over 30 years. They are fully responsible for the field methods and data quality. As such, both request that data users contact them and invite them to be co-authors on publications relying on the data. More recent versions of the data, often with important updates, can be requested directly from R. Condit (conditr@gmail.com). ACKNOWLEDGMENTS: The following should be acknowledged in publications for contributions to the 50-ha plot project: R. Foster as plot founder and the first botanist able to identify so many trees in a diverse forest; R. Pérez and S. Aguilar for species identification; S. Lao for data management; S. Dolins for database design; plus hundreds of field workers for the census work, now over 2 million tree measurements; the National Science Foundation, Smithsonian Tropical Research Institute, and MacArthur Foundation for the bulk of the financial support. File 1. RoutputFull.pdf: Detailed documentation of the 'full' tables in Rdata format (File 5). File 2. RoutputStem.pdf: Detailed documentation of the 'stem' tables in Rdata format (File 7). File 3. ViewFullTable.zip: A zip archive with a single ascii text file named ViewFullTable.txt holding a table with all census data from the BCI 50-ha plot. Each row is a single measurement of a single stem, with columns indicating the census, date, species name, plus tree and stem identifiers; all seven censuses are included. A full description of all columns in the table can be found at http://dx.doi.org/10.5479/data.bci.20130604 (ViewFullTable, pp. 21-22 of the pdf). File 4. ViewTax.txt: An ascii text table with information on all tree species recorded in the 50-ha plot. There are columns with taxonomics names (family, genus, species, and subspecies), plus the taxonomic authority. The column 'Mnemonic' gives a shortened code identifying each species, a code used in the R tables (Files 5, 7). The column 'IDLevel' indicates the depth to which the species is identified: if IDLevel='species', it is a fully identified, but if IDLevel='genus', the genus is known but not the species. IDLevel can also be 'family', or 'none' in case the species is not even known to family. File 5. bci.full.Rdata31Aug2012.zip: A zip archive holding seven R Analytical Tables, versions of the BCI 50 ha plot census data in R format. These are designed for data analysis. There are seven files, one for each of the 7 censuses: 'bci.full1.rdata' for the first census through 'bci.full7.rdata' for the seventh census. Each of the seven files is a table having one record per individual tree, and each includes a record for every tree found over the entire seven censuses (i.e. whether or not they were observed alive in the given census, there is a record). Detailed documentation of these tables is given in RoutputFull.pdf (File 1). File 6. bci.spptable.rdata: A list of the 1064 species found across all tree plots and inventories in Panama, in R format. This is a superset of species found in the BCI censuses: every BCI species is included, plus additional species never observed at BCI. The column 'sp' in this table is a code identifying the species in the R census tables (File 5, 7), and matching 'mnemomic' in ViewFullTable (File 3). File 7. bci.stem.Rdata31Aug2012.zip: A zip archive holding seven R Analytical Tables, versions of the BCI 50 ha plot census data in R format. These are designed for data analysis. There are seven files, one for each of the 7 censuses: 'bci.stem1.rdata' for the first census through 'bci.stem7.rdata' for the seventh census. Each of the seven files is a table having one record per individual stem, necessary because some individual trees have more than one stem. Each includes a record for every stem found over the entire seven censuses (i.e. whether or not they were observed alive in the given census, there is a record). Detailed documentation of these tables is given in RoutputStem.pdf (File 2). File 8. TSMAttributes.txt: An ascii text table giving full descriptions of measurement codes, which are also referred to as TSMCodes. These short codes are used in the column 'code' in R tables and in the column 'ListOfTSM' in ViewFullTable.txt, in both cases with individual codes separated by commas. File 9. bci_31August2012_mysql.zip: A zip archive holding one file, 'bci.sql', which is a mysqldump of the complete MySQL database (version 5.0.95, http://www.mysql.com) created 31 August 2012. The database includes data collected from seven censuses of the BCI 50 ha plot plus censuses of many additional plots elsewhere in Panama, plus transects where only species identifications were collected and trees were not tagged nor measurements made. Detailed documentation of all tables within the database can be found at (http://dx.doi.org/10.5479/data.bci.20130604). This version of the data is intended for experienced SQL users; for most, the R Analytical Tables in Rtables.zip are more useful.

2011 India Census data at the State, District, and Subdistrict levels (including workforce data)

Abstract: These are results from a network of 65 tree census plots in Panama. At each, every individual stem in a rectangular area of specified size is given a unique number and identified to species, then stem diameter measured in one or more censuses. Data from these numerous plots and inventories were collected following the same methods as, and species identity harmonized with, the 50-ha long-term tree census at Barro Colorado Island. Precise location of every site, elevation, and estimated rainfall (for many sites) are also included. These data were gathered over many years, starting in 1994 and continuing to the present, by principal investigators R. Condit, R. Perez, S. Lao, and S. Aguilar. Funding has been provided by many organizations.Description:marenaRecent.full.Rdata5Jan2013.zip: A zip archive holding one R Analytical Table, a version of the Marena plots' census data in R format, designed for data analysis. This and all other tables labelled 'full' have one record per individual tree found in that census. Detailed documentations of the 'full' tables is given in RoutputFull.pdf (see component 10 below); an additional column 'plot' is included because the table includes records from many different locations. Plot coordinates are given in PanamaPlot.txt (component 12 below). This one file, 'marenaRecent.full1.rdata', has data from the latest census at 60 different plots. These are the best data to use if only a single plot census is needed. marena2cns.full.Rdata5Jan2013.zip: R Analytical Tables of the style 'full' for 44 plots with two censuses: 'marena2cns.full1.rdata' for the first census and 'marena2cns.full2.rdata' for the second census. These 44 plots are a subset of the 60 found in marenaRecent.full (component 1): the 44 that have been censused two or more times. These are the best data to use if two plot censuses are needed. marena3cns.full.Rdata5Jan2013.zip. R Analytical Tables of the style 'full' for nine plots with three censuses: 'marena3cns.full1.rdata' for the first census through 'marena2cns.full3.rdata' for the third census. These nine plots are a subset of the 44 found in marena2cns.full (component 2): the nine that have been censused three or more times. These are the best data to use if three plot censuses are needed. marena4cns.full.Rdata5Jan2013.zip. R Analytical Tables of the style 'full' for six plots with four censuses: 'marena4cns.full1.rdata' for the first census through 'marena4cns.full4.rdata' for the fourth census. These six plots are a subset of the nine found in marena3cns.full (component 3): the six that have been censused four or more times. These are the best data to use if four plot censuses are needed. marenaRecent.stem.Rdata5Jan2013.zip. A zip archive holding one R Analytical Table, a version of the Marena plots' census data in R format. These are designed for data analysis. This one file, 'marenaRecent.full1.rdata', has data from the latest census at 60 different plots. The table has one record per individual stem, necessary because some individual trees have more than one stem. Detailed documentations of these tables is given in RoutputFull.pdf (see component 11 below); an additional column 'plot' is included because the table includes records from many different locations. Plot coordinates are given in PanamaPlot.txt (component 12 below). These are the best data to use if only a single plot census is needed, and individual stems are desired. marena2cns.stem.Rdata5Jan2013.zip. R Analytical Tables of the style 'stem' for 44 plots with two censuses: 'marena2cns.stem1.rdata' for the first census and 'marena3cns.stem2.rdata' for the second census. These 44 plots are a subset of the 60 found in marenaRecent.stem (component 1): the 44 that have been censused two or more times. These are the best data to use if two plot censuses are needed, and individual stems are desired. marena3cns.stem.Rdata5Jan2013.zip. R Analytical Tables of the style 'stem' for nine plots with three censuses: 'marena3cns.stem1.rdata' for the first census through 'marena3cns.stem3.rdata' for the third census. These nine plots are a subset of the 44 found in marena2cns.stem (component 6): the nine that have been censused three or more times. These are the best data to use if three plot censuses are needed, and individual stems are desired. marena4cns.stem.Rdata5Jan2013.zip. R Analytical Tables of the style 'stem' for six plots with four censuses: 'marena3cns.stem1.rdata' for the first census through 'marena3cns.stem3.rdata' for the third census. These six plots are a subset of the nine found in marena3cns.stem (component 7): the six that have been censused four or more times. These are the best data to use if four plot censuses are needed, and individual stems are desired. bci.spptable.rdata. A list of the 1414 species found across all tree plots and inventories in Panama, in R format. The column 'sp' in this table is a code identifying the species in the full census tables (marena.full and marena.stem, components 1-4 and 5-8 above). RoutputFull.pdf: Detailed documentation of the 'full' tables in Rdata format (components 1-4 above). RoutputStem.pdf: Detailed documentation of the 'stem' tables in Rdata format (component 5-8 above). PanamaPlot.txt: Locations of all tree plots and inventories in Panama.

The composition of the scientific workforce shapes the direction of scientific research, directly through the selection of questions to investigate, and indirectly through its influence on the training of future scientists. In most fields, however, complete census information is difficult to obtain, complicating efforts to study workforce dynamics and the effects of policy. This is particularly true in computer science, which lacks a single, all-encompassing directory or professional organization. A full census of computer science would serve many purposes, not the least of which is a better understanding of the trends and causes of unequal representation in computing. Previous academic census efforts have relied on narrow or biased samples, or on professional society membership rolls. A full census can be constructed directly from online departmental faculty directories, but doing so by hand is expensive and time-consuming. Here, we introduce a topical web crawler for automating the collection of faculty information from web-based department rosters, and demonstrate the resulting system on the 205 PhD-granting computer science departments in the U.S. and Canada. This method can quickly construct a complete census of the field, and achieve over 99% precision and recall. We conclude by comparing the resulting 2017 census to a hand-curated 2011 census to quantify turnover and retention in computer science, in general and for female faculty in particular, demonstrating the types of analysis made possible by automated census construction.

The data sets in this repository allow users to link people among the U.S. decennial censuses, using the "histid" identifier. The census data sets users will need are indexed by Ancestry.com and are hosted by IPUMS at https://usa.ipums.org/usa-action/samples. Users will need to download the full-count census for each year and be sure to select the "histid" variable that is available under the Person/Historical Technical drop-down menu.As of 7/12/21, links are available between the 1900-1910, 1910-1920, and 1900-1920 censuses.A detailed account of how these links are created and a description of the data and its characteristics are available in the following article:Price, J., Buckles, K., Van Leeuwen, J., & Riley, I. (2021). Combining family history and machine learning to link historical records: The Census Tree data set. Explorations in Economic History, 80, 101391.https://www.sciencedirect.com/science/article/pii/S0014498321000024

PARES Dataset v2 PARES (PArish REcord Survey) contains 535 images of handwritten census tables for years ranging from around 1650 A.D. until 1850 A.D..They come from two French cities, Vic-sur-Seille (French department of Moselle) and Echevronne (French department of Côte d'Or). While they mention very ancient times, the documents are handwritten transcriptions of even older documents and are quite recent, copied from original documents during the 1950's and 1960's for demographic studies led by the INED in France (Institut National des études démographiques − National Institute for Demographic Studies). These copies were made by only a few different writers. The documents are damaged and exhibit different types of degradations. We identified seven different document categories we call C1 to C7. C1 and C3 are generally high-quality documents, without serious damage, consisting of about 90% of the dataset. Other categories include highly damaged documents or documents with specificities. A notable aspect of this dataset is that the records are written using only two different physical paper templates. Categories n°1, 2, 3, 6 and 7 have 25 recordings while the categories 4 and 5 are higher and can record up to 35 recordings. C4 and C5 are the larger templates and differ from the rest of the documents. We published a paper, Text Line Detection in Historical Index Tables: Evaluations on a New French PArish REcord Survey Dataset (PARES), in which we better describe the dataset and the tasks it's possible to run on it.

The Current Population Survey Food Security Supplement (CPS-FSS) is the source of national and State-level statistics on food insecurity used in USDA's annual reports on household food security. The CPS is a monthly labor force survey of about 50,000 households conducted by the Census Bureau for the Bureau of Labor Statistics. Once each year, after answering the labor force questions, the same households are asked a series of questions (the Food Security Supplement) about food security, food expenditures, and use of food and nutrition assistance programs. Food security data have been collected by the CPS-FSS each year since 1995. Four data sets that complement those available from the Census Bureau are available for download on the ERS website. These are available as ASCII uncompressed or zipped files. The purpose and appropriate use of these additional data files are described below: 1) CPS 1995 Revised Food Security Status data--This file provides household food security scores and food security status categories that are consistent with procedures and variable naming conventions introduced in 1996. This includes the "common screen" variables to facilitate comparisons of prevalence rates across years. This file must be matched to the 1995 CPS Food Security Supplement public-use data file. 2) CPS 1998 Children's and 30-day Food Security data--Subsequent to the release of the April 1999 CPS-FSS public-use data file, USDA developed two additional food security scales to describe aspects of food security conditions in interviewed households not captured by the 12-month household food security scale. This file provides three food security variables (categorical, raw score, and scale score) for each of these scales along with household identification variables to allow the user to match this supplementary data file to the CPS-FSS April 1998 data file. 3) CPS 1999 Children's and 30-day Food Security data--Subsequent to the release of the April 1999 CPS-FSS public-use data file, USDA developed two additional food security scales to describe aspects of food security conditions in interviewed households not captured by the 12-month household food security scale. This file provides three food security variables (categorical, raw score, and scale score) for each of these scales along with household identification variables to allow the user to match this supplementary data file to the CPS-FSS April 1999 data file. 4) CPS 2000 30-day Food Security data--Subsequent to the release of the September 2000 CPS-FSS public-use data file, USDA developed a revised 30-day CPS Food Security Scale. This file provides three food security variables (categorical, raw score, and scale score) for the 30-day scale along with household identification variables to allow the user to match this supplementary data file to the CPS-FSS September 2000 data file. Food security is measured at the household level in three categories: food secure, low food security and very low food security. Each category is measured by a total count and as a percent of the total population. Categories and measurements are broken down further based on the following demographic characteristics: household composition, race/ethnicity, metro/nonmetro area of residence, and geographic region. The food security scale includes questions about households and their ability to purchase enough food and balanced meals, questions about adult meals and their size, frequency skipped, weight lost, days gone without eating, questions about children meals, including diversity, balanced meals, size of meals, skipped meals and hunger. Questions are also asked about the use of public assistance and supplemental food assistance. The food security scale is 18 items that measure insecurity. A score of 0-2 means a house is food secure, from 3-7 indicates low food security, and 8-18 means very low food security. The scale and the data also report the frequency with which each item is experienced. Data are available as .dat files which may be processed in statistical software or through the United State Census Bureau's DataFerret http://dataferrett.census.gov/. Data from 2010 onwards is available below and online. Data from 1995-2009 must be accessed through DataFerrett. DataFerrett is a data analysis and extraction tool to customize federal, state, and local data to suit your requirements. Through DataFerrett, the user can develop an unlimited array of customized spreadsheets that are as versatile and complex as your usage demands then turn those spreadsheets into graphs and maps without any additional software. Resources in this dataset:Resource Title: December 2014 Food Security CPS Supplement. File Name: dec14pub.zipResource Title: December 2013 Food Security CPS Supplement. File Name: dec13pub.zipResource Title: December 2012 Food Security CPS Supplement. File Name: dec12pub.zipResource Title: December 2011 Food Security CPS Supplement. File Name: dec11pub.zipResource Title: December 2010 Food Security CPS Supplement. File Name: dec10pub.zip

Note: This version supersedes version 1: https://doi.org/10.15482/USDA.ADC/1522654. In Fall of 2019 the USDA Food and Nutrition Service (FNS) conducted the third Farm to School Census. The 2019 Census was sent via email to 18,832 school food authorities (SFAs) including all public, private, and charter SFAs, as well as residential care institutions, participating in the National School Lunch Program. The questionnaire collected data on local food purchasing, edible school gardens, other farm to school activities and policies, and evidence of economic and nutritional impacts of participating in farm to school activities. A total of 12,634 SFAs completed usable responses to the 2019 Census. Version 2 adds the weight variable, “nrweight”, which is the Non-response weight. Processing methods and equipment used The 2019 Census was administered solely via the web. The study team cleaned the raw data to ensure the data were as correct, complete, and consistent as possible. This process involved examining the data for logical errors, contacting SFAs and consulting official records to update some implausible values, and setting the remaining implausible values to missing. The study team linked the 2019 Census data to information from the National Center of Education Statistics (NCES) Common Core of Data (CCD). Records from the CCD were used to construct a measure of urbanicity, which classifies the area in which schools are located. Study date(s) and duration Data collection occurred from September 9 to December 31, 2019. Questions asked about activities prior to, during and after SY 2018-19. The 2019 Census asked SFAs whether they currently participated in, had ever participated in or planned to participate in any of 30 farm to school activities. An SFA that participated in any of the defined activities in the 2018-19 school year received further questions. Study spatial scale (size of replicates and spatial scale of study area) Respondents to the survey included SFAs from all 50 States as well as American Samoa, Guam, the Northern Mariana Islands, Puerto Rico, the U.S. Virgin Islands, and Washington, DC. Level of true replication Unknown Sampling precision (within-replicate sampling or pseudoreplication) No sampling was involved in the collection of this data. Level of subsampling (number and repeat or within-replicate sampling) No sampling was involved in the collection of this data. Study design (before–after, control–impacts, time series, before–after-control–impacts) None – Non-experimental Description of any data manipulation, modeling, or statistical analysis undertaken Each entry in the dataset contains SFA-level responses to the Census questionnaire for SFAs that responded. This file includes information from only SFAs that clicked “Submit” on the questionnaire. (The dataset used to create the 2019 Farm to School Census Report includes additional SFAs that answered enough questions for their response to be considered usable.) In addition, the file contains constructed variables used for analytic purposes. The file does not include weights created to produce national estimates for the 2019 Farm to School Census Report. The dataset identified SFAs, but to protect individual privacy the file does not include any information for the individual who completed the questionnaire. Description of any gaps in the data or other limiting factors See the full 2019 Farm to School Census Report [https://www.fns.usda.gov/cfs/farm-school-census-and-comprehensive-review] for a detailed explanation of the study’s limitations. Outcome measurement methods and equipment used None Resources in this dataset:Resource Title: 2019 Farm to School Codebook with Weights. File Name: Codebook_Update_02SEP21.xlsxResource Description: 2019 Farm to School Codebook with WeightsResource Title: 2019 Farm to School Data with Weights CSV. File Name: census2019_public_use_with_weight.csvResource Description: 2019 Farm to School Data with Weights CSVResource Title: 2019 Farm to School Data with Weights SAS R Stata and SPSS Datasets. File Name: Farm_to_School_Data_AgDataCommons_SAS_SPSS_R_STATA_with_weight.zipResource Description: 2019 Farm to School Data with Weights SAS R Stata and SPSS Datasets

The dataset used in the paper is a US census tract information dataset from the American Community Survey (ACS) 5-year summary files for the 2010 to 2014 period.

This dataset contains metadata (title, abstract, date of publication, field, etc) for around 1 million academic articles. Each record contains additional information on the country of study and whether the article makes use of data. Machine learning tools were used to classify the country of study and data use.

Our data source of academic articles is the Semantic Scholar Open Research Corpus (S2ORC) (Lo et al. 2020). The corpus contains more than 130 million English language academic papers across multiple disciplines. The papers included in the Semantic Scholar corpus are gathered directly from publishers, from open archives such as arXiv or PubMed, and crawled from the internet.

We placed some restrictions on the articles to make them usable and relevant for our purposes. First, only articles with an abstract and parsed PDF or latex file are included in the analysis. The full text of the abstract is necessary to classify the country of study and whether the article uses data. The parsed PDF and latex file are important for extracting important information like the date of publication and field of study. This restriction eliminated a large number of articles in the original corpus. Around 30 million articles remain after keeping only articles with a parsable (i.e., suitable for digital processing) PDF, and around 26% of those 30 million are eliminated when removing articles without an abstract. Second, only articles from the year 2000 to 2020 were considered. This restriction eliminated an additional 9% of the remaining articles. Finally, articles from the following fields of study were excluded, as we aim to focus on fields that are likely to use data produced by countries’ national statistical system: Biology, Chemistry, Engineering, Physics, Materials Science, Environmental Science, Geology, History, Philosophy, Math, Computer Science, and Art. Fields that are included are: Economics, Political Science, Business, Sociology, Medicine, and Psychology. This third restriction eliminated around 34% of the remaining articles. From an initial corpus of 136 million articles, this resulted in a final corpus of around 10 million articles.

Due to the intensive computer resources required, a set of 1,037,748 articles were randomly selected from the 10 million articles in our restricted corpus as a convenience sample.

The empirical approach employed in this project utilizes text mining with Natural Language Processing (NLP). The goal of NLP is to extract structured information from raw, unstructured text. In this project, NLP is used to extract the country of study and whether the paper makes use of data. We will discuss each of these in turn.

To determine the country or countries of study in each academic article, two approaches are employed based on information found in the title, abstract, or topic fields. The first approach uses regular expression searches based on the presence of ISO3166 country names. A defined set of country names is compiled, and the presence of these names is checked in the relevant fields. This approach is transparent, widely used in social science research, and easily extended to other languages. However, there is a potential for exclusion errors if a country’s name is spelled non-standardly.

The second approach is based on Named Entity Recognition (NER), which uses machine learning to identify objects from text, utilizing the spaCy Python library. The Named Entity Recognition algorithm splits text into named entities, and NER is used in this project to identify countries of study in the academic articles. SpaCy supports multiple languages and has been trained on multiple spellings of countries, overcoming some of the limitations of the regular expression approach. If a country is identified by either the regular expression search or NER, it is linked to the article. Note that one article can be linked to more than one country.

The second task is to classify whether the paper uses data. A supervised machine learning approach is employed, where 3500 publications were first randomly selected and manually labeled by human raters using the Mechanical Turk service (Paszke et al. 2019).[1] To make sure the human raters had a similar and appropriate definition of data in mind, they were given the following instructions before seeing their first paper:

Each of these documents is an academic article. The goal of this study is to measure whether a specific academic article is using data and from which country the data came.

There are two classification tasks in this exercise:

1. identifying whether an academic article is using data from any country

2. Identifying from which country that data came.

For task 1, we are looking specifically at the use of data. Data is any information that has been collected, observed, generated or created to produce research findings. As an example, a study that reports findings or analysis using a survey data, uses data. Some clues to indicate that a study does use data includes whether a survey or census is described, a statistical model estimated, or a table or means or summary statistics is reported.

After an article is classified as using data, please note the type of data used. The options are population or business census, survey data, administrative data, geospatial data, private sector data, and other data. If no data is used, then mark "Not applicable". In cases where multiple data types are used, please click multiple options.[2]

For task 2, we are looking at the country or countries that are studied in the article. In some cases, no country may be applicable. For instance, if the research is theoretical and has no specific country application. In some cases, the research article may involve multiple countries. In these cases, select all countries that are discussed in the paper.

We expect between 10 and 35 percent of all articles to use data.

The median amount of time that a worker spent on an article, measured as the time between when the article was accepted to be classified by the worker and when the classification was submitted was 25.4 minutes. If human raters were exclusively used rather than machine learning tools, then the corpus of 1,037,748 articles examined in this study would take around 50 years of human work time to review at a cost of $3,113,244, which assumes a cost of $3 per article as was paid to MTurk workers.

A model is next trained on the 3,500 labelled articles. We use a distilled version of the BERT (bidirectional Encoder Representations for transformers) model to encode raw text into a numeric format suitable for predictions (Devlin et al. (2018)). BERT is pre-trained on a large corpus comprising the Toronto Book Corpus and Wikipedia. The distilled version (DistilBERT) is a compressed model that is 60% the size of BERT and retains 97% of the language understanding capabilities and is 60% faster (Sanh, Debut, Chaumond, Wolf 2019). We use PyTorch to produce a model to classify articles based on the labeled data. Of the 3,500 articles that were hand coded by the MTurk workers, 900 are fed to the machine learning model. 900 articles were selected because of computational limitations in training the NLP model. A classification of “uses data” was assigned if the model predicted an article used data with at least 90% confidence.

The performance of the models classifying articles to countries and as using data or not can be compared to the classification by the human raters. We consider the human raters as giving us the ground truth. This may underestimate the model performance if the workers at times got the allocation wrong in a way that would not apply to the model. For instance, a human rater could mistake the Republic of Korea for the Democratic People’s Republic of Korea. If both humans and the model perform the same kind of errors, then the performance reported here will be overestimated.

The model was able to predict whether an article made use of data with 87% accuracy evaluated on the set of articles held out of the model training. The correlation between the number of articles written about each country using data estimated under the two approaches is given in the figure below. The number of articles represents an aggregate total of

About this dataset

Context

The dataset tabulates the population of North Carolina by race. It includes the population of North Carolina across racial categories (excluding ethnicity) as identified by the Census Bureau. The dataset can be utilized to understand the population distribution of North Carolina across relevant racial categories.

Key observations

The percent distribution of North Carolina population by race (across all racial categories recognized by the U.S. Census Bureau): 66.22% are white, 21.16% are Black or African American, 1.10% are American Indian and Alaska Native, 3.04% are Asian, 0.07% are Native Hawaiian and other Pacific Islander, 3.59% are some other race and 4.83% are multiracial.

https://i.neilsberg.com/ch/north-carolina-population-by-race.jpeg" alt="North Carolina population by race">

Content

When available, the data consists of estimates from the U.S. Census Bureau American Community Survey (ACS) 2017-2021 5-Year Estimates.

Racial categories include:

• White
• Black or African American
• American Indian and Alaska Native
• Asian
• Native Hawaiian and Other Pacific Islander
• Some other race
• Two or more races (multiracial)

Variables / Data Columns

• Race: This column displays the racial categories (excluding ethnicity) for the North Carolina
• Population: The population of the racial category (excluding ethnicity) in the North Carolina is shown in this column.
• % of Total Population: This column displays the percentage distribution of each race as a proportion of North Carolina total population. Please note that the sum of all percentages may not equal one due to rounding of values.

Good to know

Margin of Error

Data in the dataset are based on the estimates and are subject to sampling variability and thus a margin of error. Neilsberg Research recommends using caution when presening these estimates in your research.

Custom data

If you do need custom data for any of your research project, report or presentation, you can contact our research staff at research@neilsberg.com for a feasibility of a custom tabulation on a fee-for-service basis.

Inspiration

Neilsberg Research Team curates, analyze and publishes demographics and economic data from a variety of public and proprietary sources, each of which often includes multiple surveys and programs. The large majority of Neilsberg Research aggregated datasets and insights is made available for free download at https://www.neilsberg.com/research/.

Recommended for further research

This dataset is a part of the main dataset for North Carolina Population by Race & Ethnicity. You can refer the same here

The 50 ha plot at Barro Colorado Island, Panama (utm: easting 625754, northing 1011569, zone 17), completed 4 censuses: 1981-83, 1985, 1990, and 1995. All trees >=10 mm dbh were tagged, measured, mapped and identified to species. The tab-delimited text files consist of a main census file with the location and diameter measurements, 4 multiple-stem files with the multiple stem measurements (including the largest measurement in the main database), the species list, and measurements of so-called "big" trees (i.e. trees with buttresses where the diameter was taken at a height higher than 1.3 m).

Imports - Paper & Paper Products, N.E.S.(Census Basis) in the United States increased to 817.46 USD Million in February from 812.09 USD Million in January of 2024. This dataset includes a chart with historical data for the United States Imports of Paper & Paper Products, N.e.s.(census.

This dataset contains the digitized treatments in Plazi based on the original journal article OLIVEIRA, OTTO M. P., MIRANDA, THAÍS P., ARAUJO, ENILMA M., AYÓN, PATRICIA, CEDEÑO-POSSO, CRISTINA M., CEPEDA-MERCADO, AMANCAY A., CÓRDOVA, PABLO, CUNHA, AMANDA F., GENZANO, GABRIEL N., HADDAD, MARIA ANGÉLICA, MIANZAN, HERMES W., MIGOTTO, ALVARO E., MIRANDA, LUCÍLIA S., MORANDINI, ANDRÉ C., NAGATA, RENATO M., NASCIMENTO, KARINE B., JÚNIOR, MIODELI NOGUEIRA, PALMA, SERGIO, QUIÑONES, JAVIER, RODRIGUEZ, CAROLINA S., SCARABINO, FABRIZIO, SCHIARITI, AGUSTÍN, STAMPAR, SÉRGIO N., TRONOLONE, VALQUÍRIA B., MARQUES, ANTONIO C. (2016): Census of Cnidaria (Medusozoa) and Ctenophora from South American marine waters. Zootaxa 4194 (1): 1-256, DOI: 10.11646/zootaxa.4194.1.1, URL: http://dx.doi.org/10.11646/zootaxa.4194.1.1

Exports - Newsprint & Other Paper Products (Census Basis) in the United States increased to 1095.91 USD Million in February from 1048.57 USD Million in January of 2024. This dataset includes a chart with historical data for the United States Exports of Newsprint & Other Paper Products.

The Cape Shirreff Phocid Census (CS-PHOC) dataset is part of long-term monitoring efforts at Cape Shirreff, Livingston Island. The National Oceanic and Atmospheric Administration (NOAA) United States Antarctic Marine Living Resources Program (U.S. AMLR) and the Chilean Antarctic Institute (INACH) have conducted synoptic, weekly counts of Southern Ocean phocids hauled out on Cape Shirreff during most austral summers since 1997-98. These census data, which will continue to be collected by the U.S. AMLR program and thus updated yearly, provide a rare and valuable source of information about changes in population trends and area use by Southern Ocean phocids in a climate change hot spot. CS-PHOC is a sampling event type dataset published as open data with technical support provided by SCAR Antarctic Biodiversity Portal (biodiversity.aq) (BELSPO project RT/23/ADVANCE). This dataset is described in the paper “CS-PHOC: weekly census counts of Southern Ocean phocids at Cape Shirreff, Livingston Island” (Woodman et al., 2024). This dataset contains records of Hydrurga leptonyx, Leptonychotes weddellii, Lobodon carcinophagus, and Mirounga leonina census counts at Cape Shirreff, Livingston Island (62.47° S, 60.77° W). All census records were collected by field biologists using binoculars during field expeditions at Cape Shirreff in the austral summers from December 1997 to February 2023. The data is published as a standardized Darwin Core Archive, which contains presence, absence, sex and life stage of Southern Ocean phocids observed in each survey. This dataset is published under the license CC0 1.0. Please follow the guidelines from the SCAR Data Policy (SCAR, 2023) when using the data. A manuscript describing the CS-PHOC dataset is currently in review; if you are interested in the project or have any questions regarding this dataset, please contact us via the contact information provided in the metadata or via data-biodiversity-aq@naturalsciences.be. Issues with dataset can be reported at https://github.com/us-amlr/cs-phoc This dataset is part of the U.S. Antarctic Marine Living Resources program funded by NOAA.

Two sample datasets created for training purposes from The Squirrel Census https://www.thesquirrelcensus.com/ hosted by NYC OpenData 2018 Central Park Squirrel Census - Squirrel Data | NYC Open Data, no reuse license specified.