This data and code can be used to replicate the main analysis for "Who Exhibits Cognitive Biases? Mapping Heterogeneity in Attention, Interpretation, and Rumination in Depression." Of note- to protect this dataset from deidentification consistent with best practices, we have removed the zip code variable and binned age. The analysis code may need to be adjusted slightly to account for this, and the results may very slightly from the ones in the manuscript as a result.

Observer bias and other “experimenter effects” occur when researchers’ expectations influence study outcome. These biases are strongest when researchers expect a particular result, are measuring subjective variables, and have an incentive to produce data that confirm predictions. To minimize bias, it is good practice to work “blind,” meaning that experimenters are unaware of the identity or treatment group of their subjects while conducting research. Here, using text mining and a literature review, we find evidence that blind protocols are uncommon in the life sciences and that nonblind studies tend to report higher effect sizes and more significant p-values. We discuss methods to minimize bias and urge researchers, editors, and peer reviewers to keep blind protocols in mind.

Usage Notes Evolution literature review dataExact p value datasetjournal_categoriesp values data 24 SeptProportion of significant p values per paperR script to filter and classify the p value dataQuiz answers - guessing effect size from abstractsThe answers provided by the 9 evolutionary biologists to quiz we designed, which aimed to test whether trained specialists are able to infer the relative size/direction of effect size from a paper's title and abstract.readmeDescription of the contents of all the other files in this Dryad submission.R script to statistically analyse the p value dataR script detailing the statistical analyses we performed on the p value datasets.

Bias Mitigation Tools AI Market Outlook

According to our latest research, the global Bias Mitigation Tools AI market size reached USD 412.6 million in 2024, reflecting a robust demand for AI fairness solutions across various sectors. The market is projected to grow at a CAGR of 26.8% from 2025 to 2033, reaching an estimated USD 3.27 billion by 2033. This remarkable growth is being driven by increasing regulatory scrutiny, heightened awareness of algorithmic bias, and the necessity for transparent, ethical AI systems in mission-critical applications. The surge in AI adoption across industries such as finance, healthcare, government, and retail is further propelling the market, as organizations strive to ensure compliance and build trust in AI-driven decision-making.

One of the primary growth factors in the Bias Mitigation Tools AI market is the mounting regulatory pressure from governments and international bodies to address algorithmic bias and promote responsible AI usage. Regulatory frameworks such as the European UnionÂ’s AI Act and the US Algorithmic Accountability Act have set new compliance benchmarks, compelling organizations to integrate bias mitigation tools into their AI development pipelines. These regulations not only mandate transparency and accountability but also require demonstrable efforts to identify, measure, and reduce bias in automated systems. As a result, enterprises are increasingly seeking advanced AI solutions capable of detecting and correcting biases at every stage of the model lifecycle, from data preprocessing to post-deployment monitoring.

Another significant driver for market expansion is the growing recognition among enterprises of the reputational and operational risks associated with biased AI systems. High-profile incidents of discriminatory algorithms in areas such as lending, recruitment, and healthcare have underscored the need for robust bias mitigation strategies. Organizations are now prioritizing fairness and inclusivity as core objectives in their digital transformation journeys. This shift is fueling investments in AI fairness tools that offer explainability, auditability, and continuous monitoring capabilities. The integration of these tools is not only a compliance imperative but also a competitive differentiator, as businesses aim to foster consumer trust and brand loyalty by demonstrating ethical AI practices.

Technological advancements and the proliferation of AI across diverse applications are also catalyzing the adoption of bias mitigation tools. Innovations in machine learning, natural language processing, and computer vision have expanded the use cases for AI, but they have also introduced new challenges related to data quality, representativeness, and unintended consequences. Modern bias mitigation solutions leverage advanced algorithms, synthetic data generation, and automated auditing to address these complexities. Additionally, the rise of cloud-based AI platforms has democratized access to fairness tools, enabling organizations of all sizes to deploy scalable and cost-effective solutions. As AI becomes more embedded in critical functions, the demand for sophisticated bias mitigation frameworks is expected to accelerate further.

In the realm of AI-driven solutions, AI-Generated Personalized Cognitive Bias Training is emerging as a transformative approach to enhancing bias mitigation strategies. By tailoring training programs to individual cognitive biases, organizations can foster a deeper understanding of how these biases manifest in AI systems. This personalized training not only equips teams with the skills to identify and address biases but also promotes a culture of continuous learning and adaptation. As AI systems become more sophisticated, the need for targeted cognitive bias training is becoming increasingly evident, helping organizations to preemptively address potential biases before they impact decision-making processes. This proactive approach is essential for maintaining the integrity and fairness of AI applications across various sectors.

Regionally, North America remains the dominant market for Bias Mitigation Tools AI, driven by early adoption, regulatory leadership, and a mature ecosystem of AI solution providers. However, rapid digitalization in Asia Pacific and Europe is narrowing the gap, with th

ABSTRACT

Background: The public typically believes psychotherapy to be more effective than pharmacotherapy for depression treatments. This is not consistent with current scientific evidence, which shows that both types of treatment are about equally effective.

Objective: The study investigates whether this bias towards psychotherapy guides online information search and whether the bias can be reduced by explicitly providing expert information (in a blog entry) and by providing tag clouds that implicitly reveal experts’ evaluations.

Methods: A total of 174 participants completed a fully automated Web-based study after we invited them via mailing lists. First, participants read two blog posts by experts that either challenged or supported the bias towards psychotherapy. Subsequently, participants searched for information about depression treatment in an online environment that provided more experts’ blog posts about the effectiveness of treatments based on alleged research findings. These blogs were organized in a tag cloud; both psychotherapy tags and pharmacotherapy tags were popular. We measured tag and blog post selection, efficacy ratings of the presented treatments, and participants’ treatment recommendation after information search.

Results: Participants demonstrated a clear bias towards psychotherapy (mean 4.53, SD 1.99) compared to pharmacotherapy (mean 2.73, SD 2.41; t₁₇₃=7.67, P<.001, d=0.81) when rating treatment efficacy prior to the experiment. Accordingly, participants exhibited biased information search and evaluation. This bias was significantly reduced, however, when participants were exposed to tag clouds with challenging popular tags. Participants facing popular tags challenging their bias (n=61) showed significantly less biased tag selection (F_2,168=10.61, P<.001, partial eta squared=0.112), blog post selection (F_2,168=6.55, P=.002, partial eta squared=0.072), and treatment efficacy ratings (F_2,168=8.48, P<.001, partial eta squared=0.092), compared to bias-supporting tag clouds (n=56) and balanced tag clouds (n=57). Challenging (n=93) explicit expert information as presented in blog posts, compared to supporting expert information (n=81), decreased the bias in information search with regard to blog post selection (F_1,168=4.32, P=.04, partial eta squared=0.025). No significant effects were found for treatment recommendation (Ps>.33).

Conclusions: We conclude that the psychotherapy bias is most effectively attenuated—and even eliminated—when popular tags implicitly point to blog posts that challenge the widespread view. Explicit expert information (in a blog entry) was less successful in reducing biased information search and evaluation. Since tag clouds have the potential to counter biased information processing, we recommend their insertion.

Concerns about gender bias in word embedding models have captured substantial attention in the algorithmic bias research literature. Other bias types however have received lesser amounts of scrutiny. This work describes a large-scale analysis of sentiment associations in popular word embedding models along the lines of gender and ethnicity but also along the less frequently studied dimensions of socioeconomic status, age, physical appearance, sexual orientation, religious sentiment and political leanings. Consistent with previous scholarly literature, this work has found systemic bias against given names popular among African-Americans in most embedding models examined. Gender bias in embedding models however appears to be multifaceted and often reversed in polarity to what has been regularly reported. Interestingly, using the common operationalization of the term bias in the fairness literature, novel types of so far unreported bias types in word embedding models have also been identified. Specifically, the popular embedding models analyzed here display negative biases against middle and working-class socioeconomic status, male children, senior citizens, plain physical appearance and intellectual phenomena such as Islamic religious faith, non-religiosity and conservative political orientation. Reasons for the paradoxical underreporting of these bias types in the relevant literature are probably manifold but widely held blind spots when searching for algorithmic bias and a lack of widespread technical jargon to unambiguously describe a variety of algorithmic associations could conceivably be playing a role. The causal origins for the multiplicity of loaded associations attached to distinct demographic groups within embedding models are often unclear but the heterogeneity of said associations and their potential multifactorial roots raises doubts about the validity of grouping them all under the umbrella term bias. Richer and more fine-grained terminology as well as a more comprehensive exploration of the bias landscape could help the fairness epistemic community to characterize and neutralize algorithmic discrimination more efficiently.

Methods This data set has collected several popular pre-trained word embedding models.

-Word2vec Skip-Gram trained on Google News corpus (100B tokens) https://code.google.com/archive/p/word2vec/

-Glove trained on Wikipedia 2014 + Gigaword 5 (6B tokens) http://nlp.stanford.edu/data/glove.6B.zip

-Glove trained on 2B tweets Twitter corpus (27B tokens) http://nlp.stanford.edu/data/glove.twitter.27B.zip

-Glove trained on Common Crawl (42B tokens) http://nlp.stanford.edu/data/glove.42B.300d.zip

-Glove trained on Common Crawl (840B tokens) http://nlp.stanford.edu/data/glove.840B.300d.zip

-FastText trained with subword infomation on Wikipedia 2017, UMBC webbase corpus and statmt.org news dataset (16B tokens) https://dl.fbaipublicfiles.com/fasttext/vectors-english/wiki-news-300d-1M-subword.vec.zip

-Fastext trained with subword infomation on Common Crawl (600B tokens) https://dl.fbaipublicfiles.com/fasttext/vectors-english/crawl-300d-2M-subword.zip"

The majority of cognitive bias research has been conducted in Western cultures. We examined cross-cultural differences in cognitive biases, comparing Westerners’ and East Asians’ performance and acculturation following migration to the opposite culture. Two local (UK, Hong Kong) and four migrant (short-term and long-term migrants to each culture) samples completed culturally validated tasks measuring attentional and interpretation bias. Hong Kong residents were more positively biased than people living in the UK on several measures, consistent with the lower prevalence of psychological disorders in East Asia. Migrants to the UK had reduced positive biases on some tasks, while migrants to Hong Kong were more positive, compared to their respective home counterparts, consistent with acculturation in attention and interpretation biases. These data illustrate the importance of cultural validation of findings and, if replicated, would have implications for the mental health and well-being of migrants.

Selection bias is an important but often neglected problem in comparative research. While comparative case studies pay some attention to this problem, this is less the case in broader cross-national studies, where this problem may appear through the way the data used are generated. The article discusses three examples: studies of the success of newly formed political parties, research on protest events, and recent work on ethnic conflict. In all cases the data at hand are likely to be afflicted by selection bias. Failing to take into consideration this problem leads to serious biases in the estimation of simple relationships. Empirical examples illustrate a possible solution (a variation of a Tobit model) to the problems in these cases. The article also discusses results of Monte Carlo simulations, illustrating under what conditions the proposed estimation procedures lead to improved results.

Optimistic and pessimistic cognitive biases have been described in many animals and are related to the perceived valence of the environment. We, therefore, hypothesize that such cognitive bias can be adaptive depending on environmental conditions. In reward rich environments an optimistic bias would be favored, whereas in harsh environments a pessimistic one would thrive. Here, we empirically investigated the potential adaptive value of such bias using zebrafish as a model. We first phenotyped female zebrafish in an optimistic/pessimistic axis using a previously validated judgment bias assay. Optimistic and pessimistic females were then exposed to an unpredictable chronic stress protocol for 17 days, after which fish were euthanized and the sectional area of the different ovarian structures was quantified in both undisturbed and stressed groups. Our results show that zebrafish ovarian development responded to chronic stress, and that judgment bias impacted the relative area of the vitel...

Abstract This paper discusses the cognitive biases identified from the analysis of 21 graphic and textual records produced individually by 47 university students during an Economics class. The records were the result of an activity in which students were required to complete Cartesian charts in which part of the stock price trajectory of some companies was present, and part was hidden. Students should draw the trajectory that the stock price would have taken in a past period. In this sense, it was a “retrospective projection” activity. More than 80% of the 21 graphical and textual records included chronological distortions that exemplified the cognitive challenges involved in teaching and learning financial topics. The identified phenomenon consisted of a synthesis of cognitive biases that jeopardized the perception of event chronology that are part of the students' prior knowledge, being an example of the challenges to be considered in the initiatives for the promotion of Financial Education in Mathematical Education.

Data of two user studies (282 and 307 participants), investigating the risks and benefits of warning labels with and without obfuscations to mitigate confirmation bias during web search on debated topics. Study Variables (study 1 and study 2) display_con: Search result display - Study 1 - 1: targeted warning label with obfuscation - 2: random warning label with obfuscation - 3: regular (no intervention) - Study 2 - 1: targeted warning label with obfuscation - 2: targeted warning label without obfuscation - 3: random warning label with obfuscation - 4: random warning label without obfuscation - 5: regular (no intervention)- CRT_cat: Cognitive reflection - 1: intuitive - 2: analytic- topic: Assigned debated topic - 1: Is drinking milk healthy for humans? - 2: Is homework beneficial? - 3: Should people become vegetarian? - 4: Should students have to wear school uniforms?- clicksup_prop: Clicks on attitude-confirming (AC) search results (proportion of all clicks)- clickwarn_prop: Clicks on warning label (WL) search results (proportion of all clicks)- show_clicked: Clicks on show-button (number of clicks, only in conditions with obfuscation)- accuracy_bias: Accuracy bias estimation (Difference between a) observed bias (as the proportion of attitude-confirming clicks) and b) perceived bias (reported in the post-interaction questionnaire and re-coded into values from 0 to 1), positive values indicate an overestimation of bias)- att_change: Attitude change (Difference between attitude reported in the pre-interaction questionnaire and the post-interaction questionnaire. Negative values indicate an attitude change in the attitude-opposing direction, while positive values indicate an attitude strengthening in the attitude-supporting direction.)- knowledge_1: Self-reported prior knowledge (Reported on a seven-point Likert scale ranging from non-existent to excellent as a response to how they would describe their knowledge on the topic they were assigned to)- N_clicks: Cumulative clicks (Number of all clicks on search results)- NFC: Need for Cognition (Mean response to 4-item subset of the NFC questionnaire)- UX_usability: Usability (Mean of responses on a seven-point Likert scale to the module "usability"from the meCUE 2.0 questionnaire)- UX_usefulness: Usefulness (Mean of responses on a seven-point Likert scale to the module "usefulness"from the meCUE 2.0 questionnaire)

A pervasive concern with the use of subjective data in choice models is that they are biased and endogenous. This paper examines the extent to which cognitive biases plague subjective data, and specifically addresses the questions of: (1) whether cognitive dissonance affects the reporting of beliefs; and (2) whether individuals exert sufficient mental effort when probed about their subjective beliefs. For this purpose, I collect a unique panel dataset of Northwestern University undergraduates which contains their subjective expectations about major-specific outcomes for their chosen major as well as for other alternatives in their choice set. I do not find evidence of cognitive biases systematically affecting the reporting of beliefs. By analyzing patterns of belief updating, I can rule out cognitive dissonance being of serious concern in the current setting. There does not seem to be any systematic (non-classical) measurement error in the reporting of beliefs: I do not find systematic patterns in mental recall of previous responses, or in the extent of rounding in the reported beliefs for the various majors. Comparison of subjective beliefs with objective measures suggests that students have well-formed expectations. Overall, the results paint a favorable picture for the use of subjective expectations data in choice models.

Bias Detection as a Service Market Outlook

According to our latest research, the global Bias Detection as a Service market size reached USD 1.17 billion in 2024, propelled by the increasing demand for fairness and transparency in automated decision-making systems. The market is expected to grow at a robust CAGR of 25.6% from 2025 to 2033, reaching a forecasted size of USD 9.04 billion by 2033. This accelerated expansion is driven by growing regulatory scrutiny, heightened awareness of ethical AI, and the proliferation of AI and machine learning technologies across industries.

One of the primary growth factors for the Bias Detection as a Service market is the exponential rise in AI adoption across critical sectors such as finance, healthcare, and government. As organizations increasingly rely on automated systems for decision-making, the risk of embedded bias in algorithms has become a pressing concern. Regulatory bodies worldwide are enacting stricter guidelines to ensure algorithmic fairness, pushing enterprises to seek advanced bias detection solutions. The need for proactive bias identification not only helps organizations comply with regulations but also safeguards their reputation and fosters consumer trust, further fueling market expansion.

Technological advancements in machine learning and natural language processing are significantly enhancing the capabilities of bias detection platforms. Modern solutions leverage sophisticated analytics to identify, quantify, and mitigate both explicit and implicit biases in data and models. The integration of explainable AI (XAI) features is enabling stakeholders to understand and address the root causes of bias, which is especially critical in high-stakes applications like healthcare diagnostics and financial underwriting. Additionally, the growing ecosystem of cloud-based AI services is making bias detection tools more accessible to small and medium enterprises, democratizing their adoption and driving overall market growth.

Another vital driver is the increasing public and stakeholder demand for ethical AI. High-profile incidents involving biased AI systems have drawn attention to the societal impact of algorithmic decisions, prompting organizations to prioritize fairness as a core value. This shift is evident in sectors such as recruitment, lending, and law enforcement, where biased outcomes can have severe consequences. As a result, organizations are investing in Bias Detection as a Service solutions not only to mitigate risks but also to demonstrate their commitment to responsible AI practices. This trend is expected to intensify as AI systems become more pervasive in everyday life.

From a regional perspective, North America currently dominates the Bias Detection as a Service market, accounting for over 40% of the global revenue in 2024. This leadership is attributed to the region’s early adoption of AI technologies and a strong regulatory environment emphasizing fairness and accountability. Europe follows closely, with significant investments in ethical AI frameworks and compliance with GDPR-related mandates. Meanwhile, the Asia Pacific region is emerging as a high-growth market, fueled by rapid digital transformation, expanding AI research capabilities, and increasing government initiatives to address algorithmic bias. Latin America and the Middle East & Africa are also witnessing steady adoption, albeit at a slower pace due to infrastructural and regulatory challenges.

Component Analysis

The Bias Detection as a Service market is segmented by component into software and services, each playing a pivotal role in enabling organizations to address algorithmic bias. Software solutions form the backbone of the market, offering automated tools that integrate seamlessly with existing data pipelines and AI workflows. These platforms leverage advanced algorithms to scan datasets and models for signs of bias, providing actionable insights and recommendations for remediation. The software segment is characterized by continuous innovation, with vendors introducing features such as real-time bias monitoring, customizable fairness metrics, and integration with explainable AI modules. The demand for scalable, user-friendly, and interoperable software solutions is particularly strong among large enterprises and regulated industries.

On the services side, consulting, implementation, and managed service

Behavioural studies over half a century indicate that making categorical choices alters beliefs about the state of the world. People seem biased to confirm previous choices, and to suppress contradicting information. These choice-dependent biases imply a fundamental bound of human rationality. However, it remains unclear whether these effects extend to lower level decisions, and only little is known about the computational mechanisms underlying them. Building on the framework of sequential-sampling models of decision-making, we developed novel psychophysical protocols that enable us to dissect quantitatively how choices affect the way decision-makers accumulate additional noisy evidence. We find robust choice-induced biases in the accumulation of abstract numerical (experiment 1) and low-level perceptual (experiment 2) evidence. These biases deteriorate estimations of the mean value of the numerical sequence (experiment 1) and reduce the likelihood to revise decisions (experiment 2). Computational modelling reveals that choices trigger a reduction of sensitivity to subsequent evidence via multiplicative gain modulation, rather than shifting the decision variable towards the chosen alternative in an additive fashion. Our results thus show that categorical choices alter the evidence accumulation mechanism itself, rather than just its outcome, rendering the decision-maker less sensitive to new information.

This repository contains the data collected during and the code used to analyze the Beauty Survey. A pre-print of the associated paper including our analysis and findings can be found at this link. Zip files have been uploaded to this repository to stay within the file limit enforced by Zenodo. To use our code and data simply unzip these folders while mainting the current directory structure. While the code here is complete and stands on its own, any new features added can be found in the GitHub repository associated with this project (link). AG and NO are supported by a nominal grant received at the ELLIS Unit Alicante Foundation from the Regional Government of Valencia in Spain (Convenio Singular signed with Generalitat Valenciana, Conselleria de Innovacion, Industria, Comercio y Turismo, Direccion General de Innovacion), along with grants from the European Union’s Horizon 2020 research and innovation programme - ELISE (grant agreement 951847) and ELIAS (grant agreement 101120237), and by grants from the Banc Sabadell Foundation and Intel corporation. BL is partially supported by the European Union’s Horizon Europe research and innovation program under grant agreement No. 101120237 (ELIAS) and by the PNRR project FAIR - Future AI Research (PE00000013), under the NRRP MUR program funded by the NextGenerationEU.

Differential privacy (DP) is a rigorous notion of data privacy, used for private statistics. The canonical algorithm for differentially private mean estimation is to first clip the samples to a bounded range and then add noise to their empirical mean. Clipping controls the sensitivity and, hence, the variance of the noise that we add for privacy. But clipping also introduces statistical bias. This tradeoff is inherent: we prove that no algorithm can simultaneously have low bias, low error, and low privacy loss for arbitrary distributions. Additionally, we show that under strong notions of DP (i.e., pure or concentrated DP), unbiased mean estimation is impossible, even if we assume that the data is sampled from a Gaussian. On the positive side, we show that unbiased mean estimation is possible under a more permissive notion of differential privacy (approximate DP) if we assume that the distribution is symmetric. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

A recent article reported evidence from a survey experiment indicating that Americans reward whites more than blacks for hard work but penalize blacks more than whites for laziness. However, the present study demonstrates that these inferences were based on an unrepresentative selection of possible analyses: strength of inferences from results reported in the original article were weakened when combined with results from equivalent or relevant analyses not reported in the original article; moreover, newly-reported evidence revealed heterogeneity in racial bias: respondents given a direct choice between equivalent targets of different races favored the black target over the white target. Results illustrate how the presence of researcher degrees of freedom can foster production of inferences that are not representative of all inferences that could have been produced from a set of data, thus illustrating the value in preregistering research design protocols and requiring public posting of data.

Artificial intelligence (AI) technologies have been applied in various medical domains to predict patient outcomes with high accuracy. As AI becomes more widely adopted, the problem of model bias is increasingly apparent. In this study, we investigate the model bias that can occur when training a model using datasets for only one particular gender and aim to present new insights into the bias issue. For the investigation, we considered an AI model that predicts severity at an early stage based on the medical records of coronavirus disease (COVID-19) patients. For 5,601 confirmed COVID-19 patients, we used 37 medical records, namely, basic patient information, physical index, initial examination findings, clinical findings, comorbidity diseases, and general blood test results at an early stage. To investigate the gender-based AI model bias, we trained and evaluated two separate models—one that was trained using only the male group, and the other using only the female group. When the model trained by the male-group data was applied to the female testing data, the overall accuracy decreased—sensitivity from 0.93 to 0.86, specificity from 0.92 to 0.86, accuracy from 0.92 to 0.86, balanced accuracy from 0.93 to 0.86, and area under the curve (AUC) from 0.97 to 0.94. Similarly, when the model trained by the female-group data was applied to the male testing data, once again, the overall accuracy decreased—sensitivity from 0.97 to 0.90, specificity from 0.96 to 0.91, accuracy from 0.96 to 0.91, balanced accuracy from 0.96 to 0.90, and AUC from 0.97 to 0.95. Furthermore, when we evaluated each gender-dependent model with the test data from the same gender used for training, the resultant accuracy was also lower than that from the unbiased model.

There is substantial evidence that systemic biases influence the scholarly peer review process. Many scholars have advocated for double-blind peer review (also known as double-anonymous review) to reduce these biases. However, the effectiveness of double‐blind peer review in eliminating biases is uncertain because few randomized trials have manipulated blinding of author identities for journal submissions, and those that have are generally small or provide few insights on how it influences reviewer biases. In 2019, Functional Ecology began a large randomized trial, using real manuscript submissions, to evaluate the various consequences of shifting to double-blind peer review. Research papers submitted to the journal were randomly assigned to be reviewed with author identities blinded to reviewers (double‐blind review) or with authors identified to reviewers (single-blind review). In this paper, we explore the effect of blinding on the outcomes of peer review, examining reviewer ratings and editorial decisions, and ask whether author gender and/or location mediate the effects of review type. Double-blind review reduced the average success of manuscripts in peer review; papers reviewed with author identities blinded received on average lower ratings from reviewers and were less likely to be invited for revision or resubmission. However, the effect of review treatment varied with author location. Papers with first authors residing in countries with a higher human development index (HDI) and/or higher average English proficiency fared much better than those from countries with a lower HDI and lower English proficiency, but only when author identities were known to reviewers; outcomes were similar between demographic groups when author identities were not known to reviewers. Blinding author identities had no effect on gender differences in reviewer ratings or editor decisions. Our data provide strong evidence that authors from higher income and/or English-speaking countries receive significant benefits (a large positive bias) to being identified to reviewers during the peer review process, and that anonymizing author-identities (e.g., double-blind review) reduces this bias, making the peer review process more equitable. We suggest that offering optional blinding of author identities, as some journals allow, is unlikely to substantially reduce the biases that exist because authors from higher-income and English-speaking countries are the least likely to choose to be reviewed with their identity anonymized.

To address the need for regularly updated wind resource data, NREL has processed the High-Resolution Rapid Refresh (HRRR) outputs for use in grid integration modeling. The HRRR is an hourly-updated operational forecast product produced by the National Oceanic and Atmospheric Administration (NOAA) (Dowell et al., 2022). Several barriers have prevented the HRRR's widespread proliferation in the wind energy industry: missing timesteps (prior to 2019), challenging file format for wind energy analysis, limited vertical height resolution, and negative bias versus legacy WIND Toolkit data (2007-2013). NREL has applied re-gridding, interpolation, and bias-correction to the native HRRR data to overcome these limitations. This results in the now-publicly-available bias corrected and interpolated HRRR (BC-HRRR) dataset for weather years 2015 to 2023. Bias correction is necessary for wind resource consistency across weather years to be used simultaneously in planning-focused grid integration studies alongside the original WIND Toolkit data. We show that quantile mapping with the WIND Toolkit as a historical baseline is an effective method for bias correcting the interpolated HRRR data: the BC-HRRR has reduced mean bias versus comparable gridded wind resource datasets (+0.12 m/s versus Vortex) and has very low mean bias versus ground measurement stations (+0.01 m/s) (Buster et al., 2024). BC-HRRR's consistency with the legacy WIND Toolkit allows NREL to extend grid integration analysis to 15+ weather years of wind data with low-overhead extensibility to future years as they are made available by NOAA. As with historical datasets like the WIND Toolkit, BC-HRRR is intended for use in grid integration modeling (e.g., capacity expansion, production cost, and resource adequacy modeling) both independently and alongside the legacy WIND Toolkit.

Summary

This dataset is released by Criteo to foster research and innovation on Fairness in Advertising and AI systems in general. See also Criteo pledge for Fairness in Advertising.

The dataset is intended to learn click predictions models and evaluate by how much their predictions are biased between different gender groups.

Data description

The dataset contains pseudononymized users' context and publisher features that was collected from a job targeting campaign ran for 5 months by Criteo AdTech company. Each line represents a product that was shown to a user. Each user has an impression session where they can see several products at the same time. Each product can be clicked or not clicked by the user. The dataset consists of 1072226 rows and 55 columns.

• features
  • user_id is a unique identifier assigned to each user. This identifier has been anonymized and does not contain any information related to the real users.
  • product_id is a unique identifier assigned to each product, i.e. job offer.
  • impression_id is a unique identifier assigned to each impression, i.e. online session that can have several products at the same time.
  • cat0 to cat5 are anonymized categorical user features.
  • cat6 to cat12 are anonymized categorical product features.
  • num13 to num47 are anonymized numerical user features.
• labels
  • protected_attribute is a binary feature that describes user gender proxy, i.e. female is 0, male is 1. The detailed description on the meaning can be found below.
  • senior is a binary feature that describes the seniority of the job position, i.e. an assistant role is 0, a managerial role is 1. This feature was created during data processing step from the product title feature: if the product title contains words describing managerial role (e.g. 'president', 'ceo', and others), it is assigned to 1, otherwise to 0.
  • rank is a numerical feature that corresponds to the positional rank of the product on the display for given impression_id. Usually, the position on the display creates the bias with respect to the click: lower rank means higher position of the product on the display.
  • displayrandom is a binary feature that equals 1 if the display position on the banner of the products associated with the same impression_id was randomized. The click-rank metric should be computed on displayrandom = 1 to avoid positional bias.
  • click is a binary feature that equals 1 if the product product_id in the impression impression_id was clicked by the user user_id.

Data statistics

License

The data is released under the CC-BY-NC-SA 4.0 license. You are free to Share and Adapt this data provided that you respect the Attribution, NonCommercial and ShareAlike conditions. Please read carefully the full license before using.

Protected attribute

As Criteo does not have access to user demographics we report a proxy of gender as protected attribute. This proxy is reported as binary for simplicity yet we acknowledge gender is not necessarily binary.

The value of the proxy is computed as the majority of gender attributes of products seen in the user timeline. Product having a gender attribute are typically fashion and clothing. We acknowledge that this proxy does not necessarily represent how users relate to a given gender yet we believe it to be a realistic approximation for research purposes.

We encourage research in Fairness defined with respect to other attributes as well.

Limitations and interpretations

We remark that the proposed gender proxy does not give a definition of the gender. Since we do not have access to the sensitive information, this is the best solution we have identified at this stage to idenitify bias on pseudonymised data, and we encourage any discussion on better approximations. This proxy is reported as binary for simplicity yet we acknowledge gender is not necessarily binary. Although our research focuses on gender, this should not diminish the importance of investigating other types of algorithmic discrimination. While this dataset provides important application of fairness-aware algorithms in a high-risk domain, there are several fundamental limitation that can not be addressed easily through data collection or curation processes. These limitations in...