Objective: Our objective is to evaluate the efficacy of ChatGPT 4 in accurately and effectively delivering genetic information, building on previous findings with ChatGPT 3.5. We focus on assessing the utility, limitations, and ethical implications of using ChatGPT in medical settings. Materials and Methods: A structured questionnaire, including the Brief User Survey (BUS-15) and custom questions, was developed to assess ChatGPT 4's clinical value. An expert panel of genetic counselors and clinical geneticists independently evaluated ChatGPT 4's responses to these questions. We also involved comparative analysis with ChatGPT 3.5, utilizing descriptive statistics and using R for data analysis. Results: ChatGPT 4 demonstrated improvements over 3.5 in context recognition, relevance, and informativeness. However, performance variability and concerns about the naturalness of the output were noted. No significant difference in accuracy was found between ChatGPT 3.5 and 4.0. Notably, the effic..., Study Design This study was conducted to evaluate the performance of ChatGPT 4 (March 23rd, 2023)  Model) in the context of genetic counseling and education. The evaluation involved a structured questionnaire, which included questions selected from the Brief User Survey (BUS-15) and additional custom questions designed to assess the clinical value of ChatGPT 4's responses. Questionnaire Development The questionnaire was built on Qualtrics, which comprised twelve questions: seven selected from the BUS-15 preceded by two additional questions that we designed. The initial questions focused on quality and answer relevancy: 1.    The overall quality of the Chatbot’s response is: (5-point Likert: Very poor to Very Good) 2.    The Chatbot delivered an answer that provided the relevant information you would include if asked the question. (5-point Likert: Strongly disagree to Strongly agree) The BUS-15 questions (7-point Likert: Strongly disagree to Strongly agree) focused on: 1.    Recogniti..., , # A comparative evaluation of ChatGPT 3.5 and ChatGPT 4 in responses to selected genetics questions - Full study data

https://doi.org/10.5061/dryad.s4mw6m9cv

This data was captured when evaluating the ability of ChatGPT to address questions patients may ask it about three genetic conditions (BRCA1, HFE, and MLH1). This data is associated with the JAMIA article of the similar name with the DOIÂ 10.1093/jamia/ocae128

Description of the data and file structure

1. Key: This tab contains the data structure, explaining the survey questions, and potential responses available.
2. Prompt Responses: This tab contains the prompts used for ChatGPT, and the response provided from each model (3.5 and 4)
3. GPT 4 Results: This tab provides the responses collected from the medical experts (genetic counselors and clinical geneticist) from the Qualtrics survey.
4. Accuracy (Qx_1): This tab contains the subset of results from both the Ch...

The dataset contains the results of developing alternative text for images using chatbots based on large language models. The study was carried out in April-June 2024. Microsoft Copilot, Google Gemini, and YandexGPT chatbots were used to generate 108 text descriptions for 12 images. Descriptions were generated by chatbots using keywords specified by a person. The experts then rated the resulting descriptions on a Likert scale (from 1 to 5). The data set is presented in a Microsoft Excel table on the “Data” sheet with the following fields: record number; image number; chatbot; image type (photo, logo); request date; list of keywords; number of keywords; length of keywords; time of compilation of keywords; generated descriptions; required length of descriptions; actual length of descriptions; description generation time; usefulness; reliability; completeness; accuracy; literacy. The “Images” sheet contains links to the original images. Alternative descriptions are presented in English.

This study investigates the potential of ChatGPT 4 in the assessment of personality traits based on written texts. Using two publicly available datasets containing both written texts and self-assessments of the authors’ psychological traits based on the Big Five model, we aimed to evaluate the predictive performance of ChatGPT 4. For each sample text, we asked for numerical predictions on an eleven-point scale and compared them with the self-assessments. We also asked for ChatGPT 4 confidence scores on an eleven-point scale for each prediction. To keep the study within a manageable scope, a zero-prompt modality was chosen, although more sophisticated prompting strategies could potentially improve performance. The results show that ChatGPT 4 has moderate but significant abilities to automatically infer personality traits from written text. However, it also shows limitations in recognizing whether the input text is appropriate or representative enough to make accurate inferences, which could hinder practical applications. Furthermore, the results suggest that improved benchmarking methods could increase the efficiency and reliability of the evaluation process. These results pave the way for a more comprehensive evaluation of the capabilities of Large Language Models in assessing personality traits from written texts.

Data for this study were collected at the University of California – Irvine (UCI) as part of the UCI-MUST (Measuring Undergraduate Success Trajectories) Project, a larger longitudinal measurement project aimed at improving understanding of undergraduate experience, trajectories and outcomes, while supporting campus efforts to improve institutional performance and enhance educational equity (Arum et. al. 2021). The project is focused on student educational experience at a selective large, research-oriented public university on the quarter system with half of its students first-generation and 85 percent Hispanic, Asian, African-American, Pacific Islander or Native American. Since Fall 2019, the project has tracked annually new cohorts of freshmen and juniors with longitudinal surveys administered at the end of every academic quarter. Data from the Winter 2023 end of term assessment, administered in the first week of April, was pooled for four longitudinal study cohorts for this study (i.e., Fall 2019-2022 cohorts). There was an overall response rate of 42.5 percent for the Winter 2023 end of term assessment. This allowed us to consider student responses from freshmen through senior years enrolled in courses throughout the university. Students completed questionnaire items about their knowledge and use of ChatGPT in and out of the classroom during the winter 2023 academic term. In total 1,129 students completed the questionnaire, which asked questions about: knowledge of ChatGPT (“Do you know what ChatGPT is?”); general use (“Have you used ChatGPT before?”); and instructor attitude (“What was the attitude of the instructor for [a specific course students enrolled in] regarding the use of ChatGPT?”). Of those 1,129 students, 191 had missing data for at least one variable of interest and were subsequently dropped from analysis, resulting in a final sample of 938 students. In addition, for this study we merged our survey data with administrative data from campus that encompasses details on student background, including gender, race, first-generation college-going, and international student status. Campus administrative data also provides course-level characteristics, including whether a particular class is a lower- or upper-division course as well as the academic unit on campus offering the course. In addition, we used administrative data on all students enrolled at the university to generate classroom composition measures for every individual course taken by students in our sample – specifically the proportion of underrepresented minority students in the class, the proportion of international students in the class and the proportion of female students in the class. For our student-level analysis [R1], we used binary logistic regressions to examine the association between individual characteristics and (1) individual awareness and (2) individual academic use of ChatGPT utilizing the student-level data of 938 students. Individual characteristics include gender, underrepresented minority student status, international student status, first generation college-going student status, student standing (i.e. lower or upper classmen), cumulative grade point average and field of study. Field of study was based on student major assigned to the broad categories of physical sciences (i.e. physical sciences, engineering, and information and computer science), health sciences (i.e. pharmacy, biological sciences, public health, and nursing), humanities, social sciences (i.e. business, education, and social sciences), the arts, or undeclared. We defined awareness of ChatGPT as an affirmative response to the question “Do you know what ChatGPT is?” Regarding ChatGPT use, we focused on academic use which was defined as an affirmative response of either “Yes, for academic use” or “Yes, for academic and personal use” to the question “Have you used ChatGPT before?” For our course-level analysis [R2], we constructed a measure – course-level instructor encouragement for ChatGPT use – based on student responses to the end of the term survey conducted at the completion of the Winter 2023 term. In the survey, students were asked to indicate the extent to which their instructors encouraged them to use ChatGPT in each of their enrolled courses. The response

This dataset is used for the research article 'Unveiling the ChatGPT Educational Revolution: Assessing the Dynamic Impact on Students and Educators' analysis. Data Availability StatementData are available from the corresponding authors upon request for York St John University research use only.

### Data Availability Statement (for the paper)

All dialogue logs and final responses collected in this study are publicly available in the PROSPECT repository on Zenodo (DOI: [to be assigned]). The repository contains PDF files of complete dialogue histories and Markdown files of final comprehensive analyses for all conditions and models used in this study, allowing for reproducibility and further analysis.

### README.md for Zenodo

# PROSPECT: Professional Role Effects on Specialized Perspective Enhancement in Conversational Task

## Overview
This repository (PROSPECT) contains the dataset associated with the paper:
> "Empirical Investigation of Expertise, Multiperspectivity, and Abstraction Induction in Conversational AI Outputs through Professional Role Assignment to Both User and AI"

This research analyzed changes in dialogue logs and final responses when professional roles were assigned to both user and AI sides across multiple Large Language Models (LLMs). This repository provides the complete dialogue logs (PDF format) and final responses (Markdown format) used in the analysis.

## Directory Structure
The repository structure under the top directory (`PROSPECT/`) is as follows:

```
PROSPECT/
├── dialogue/ # Dialogue histories (PDF)
│ ├── none/
│ ├── ai_only/
│ ├── user_only/
│ └── both/
└── final_answers/ # Final responses (Markdown)
├── none/
├── ai_only/
├── user_only/
└── both/
```

- **dialogue/**
- Contains raw dialogue logs in PDF format. Subdirectories represent role assignment conditions:
- `none/`: No roles assigned to either user or AI
- `ai_only/`: Role assigned to AI only
- `user_only/`: Role assigned to user only
- `both/`: Roles assigned to both user and AI
- **final_answers/**
- Contains final comprehensive analysis responses in Markdown format. Directory structure mirrors that of `dialogue/`.

## File Naming Convention
Files in each directory follow this naming convention:
```
[AI]_[conditionNumber]-[roleNumber].pdf
[AI]_[conditionNumber]-[roleNumber].md
```
- `[AI]`: AI model name used for dialogue (e.g., ChatGPT, ChatGPT-o1, Claude, Gemini)
- `[conditionNumber]`: Number indicating role assignment condition
- 0: none
- 1: ai_only
- 2: user_only
- 3: both
- `[roleNumber]`: Professional role number
- 0: No role
- 1: Detective
- 2: Psychologist
- 3: Artist
- 4: Architect
- 5: Natural Scientist

### Examples:
- `ChatGPT_3-1.pdf`: Dialogue log with ChatGPT-4o model under "both" condition (3) with detective role (1)
- `Gemini_1-4.md`: Final response from Gemini model under "ai_only" condition (1) with architect role (4)

## Role Number Reference
| roleNumber | Professional Role |
|-----------:|:-----------------|
| 0 | No role |
| 1 | Detective |
| 2 | Psychologist |
| 3 | Artist |
| 4 | Architect |
| 5 | Natural Scientist|

## Data Description
- **Dialogue Histories (PDF format)**
Complete logs of questions and responses from each session, preserved as captured during the research. All dialogues were conducted in Japanese. While assistant version information is not included, implementation dates and model names are recorded within the files.
- **Final Responses (Markdown format)**
Excerpted responses to the final "comprehensive analysis request" as Markdown files, intended for text analysis and keyword extraction. All responses are in Japanese.

*Note: This dataset contains dialogues and responses exclusively in Japanese. Researchers interested in lexical analysis or content analysis should consider this language specification.

## How to Use
1. Please maintain the folder hierarchy after downloading.
2. For meta-analysis or lexical analysis, refer to PDFs for complete dialogues and Markdown files for final responses.
3. Utilize for research reproduction, secondary analysis, or meta-analysis.

## License
This dataset is released under the **CC BY 4.0** License.
- Free to use and modify, but please cite this repository (DOI) and the associated paper when using the data.

## Related Publication

## Disclaimer
- The dialogue logs contain no personal information or confidential data.
- The provided logs and responses reflect the research timing; identical prompts may yield different responses due to AI model updates.
- The creators assume no responsibility for any damages resulting from the use of this dataset.

## Contact
For questions or requests, please contact skeisuke@ibaraki-ct.ac.jp.

Snapshot img

Chatbot Market Size 2025-2029

The chatbot market size is forecast to increase by USD 9.63 billion, at a CAGR of 42.9% between 2024 and 2029.

The market is witnessing significant growth, driven by the integration of chatbots with various communication channels such as social media, websites, and messaging apps. This integration enables businesses to engage with customers in real-time, providing instant responses and enhancing customer experience. However, the market faces challenges, including the lack of awareness and standardization of chatbot services. Despite these obstacles, the potential benefits of chatbots, including cost savings, increased efficiency, and improved customer engagement, make it an attractive investment for businesses seeking to enhance their digital presence and streamline operations. Companies looking to capitalize on this market opportunity should focus on developing chatbot solutions that offer customizable features, seamless integration with existing systems, and natural language processing capabilities to deliver human-like interactions. Navigating the challenges of awareness and standardization will require targeted marketing efforts and collaborations with industry partners to establish best practices and industry standards.

What will be the Size of the Chatbot Market during the forecast period?

Explore in-depth regional segment analysis with market size data - historical 2019-2023 and forecasts 2025-2029 - in the full report.
Request Free SampleThe market continues to evolve, with dynamic market dynamics shaping its growth and applications across various sectors. Conversational AI, a key component of chatbots, is advancing with the integration of sentiment analysis, emotional intelligence, and meteor score to enhance user experience. Pre-trained models and language understanding are being utilized to improve performance metrics, while neural networks and contextual awareness enable more accurate intent recognition. Deployment strategies, including policy learning and cloud platforms, are evolving to support cross-platform compatibility and multi-lingual support. Performance metrics, such as F1-score and response time, are crucial in evaluating model effectiveness. Reinforcement learning and knowledge base integration are essential for chatbot development and lead generation. Error rate and character error rate are critical in speech recognition, while API integration and dialogue state tracking facilitate seamless conversational experiences. Technical support and customer engagement are primary applications of chatbots, with sales conversion and automated responses optimizing business operations. Deep learning architectures and transfer learning are driving advancements in question answering and natural language processing. Contextualized word embeddings and dialogue management are essential for effective user interaction. Overall, the market is an ever-evolving landscape, with continuous innovation and integration of advanced technologies shaping its future.

How is this Chatbot Industry segmented?

The chatbot industry research report provides comprehensive data (region-wise segment analysis), with forecasts and estimates in 'USD million' for the period 2025-2029, as well as historical data from 2019-2023 for the following segments. End-userRetailBFSIGovernmentTravel and hospitalityOthersProductSolutionsServicesDeploymentCloud-BasedOn-PremiseHybridApplicationCustomer ServiceSales and MarketingHealthcare SupportE-Commerce AssistanceGeographyNorth AmericaUSCanadaEuropeFranceGermanyItalyUKMiddle East and AfricaEgyptKSAOmanUAEAPACChinaIndiaJapanSouth AmericaArgentinaBrazilRest of World (ROW)

By End-user Insights

The retail segment is estimated to witness significant growth during the forecast period.The market is experiencing significant growth, particularly in the retail sector. E-commerce giants like Amazon, Flipkart, Alibaba, and Snapdeal are leading this trend, integrating chatbots to improve customer experience during online product searches. These AI-powered bots facilitate quick and effective resolution of payment-related queries, enhancing the shopping experience. However, retailers face challenges in ensuring a seamless user experience, as consumers increasingly prefer mobile shopping. Deep learning architectures and natural language processing (NLP) are crucial components of chatbot development. NLP enables intent recognition, sentiment analysis, and entity extraction, while deep learning models provide contextual awareness and dialogue management. Speech recognition and dialogue state tracking further enhance the user experience. Cross-platform compatibility and multi-lingual support are essential features for chatbots, catering to diverse user bases. Pre-trained models and transfer learning enable faster development and deployment. Reinforcement learning and policy learning optimize bot

AbstractBackground: Birth control methods (BCMs) are often underutilized or misunderstood, especially among young individuals entering their reproductive years. With the growing reliance on artificial intelligence (AI) platforms for health-related information, this study evaluates the performance of GPT-4 (OpenAI, San Francisco, CA, USA) and Google Gemini (Google, Mountain View, CA, USA) in addressing commonly asked questions about BCMs.Methods: Thirty questions, derived from the American College of Obstetrics and Gynecologists website, were posed to both AI platforms. Questions spanned four categories: general contraception, specific contraceptive types, emergency contraception, and other topics. Responses were evaluated using a 5-point rubric assessing accuracy, completeness, and lack of false information. Overall scores were calculated by averaging the rubric scores. Statistical analysis, including the Wilcoxon signed-rank and Kruskal-Wallis tests, was performed to compare performance metrics.Results: ChatGPT and Google Gemini both provided high-quality responses, with overall scores averaging 4.38 ± 0.58 and 4.37 ± 0.52, respectively, categorized as "excellent." ChatGPT outperformed in reducing false information (4.70 ± 0.60 vs. 4.47 ± 0.73), while Google Gemini excelled in accuracy (4.53 ± 0.57 vs. 4.30 ± 0.70). Completeness scores were comparable. No significant differences were found in overall performance (p = 0.548), though Google Gemini showed a significant edge in accuracy (p = 0.035). Both platforms scored consistently across question categories, with no statistically significant differences noted.Conclusions: GPT-4 and Google Gemini provide reliable and accurate responses to BCM-related queries, with slight differences in strengths. These findings underscore the potential of AI tools in addressing public health information needs, particularly for young individuals seeking guidance on contraception. Further studies with larger datasets may elucidate nuanced differences between AI platforms.

AI marketing products include:Machine learning algorithms: Models that analyze customer data to identify patterns and predict behavior.Natural language processing (NLP): Tools that interpret and generate human language for effective communication with customers.Computer vision: Technologies that enable AI to "see" and analyze images and videos.Predictive analytics: Platforms that forecast future customer behavior and trends.Chatbots: Conversational AI agents that provide customer support and automate interactions. Recent developments include: Our personal and professional lives are beginning to change as a result of artificial intelligence (AI). Leading companies are beginning to take advantage of the potential presented by technology, showing that the marketing sector is not immune to this digital upheaval. One of the main advantages of AI in marketing is the improved understanding of consumer behaviour., in February 2023, Dealtale, a Vianai Company and the industry pioneer in causal AI for marketers, today announced the creation of Marketing Co-pilot, a ChatGPT-like feature that enables marketers to ask complex questions about their past, present, and future performance metrics and receive instant responses based on information from all parts of their marketing and sales stack, such as Salesforce, HubSpot, Google Analytics, and social media channels., From the marketing manager to the CMO, Marketing Co-pilot was created to support marketers of all specialties in making the greatest strategic choices. A content marketer might inquire as to which blogs generated the most revenue, a demand generation marketer might inquire as to why particular segments are underperforming, a digital marketer might seek to identify the gaps in their customer journey, and a CMO might seek to see year-over-year conversion trends.December 2022: By acquiring Octo, IBM becomes one of the largest digital transformation partners to the federal government. As IBM Consulting's public and federal market organization grows to 4,200 highly skilled employees, it will be able to support federal agencies with a flexible, modern approach to digital transformation, enhancing its ability to support federal agencies with a flexible, modern approach to digital transformation. Octo also brings deep federal mission experience and certifications in the technologies most commonly used across government, as well as a proven track record of enabling fast IT modernization and seamless citizen engagement.. Key drivers for this market are: Technological advancements in AI algorithms and NLP Growing demand for personalized customer experiences. Potential restraints include: Data privacy and security concerns Lack of skilled professionals. Notable trends are: The growing need for business agility and faster time-to-market is driving market growth. Generative AI: AI-powered tools that create original content, such as images, videos, and text. Hyper-personalization: Marketing campaigns tailored to individual customer preferences in real time..

Comparison of Tokens used to run all evaluations in the Artificial Analysis Intelligence Index by Model

Comparison of Represents the average of coding benchmarks in the Artificial Analysis Intelligence Index (LiveCodeBench & SciCode) by Model

Comparison of Represents the average of math benchmarks in the Artificial Analysis Intelligence Index (AIME) by Model

Comprehensive comparison of Artificial Analysis Intelligence Index vs. Output Tokens Used in Artificial Analysis Intelligence Index (Log Scale) by Model

Comparison of Cost (USD) to run all evaluations in the Artificial Analysis Intelligence Index by Model

Comparison of Seconds to Output 500 Tokens, including reasoning model 'thinking' time; Lower is better by Model

Comprehensive comparison of Latency (Time to First Token) vs. Output Speed (Output Tokens per Second) by Model

Comparison of Image Input Price: USD per 1k images at 1MP (1024x1024) by Model

Comprehensive comparison of Artificial Analysis Intelligence Index vs. Price (USD per M Tokens, Log Scale, More Expensive to Cheaper) by Model

Comprehensive comparison of Output Speed (Output Tokens per Second) vs. Price (USD per M Tokens) by Model