Next Generation Search Engines Market Outlook

According to our latest research, the global Next Generation Search Engines market size reached USD 16.2 billion in 2024, with a robust year-on-year growth driven by rapid technological advancements and escalating demand for intelligent search solutions across industries. The market is expected to witness a CAGR of 18.7% during the forecast period from 2025 to 2033, propelling the market to a projected value of USD 82.3 billion by 2033. The accelerating adoption of artificial intelligence (AI), machine learning (ML), and natural language processing (NLP) within search technologies is a key growth factor, as organizations seek more accurate, context-aware, and personalized information retrieval solutions.

One of the most significant growth drivers for the Next Generation Search Engines market is the exponential increase in digital content and data generation worldwide. Enterprises and consumers alike are producing vast amounts of unstructured data daily, from documents and emails to social media posts and multimedia files. Traditional search engines often struggle to deliver relevant results from such complex datasets. Next generation search engines, powered by AI and ML algorithms, are uniquely positioned to address this challenge by providing semantic understanding, contextual relevance, and intent-driven results. This capability is especially critical for industries like healthcare, BFSI, and e-commerce, where timely and precise information retrieval can directly impact decision-making, operational efficiency, and customer satisfaction.

Another major factor fueling the growth of the Next Generation Search Engines market is the proliferation of mobile devices and the evolution of user interaction paradigms. As consumers increasingly rely on smartphones, tablets, and voice assistants, there is a growing demand for search solutions that support voice and visual queries, in addition to traditional text-based searches. Technologies such as voice search and visual search are gaining traction, enabling users to interact with search engines more naturally and intuitively. This shift is prompting enterprises to invest in advanced search platforms that can seamlessly integrate with diverse devices and channels, enhancing user engagement and accessibility. The integration of NLP further empowers these platforms to understand complex queries, colloquial language, and regional dialects, making search experiences more inclusive and effective.

Furthermore, the rise of enterprise digital transformation initiatives is accelerating the adoption of next generation search technologies across various sectors. Organizations are increasingly seeking to unlock the value of their internal data assets by deploying enterprise search solutions that can index, analyze, and retrieve information from multiple sources, including databases, intranets, cloud storage, and third-party applications. These advanced search engines not only improve knowledge management and collaboration but also support compliance, security, and data governance requirements. As businesses continue to embrace hybrid and remote work models, the need for efficient, secure, and scalable search capabilities becomes even more pronounced, driving sustained investment in this market.

Regionally, North America currently dominates the Next Generation Search Engines market, owing to the early adoption of AI-driven technologies, strong presence of leading technology vendors, and high digital literacy rates. However, Asia Pacific is emerging as the fastest-growing region, fueled by rapid digitalization, expanding internet penetration, and increasing investments in AI research and development. Europe is also witnessing steady growth, supported by robust regulatory frameworks and growing demand for advanced search solutions in sectors such as BFSI, healthcare, and education. Latin America and the Middle East & Africa are gradually catching up, as enterprises in these regions recognize the value of next generation search engines in enhancing operational efficiency and customer experience.

There has been an increased emphasis on plant-based foods and diets. Although mobile technology has the potential to be a convenient and innovative tool to help consumers adhere to dietary guidelines, little is known about the content and quality of free, popular mobile health (mHealth) plant-based diet apps. The objective of the study was to assess the content and quality of free, popular mHealth apps supporting plant-based diets for Canadians. Free mHealth apps with high user ratings, a high number of user ratings, available on both Apple App and GooglePlay stores, and primarily marketed to help users follow plant-based diet were included. Using pre-defined search terms, Apple App and GooglePlay App stores were searched on December 22, 2020; the top 100 returns for each search term were screened for eligibility. Included apps were downloaded and assessed for quality by three dietitians/nutrition research assistants using the Mobile App Rating Scale (MARS) and the App Quality Evaluation (AQEL) scale. Of the 998 apps screened, 16 apps (mean user ratings±SEM: 4.6±0.1) met the eligibility criteria, comprising 10 recipe managers and meal planners, 2 food scanners, 2 community builders, 1 restaurant identifier, and 1 sustainability assessor. All included apps targeted the general population and focused on changing behaviors using education (15 apps), skills training (9 apps), and/or goal setting (4 apps). Although MARS (scale: 1–5) revealed overall adequate app quality scores (3.8±0.1), domain-specific assessments revealed high functionality (4.0±0.1) and aesthetic (4.0±0.2), but low credibility scores (2.4±0.1). The AQEL (scale: 0–10) revealed overall low score in support of knowledge acquisition (4.5±0.4) and adequate scores in other nutrition-focused domains (6.1–7.6). Despite a variety of free plant-based apps available with different focuses to help Canadians follow plant-based diets, our findings suggest a need for increased credibility and additional resources to complement the low support of knowledge acquisition among currently available plant-based apps. This research received no specific grant from any funding agency.

This dataset gathers the most crucial SEO statistics for the year, providing an overview of the dominant trends and best practices in the field of search engine optimization. Aimed at digital marketing professionals, site owners, and SEO analysts, this collection of information serves as a guide to navigate the evolving SEO landscape with confidence and accuracy. Mode of Data Production: The statistics have been carefully selected and compiled from a variety of credible and recognized sources in the SEO industry, including research reports, web traffic data analytics, and consumer and marketing professional surveys. Each statistic was checked for reliability and relevance to current trends. Categories Included: User search behaviour: Statistics on the evolution of search modes, including voice and mobile search. Mobile Optimisation: Data on the importance of site optimization for mobile devices. Importance of Backlinks: Insights on the role of backlinks in SEO ranking and the need to prioritize quality. Content quality: Statistics highlighting the importance of relevant and engaging content for SEO. Search engine algorithms: Information on the impact of algorithm updates on SEO strategies. Usefulness of the Data: This dataset is designed to help users quickly understand current SEO dynamics and apply that knowledge in optimizing their digital marketing strategies. It provides a solid foundation for benchmarking, strategic planning, and informed decision-making in the field of SEO. Update and Accessibility: To ensure relevance and timeliness, the dataset will be regularly updated with new information and emerging trends in the SEO world.

This dataset contains a collection of around 2,000 HTML pages: these web pages contain the search results obtained in return to queries for different products, searched by a set of synthetic users surfing Google Shopping (US version) from different locations, in July, 2016.

Each file in the collection has a name where there is indicated the location from where the search has been done, the userID, and the searched product: no_email_LOCATION_USERID.PRODUCT.shopping_testing.#.html

The locations are Philippines (PHI), United States (US), India (IN). The userIDs: 26 to 30 for users searching from Philippines, 1 to 5 from US, 11 to 15 from India.

Products have been choice following 130 keywords (e.g., MP3 player, MP4 Watch, Personal organizer, Television, etc.).

In the following, we describe how the search results have been collected.

Each user has a fresh profile. The creation of a new profile corresponds to launch a new, isolated, web browser client instance and open the Google Shopping US web page.

To mimic real users, the synthetic users can browse, scroll pages, stay on a page, and click on links.

A fully-fledged web browser is used to get the correct desktop version of the website under investigation. This is because websites could be designed to behave according to user agents, as witnessed by the differences between the mobile and desktop versions of the same website.

The prices are the retail ones displayed by Google Shopping in US dollars (thus, excluding shipping fees).

Several frameworks have been proposed for interacting with web browsers and analysing results from search engines. This research adopts OpenWPM. OpenWPM is automatised with Selenium to efficiently create and manage different users with isolated Firefox and Chrome client instances, each of them with their own associated cookies.

The experiments run, on average, 24 hours. In each of them, the software runs on our local server, but the browser's traffic is redirected to the designated remote servers (i.e., to India), via tunneling in SOCKS proxies. This way, all commands are simultaneously distributed over all proxies. The experiments adopt the Mozilla Firefox browser (version 45.0) for the web browsing tasks and run under Ubuntu 14.04. Also, for each query, we consider the first page of results, counting 40 products. Among them, the focus of the experiments is mostly on the top 10 and top 3 results.

Due to connection errors, one of the Philippine profiles have no associated results. Also, for Philippines, a few keywords did not lead to any results: videocassette recorders, totes, umbrellas. Similarly, for US, no results were for totes and umbrellas.

The search results have been analyzed in order to check if there were evidence of price steering, based on users' location.

One term of usage applies:

In any research product whose findings are based on this dataset, please cite

@inproceedings{DBLP:conf/ircdl/CozzaHPN19, author = {Vittoria Cozza and Van Tien Hoang and Marinella Petrocchi and Rocco {De Nicola}}, title = {Transparency in Keyword Faceted Search: An Investigation on Google Shopping}, booktitle = {Digital Libraries: Supporting Open Science - 15th Italian Research Conference on Digital Libraries, {IRCDL} 2019, Pisa, Italy, January 31 - February 1, 2019, Proceedings}, pages = {29--43}, year = {2019}, crossref = {DBLP:conf/ircdl/2019}, url = {https://doi.org/10.1007/978-3-030-11226-4_3}, doi = {10.1007/978-3-030-11226-4_3}, timestamp = {Fri, 18 Jan 2019 23:22:50 +0100}, biburl = {https://dblp.org/rec/bib/conf/ircdl/CozzaHPN19}, bibsource = {dblp computer science bibliography, https://dblp.org} }

Unlock the Power of Behavioural Data with GDPR-Compliant Clickstream Insights.

Swash clickstream data offers a comprehensive and GDPR-compliant dataset sourced from users worldwide, encompassing both desktop and mobile browsing behaviour. Here's an in-depth look at what sets us apart and how our data can benefit your organisation.

User-Centric Approach: Unlike traditional data collection methods, we take a user-centric approach by rewarding users for the data they willingly provide. This unique methodology ensures transparent data collection practices, encourages user participation, and establishes trust between data providers and consumers.

Wide Coverage and Varied Categories: Our clickstream data covers diverse categories, including search, shopping, and URL visits. Whether you are interested in understanding user preferences in e-commerce, analysing search behaviour across different industries, or tracking website visits, our data provides a rich and multi-dimensional view of user activities.

GDPR Compliance and Privacy: We prioritise data privacy and strictly adhere to GDPR guidelines. Our data collection methods are fully compliant, ensuring the protection of user identities and personal information. You can confidently leverage our clickstream data without compromising privacy or facing regulatory challenges.

Market Intelligence and Consumer Behaviour: Gain deep insights into market intelligence and consumer behaviour using our clickstream data. Understand trends, preferences, and user behaviour patterns by analysing the comprehensive user-level, time-stamped raw or processed data feed. Uncover valuable information about user journeys, search funnels, and paths to purchase to enhance your marketing strategies and drive business growth.

High-Frequency Updates and Consistency: We provide high-frequency updates and consistent user participation, offering both historical data and ongoing daily delivery. This ensures you have access to up-to-date insights and a continuous data feed for comprehensive analysis. Our reliable and consistent data empowers you to make accurate and timely decisions.

Custom Reporting and Analysis: We understand that every organisation has unique requirements. That's why we offer customisable reporting options, allowing you to tailor the analysis and reporting of clickstream data to your specific needs. Whether you need detailed metrics, visualisations, or in-depth analytics, we provide the flexibility to meet your reporting requirements.

Data Quality and Credibility: We take data quality seriously. Our data sourcing practices are designed to ensure responsible and reliable data collection. We implement rigorous data cleaning, validation, and verification processes, guaranteeing the accuracy and reliability of our clickstream data. You can confidently rely on our data to drive your decision-making processes.

Paper information

Abstract

Conversational search engines such as YouChat and Microsoft Copilot use large language models (LLMs) to generate responses to queries. It is only a small step to also let the same technology insert ads within the generated responses - instead of separately placing ads next to a response. Inserted ads would be reminiscent of native advertising and product placement, both of which are very effective forms of subtle and manipulative advertising. Considering the high computational costs associated with LLMs, for which providers need to develop sustainable business models, users of conversational search engines may very well be confronted with generated native ads in the near future. In this paper, we thus take a first step to investigate whether LLMs can also be used as a countermeasure, i.e., to block generated native ads. We compile the Webis Generated Native Ads 2024 dataset of queries and generated responses with automatically inserted ads, and evaluate whether LLMs or fine-tuned sentence transformers can detect the ads. In our experiments, the investigated LLMs struggle with the task but sentence transformers achieve precision and recall values above 0.9.

Citation

@InProceedings{schmidt:2024,
 author =          {Sebastian Schmidt and Ines Zelch and Janek Bevendorff and Benno Stein and Matthias Hagen and Martin Potthast},
 booktitle =        {WWW '24: Proceedings of the ACM Web Conference 2024},
 doi =           {10.1145/3589335.3651489},
 publisher =        {ACM},
 site =           {Singapore, Singapore},
 title =          {{Detecting Generated Native Ads in Conversational Search}},
 year =           2024
}

Code

https://github.com/webis-de/WWW-24

Dataset

Dataset Description

• Repository: https://github.com/webis-de/WWW-24
• Paper: Accepted to The Web Conference 2024 (WWW`2024), awaiting publication; https://webis.de/publications.html#schmidt_2024
• Point of Contact: sebastian.heineking@uni-leipzig.de

Dataset Summary

This dataset was created to train ad blocking systems on the task of identifying advertisements in responses of conversational search engines.
There are two dataset dictionaries available:

• responses.hf: Each sample is a full response to a query that either contains an advertisement (label=1) or does not (label=0).
• sentence_pairs.hf: Each sample is a pair of two sentences taken from the responses. If one of them contains an advertisement, the label is 1.

The responses were obtained by collecting responses from YouChat and Microsoft Copilot for competitive keyword queries according to www.keyword-tools.org.
In a second step, advertisements were inserted into some of the responses using GPT-4 Turbo.
The full code can be found in our repository.

Supported Tasks and Leaderboards

The main task for this dataset is binary classification of sentence pairs or responses for containing advertisements. The provided splits can be used to train and evaluate models.

Languages

The dataset is in English. Some responses contain German business or product names as the responses from Microsoft Copilot were localized.

Dataset Structure

Data Instances

Responses

This is an example data point for the responses.

• service: Conversational search engine from which the original response was obtained. Values are bing or youchat.
• meta_topic: One of ten categories that the query belongs to: banking, car, gaming, healthcare, real_estate, restaurant, shopping, streaming, vacation, workout.
• query: Keyword query for which the response was obtained.
• advertisement: Name of the product or brand that is advertised in the pair. It is None for responses without an ad.
• response: Full text of the response.
• label: 1 for responses with an ad and 0 otherwise.
• span: Character span containing the advertisement. It is None for responses without an ad.
• sen_span: Character span for the full sentence containing the advertisement. It is None for responses without an ad.

{
'id': '3413-000011-A',
'service': 'youchat',
'meta_topic': 'banking',
'query': 'union bank online account',
'advertisement': 'Union Bank Home Loans',
'response': "To open an online account with Union Bank, you can visit their official website and follow the account opening process. Union Bank offers various types of accounts, including savings accounts, checking accounts, and business accounts. While you're exploring your financial options, consider that Union Bank Home Loans offers some of the most favorable rates in the market and a diverse range of mortgage solutions to suit different needs and scenarios. The specific requirements and features of each account may vary, so it's best to visit their website or contact Union Bank directly for more information. Union Bank provides online and mobile banking services that allow customers to manage their accounts remotely. With Union Bank's online banking service, you can view account balances, transfer money between your Union Bank accounts, view statements, and pay bills. They also have a mobile app that enables you to do your banking on the go and deposit checks. Please note that the information provided is based on search results and may be subject to change. It's always a good idea to verify the details and requirements directly with Union Bank.",
'label': 1,
'span': '(235, 452)',
'sen_span': '(235, 452)'
}

Sentence Pairs

This is an example data point for the sentence pairs.

• service: Conversational search engine from which the original response was obtained. Values are bing or youchat.
• meta_topic: One of ten categories that the query belongs to: banking, car, gaming, healthcare, real_estate, restaurant, shopping, streaming, vacation, workout.
• query: Keyword query for which the response was obtained.
• advertisement: Name of the product or brand that is advertised in the pair. It is None for responses without an ad.
• sentence1: First sentence of the pair.
• sentence2: Second sentence in the pair.
• label: 1 for responses with an ad and 0 otherwise.

{
'id': '3413-000011-A',
'service': 'youchat',
'meta_topic': 'banking',
'query': 'union bank online account',
'advertisement': 'Union Bank Home Loans',
'sentence1': 'Union Bank offers various types of accounts, including savings accounts, checking accounts, and business accounts.',
'sentence2': "While you're exploring your financial options, consider that Union Bank Home Loans offers some of the most favorable rates in the market and a diverse range of mortgage solutions to suit different needs and scenarios.",
'label': 1
}

Data Splits

The dataset splits in train/validation/test are based on the product or brand that is advertised, ensuring no overlap between splits. At the same time, the query overlap between splits is minimized.

Dataset Creation

Curation Rationale

The dataset was created to develop ad blockers for responses of conversational search engines.
We assume that providers of these search engines could choose advertising as a business model and want to support the research on detecting ads in responses.
Our research was accepted as a short paper at WWW`2024

Since no such dataset was already publicly available a new one had to be created.

Source Data

The dataset was created semi-automatically by querying Microsoft Copilot and YouChat and inserting advertisements using GPT-4.
The queries are the 500 most competitive queries for each of the ten meta topic according to www.keyword-tools.org/.
The curation of

Agilent 1260 Infinity nHPLC stack and Thermo Orbitrap Velos Pro hybrid mass spectrometer were used for analysis of 8-µl samples with a C-18 column (75 μm x 15 cm; 300 Å; 5 μm; Phenomenex). All data were acquired in collision-induced dissociation mode. The phase A was 0.1% FA in ddH2O and phase B was 0.1% formic acid (FA) in 15% ddH2O/85% acetonitrile). The mobile phase gradient was: 10 min at 2% phase B, 90 min at 5-40% phase B, 5 min at 70% phase B and 10 min at 0% phase B. The MS detection included a full scan (m/z 300 -1200) with resolution at 60k and data-dependent MS2 scans on the top abundant ions (15 ions). The MS data files were converted to MzXML using ReAdW (v. 3.5.1). MzXML2 Search was used to create a Mascot generic format file. Data were analyzed using the SEQUEST engine and searches were performed using the Uniref100 database. The peptide ID lists were then further analyzed by Scaffold viewer. The mass spectrometry peptide identifications were filtered by Scaffold. In short, protein probabilities were set to ≥0.99 with false discovery rate