Tough Tables (2T) is a dataset designed to evaluate table annotation approaches in solving the CEA and CTA tasks.
The dataset is compliant with the data format used in SemTab 2019, and it can be used as an additional dataset without any modification. The target knowledge graph is DBpedia 2016-10.
Check out the 2T GitHub repository for more details about the dataset generation.

New in v3.0: We release the updated version of 2T! The target knowledge graphs are DBpedia 2016-10 and Wikidata 20220521. Starting from this version, the dataset is split into valid and test sets.

This work is based on the following paper:

Cutrona, V., Bianchi, F., Jimenez-Ruiz, E. and Palmonari, M. (2020). Tough Tables: Carefully Evaluating Entity Linking for Tabular Data. ISWC 2020, LNCS 12507, pp. 1–16.

Note on License: This dataset includes data from the following sources. Refer to each source for license details:
- Wikipedia https://www.wikipedia.org/
- DBpedia https://dbpedia.org/
- Wikidata https://www.wikidata.org/
- SemTab 2019 https://doi.org/10.5281/zenodo.3518539
- GeoDatos https://www.geodatos.net
- The Pudding https://pudding.cool/
- Offices.net https://offices.net/
- DATA.GOV https://www.data.gov/

THIS DATA IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Changelog:

v3.0

• Both datasets require SemTab2020 CEA format (tab_id, row_id, col_id, entity).
• Tables IDs and artificial noise values differ from previous versions.
• Datasets are split into Valid and Test sets of tables.
• New GT for ToughTables-WD (2T_WD)
  • Entities Q23772518 and Q7327323 have been removed because they are no longer represented in WD
  • Updated ancestor/descendant hierarchies to evaluate CTA.
• Evaluation scripts are provided with the data sets.

v2.0

• New GT for 2T_WD
  • A few entities have been removed from the CEA GT, because they are no longer represented in WD (e.g., dbr:Devonté points to wd:Q21155080, which does not exist)
  • Tables codes and values differ from the previous version, because of the random noise.
  • Updated ancestor/descendant hierarchies to evaluate CTA.

v1.0

• New Wikidata version (2T_WD)
• Fix header for tables CTRL_DBP_MUS_rock_bands_labels.csv and CTRL_DBP_MUS_rock_bands_labels_NOISE2.csv (column 2 was reported with id 1 in target - NOTE: the affected column has been removed from the SemTab2020 evaluation)
• Remove duplicated entries in tables
• Remove rows with wrong values (e.g., the Kazakhstan entity has an empty name "''")
• Many rows and noised columns are shuffled/changed due to the random noise generator algorithm
• Remove row "Florida","Floorida","New York, NY" from TOUGH_WEB_MISSP_1000_us_cities.csv (and all its NOISE1 variants)
• Fix header of tables:
  • CTRL_WIKI_POL_List_of_current_monarchs_of_sovereign_states.csv
  • CTRL_WIKI_POL_List_of_current_monarchs_of_sovereign_states_NOISE2.csv
  • TOUGH_T2D_BUS_29414811_2_4773219892816395776_videogames_developers.csv
  • TOUGH_T2D_BUS_29414811_2_4773219892816395776_videogames_developers_NOISE2.csv

v0.1-pre

• First submission. It contains only tables, without GT and Targets.

Computer Science Named Entity Recognition in the Open Research Knowledge Graph

1) About

This work proposes a standardized CS-NER task by defining a set of seven contribution-centric scholarly entities for CS NER viz., research problem , solution , resource , language , tool , method , and dataset .

The main contributions are:

1) Merges annotations for contribution-centric named entities from related work as the following datasets:

• The dataset proposed in Analyzing the Dynamics of Research by Extracting Key Aspects of Scientific Papers (Gupta & Manning, IJCNLP 2011) is the source for ftd, annotated for both titles and abstracts for the following select entities mapped to our standardized types focus -> solution ; domain -> research problem ; and technique -> method

• The dataset proposed in Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction (Luan et al., EMNLP 2018) is the source for scierc, annotated for abstracts for the following select entities with mappings task -> research problem

  • The dataset proposed in SemEval-2021 Task 11: NLPContributionGraph - Structuring Scholarly NLP Contributions for a Research Knowledge Graph (D’Souza et al., SemEval 2021) is the source for ncg, annotated for both titles and abstracts for research problem

  • https://paperswithcode.com/ as the pwc annotated for both titles and abstracts for task -> research problem and method entities.

2) Additionally, supplies a new annotated dataset for the titles in the ACL anthology in the acl repository where titles are annotated with all seven entities.

2) Dataset Statistics for full dataset

Titles

train.data

| NER | Count |

| --- | --- |

| solution | 65,213 |

| research problem | 43,033 |

| resource | 19,759 |

| method | 19,645 |

| tool | 4,856 |

| dataset | 4,062 |

| language | 1,704 |

dev.data

| NER | Count |

| --- | --- |

| solution | 3,685 |

| research problem | 2,717 |

| resource | 1,224 |

| method | 1,172 |

| tool | 264 |

| dataset | 191 |

| language | 79 |

test.data

| NER | Count |

| --- | --- |

| solution | 29,287 |

| research problem | 11,093 |

| resource | 8,511 |

| method | 7,009 |

| tool | 2,272 |

| dataset | 947 |

| language | 690 |

Abstracts

train-abs.data

| NER | Count |

| --- | --- |

| research problem | 15,498 |

| method | 12,932 |

dev-abs.data

| NER | Count |

| --- | --- |

| research problem | 1,450 |

| method | 839 |

test-abs.data

| NER | Count |

| --- | --- |

| research problem | 4,123 |

| method | 3,170 |

The reamining repositories have specialized README files with the respective dataset statistics.

3) Citation

Accepted for publication in ICADL 2022 proceedings.

Citation information forthcoming

Preprint

@article{d2022computer,
 title={Computer Science Named Entity Recognition in the Open Research Knowledge Graph},
 author={D'Souza, Jennifer and Auer, S{\"o}ren},
 journal={arXiv preprint arXiv:2203.14579},
 year={2022}
}

4) Additional resources

CS NER Software trained on the dataset in this repository

Codebase: https://gitlab.com/TIBHannover/orkg/nlp/orkg-nlp-experiments/-/tree/master/orkg_cs_ner

Service URL - REST API: https://orkg.org/nlp/api/docs#/annotation/annotates_paper_annotation_csner_post

Service URL - PyPi: https://orkg-nlp-pypi.readthedocs.io/en/latest/services/services.html#cs-ner-computer-science-named-entity-recognition

Entity resolution (ER; also known as record linkage or de-duplication) is the process of merging noisy databases, often in the absence of unique identifiers. A major advancement in ER methodology has been the application of Bayesian generative models, which provide a natural framework for inferring latent entities with rigorous quantification of uncertainty. Despite these advantages, existing models are severely limited in practice, as standard inference algorithms scale quadratically in the number of records. While scaling can be managed by fitting the model on separate blocks of the data, such a naïve approach may induce significant error in the posterior. In this article, we propose a principled model for scalable Bayesian ER, called “distributed Bayesian linkage” or d-blink, which jointly performs blocking and ER without compromising posterior correctness. Our approach relies on several key ideas, including: (i) an auxiliary variable representation that induces a partition of the entities and records into blocks; (ii) a method for constructing well-balanced blocks based on k-d trees; (iii) a distributed partially collapsed Gibbs sampler with improved mixing; and (iv) fast algorithms for performing Gibbs updates. Empirical studies on six datasets—including a case study on the 2010 Decennial Census—demonstrate the scalability and effectiveness of our approach. Supplementary materials for this article are available online.

Identity Resolution Software Market Outlook

The global identity resolution software market size was valued at approximately $1.25 billion in 2023 and is projected to reach around $3.75 billion by 2032, growing at a compound annual growth rate (CAGR) of 12.5% from 2024 to 2032. The growth of the market is driven primarily by the increasing need for organizations to manage and mitigate identity-related risks, enhance customer experience, and comply with regulatory requirements.

One of the significant growth factors for the identity resolution software market is the rising incidence of identity theft and fraud. As digital transactions become more common, the potential for misuse of personal information has grown, leading organizations to adopt robust identity resolution solutions. Furthermore, the increasing complexity of cyber threats has accelerated the demand for advanced software capable of accurately identifying and authenticating users. This growing need for security and fraud prevention underpins the market’s rapid expansion.

Another driving force is the burgeoning adoption of digital services across various industries. With the proliferation of digital platforms, businesses are increasingly relying on identity resolution software to provide a seamless and personalized customer experience. This software helps organizations understand their customers better by combining data from various sources to create a comprehensive profile. By leveraging these insights, businesses can tailor their services and communications to meet individual customer needs, thereby enhancing customer satisfaction and loyalty.

The regulatory landscape also plays a crucial role in market growth. Governments worldwide are implementing stringent data protection regulations, such as the General Data Protection Regulation (GDPR) in Europe and the California Consumer Privacy Act (CCPA) in the United States. These regulations mandate organizations to have precise and up-to-date customer data, which can be efficiently managed through identity resolution software. The need to comply with these regulations is prompting companies to invest in advanced identity resolution solutions, further propelling market growth.

From a regional perspective, North America is expected to hold a significant share of the identity resolution software market due to the presence of major market players and early adoption of advanced technologies. The region's strong regulatory framework and high awareness about cybersecurity also contribute to the market's dominance. Meanwhile, the Asia Pacific region is anticipated to witness the fastest growth during the forecast period, driven by increasing digitalization, rising internet penetration, and growing awareness about identity-related threats and solutions.

Component Analysis

The identity resolution software market can be segmented by component into software and services. Software solutions are the backbone of this market, offering various functionalities such as data integration, data cleansing, identity matching, and identity verification. These software solutions are continuously evolving, integrating advanced technologies like artificial intelligence and machine learning to enhance their accuracy and efficiency. Companies are investing heavily in R&D to develop innovative software that can handle the complexities of modern identity management, thereby driving the software segment’s growth.

Services, on the other hand, play a complementary role in the identity resolution ecosystem. These services include consulting, implementation, training, and support. As organizations deploy identity resolution software, they often require expert guidance to ensure smooth implementation and optimal utilization of the software. Consulting services help in selecting the right solution tailored to the specific needs of an organization, while implementation services ensure that the software is correctly integrated with existing systems. Training and support services are crucial for enabling staff to effectively use the software and troubleshoot any issues that may arise, thereby enhancing the overall efficiency and effectiveness of identity management initiatives.

The integration of software and services is essential for the successful adoption of identity resolution solutions. While software provides the necessary tools and functionalities, services ensure that these tools are correctly implemented and effectively used. This symbiotic relationship between software and services is vital for organizations aiming to achieve compr

Salutary Data is a boutique, B2B contact and company data provider that's committed to delivering high quality data for sales intelligence, lead generation, marketing, recruiting / HR, identity resolution, and ML / AI. Our database currently consists of 148MM+ highly curated B2B Contacts ( US only), along with over 4M+ companies, and is updated regularly to ensure we have the most up-to-date information.

We can enrich your in-house data ( CRM Enrichment, Lead Enrichment, etc.) and provide you with a custom dataset ( such as a lead list) tailored to your target audience specifications and data use-case. We also support large-scale data licensing to software providers and agencies that intend to redistribute our data to their customers and end-users.

What makes Salutary unique? - We offer our clients a truly unique, one-stop aggregation of the best-of-breed quality data sources. Our supplier network consists of numerous, established high quality suppliers that are rigorously vetted. - We leverage third party verification vendors to ensure phone numbers and emails are accurate and connect to the right person. Additionally, we deploy automated and manual verification techniques to ensure we have the latest job information for contacts. - We're reasonably priced and easy to work with.

Products: API Suite Web UI Full and Custom Data Feeds

Services: Data Enrichment - We assess the fill rate gaps and profile your customer file for the purpose of appending fields, updating information, and/or rendering net new “look alike” prospects for your campaigns. ABM Match & Append - Send us your domain or other company related files, and we’ll match your Account Based Marketing targets and provide you with B2B contacts to campaign. Optionally throw in your suppression file to avoid any redundant records. Verification (“Cleaning/Hygiene”) Services - Address the 2% per month aging issue on contact records! We will identify duplicate records, contacts no longer at the company, rid your email hard bounces, and update/replace titles or phones. This is right up our alley and levers our existing internal and external processes and systems.

Distinguishing among policy problems based on whether or not their definition and resolution are independent or interdependent of other policy problems.