Question Paper Solutions of Data Warehousing and Data Mining (Old),7th Semester,Computer Science and Engineering,Maulana Abul Kalam Azad University of Technology

Question Paper Solutions of chapter Module III of Data Warehousing and Data Mining, 7th Semester , Computer Science and Engineering

Question Paper Solutions of chapter Module IV of Data Warehousing and Data Mining, 7th Semester , Computer Science and Engineering

Amazon question/answer Dataset

Find it on:

Description:

This dataset contains Question and Answer data from Amazon, totaling around 1.4 million answered questions. The data were collected and made available by Prof. Julian McAuley of UCSD (University of San Diego)

Acknowledgements

This data was acquired from Prof. Julian's website. Different categories of data were present in independent archive files (each question being a separate JSON file in the archive). I joined them and added the attribute Category for ease of use. Please go through the ICDM paper, in particular, it goes into great detail on the potential of this dataset. That was one of my major inspirations for creating this dataset.

Reference

Modeling ambiguity, subjectivity, and diverging viewpoints in opinion question answering systems Mengting Wan, Julian McAuley International Conference on Data Mining (ICDM), 2016 PDF

Addressing complex and subjective product-related queries with customer reviews Julian McAuley, Alex Yang World Wide Web (WWW), 2016 PDF

Addressing the heterogeneity of both the outcome of a disease and the treatment response to an intervention is a mandatory pathway for regulatory approval of medicines. In randomized clinical trials (RCTs), confirmatory subgroup analyses focus on the assessment of drugs in predefined subgroups, while exploratory ones allow a posteriori the identification of subsets of patients who respond differently. Within the latter area, subgroup discovery (SD) data mining approach is widely used—particularly in precision medicine—to evaluate treatment effect across different groups of patients from various data sources (be it from clinical trials or real-world data). However, both the limited consideration by standard SD algorithms of recommended criteria to define credible subgroups and the lack of statistical power of the findings after correcting for multiple testing hinder the generation of hypothesis and their acceptance by healthcare authorities and practitioners. In this paper, we present the Q-Finder algorithm that aims to generate statistically credible subgroups to answer clinical questions, such as finding drivers of natural disease progression or treatment response. It combines an exhaustive search with a cascade of filters based on metrics assessing key credibility criteria, including relative risk reduction assessment, adjustment on confounding factors, individual feature’s contribution to the subgroup’s effect, interaction tests for assessing between-subgroup treatment effect interactions and tests adjustment (multiple testing). This allows Q-Finder to directly target and assess subgroups on recommended credibility criteria. The top-k credible subgroups are then selected, while accounting for subgroups’ diversity and, possibly, clinical relevance. Those subgroups are tested on independent data to assess their consistency across databases, while preserving statistical power by limiting the number of tests. To illustrate this algorithm, we applied it on the database of the International Diabetes Management Practice Study (IDMPS) to better understand the drivers of improved glycemic control and rate of episodes of hypoglycemia in type 2 diabetics patients. We compared Q-Finder with state-of-the-art approaches from both Subgroup Identification and Knowledge Discovery in Databases literature. The results demonstrate its ability to identify and support a short list of highly credible and diverse data-driven subgroups for both prognostic and predictive tasks.