- P
BigCodeBench Dataset
- paperswithcode.com
Share
Facebook
Twitter
EmailClick to copy linkLink copiedCiteBigCodeBench Dataset [Dataset]. https://paperswithcode.com/dataset/bigcodebenchAuthorsTerry Yue Zhuo; Minh Chien Vu; Jenny Chim; Han Hu; Wenhao Yu; Ratnadira Widyasari; Imam Nur Bani Yusuf; Haolan Zhan; Junda He; Indraneil Paul; Simon Brunner; Chen Gong; Thong Hoang; Armel Randy Zebaze; Xiaoheng Hong; Wen-Ding Li; Jean Kaddour; Ming Xu; Zhihan Zhang; Prateek Yadav; Naman jain; Alex Gu; Zhoujun Cheng; Jiawei Liu; Qian Liu; Zijian Wang; David Lo; Binyuan Hui; Niklas Muennighoff; Daniel Fried; Xiaoning Du; Harm de Vries; Leandro von WerraDescriptionBigCodeBench is an easy-to-use benchmark for code generation with practical and challenging programming tasks¹. It aims to evaluate the true programming capabilities of large language models (LLMs) in a more realistic setting¹. The benchmark is designed for HumanEval-like function-level code generation tasks, but with much more complex instructions and diverse function calls¹.
Here are some key features of BigCodeBench: - Precise evaluation & ranking: It provides a leaderboard for latest LLM rankings before & after rigorous evaluation¹. - Pre-generated samples: BigCodeBench accelerates code intelligence research by open-sourcing LLM-generated samples for various models¹. - Execution Environment: The execution environment in BigCodeBench is less bounded than EvalPlus to support tasks with diverse library dependencies¹. - Test Evaluation: BigCodeBench relies on unittest for evaluating the generated code¹.
(1) GitHub - bigcode-project/bigcodebench: BigCodeBench: The Next .... https://github.com/bigcode-project/bigcodebench/.
- h
bigcodebench-hard
- huggingface.co
Updated Nov 12, 2024+ more versionsShareFacebookTwitterEmailClick to copy linkLink copiedCitebigcodebench-hard [Dataset]. https://huggingface.co/datasets/bigcode/bigcodebench-hardCroissantCroissant is a format for machine-learning datasets. Learn more about this at mlcommons.org/croissant.Dataset updatedNov 12, 2024Dataset authored and provided byBigCodeDescriptionbigcode/bigcodebench-hard dataset hosted on Hugging Face and contributed by the HF Datasets community
- h
bigcodebench-hard-results
- huggingface.co
ShareFacebookTwitterEmailClick to copy linkLink copiedCiteBigCode, bigcodebench-hard-results [Dataset]. https://huggingface.co/datasets/bigcode/bigcodebench-hard-resultsDataset authored and provided byBigCodeDescriptionbigcode/bigcodebench-hard-results dataset hosted on Hugging Face and contributed by the HF Datasets community
- h
bigcodebench-perf
- huggingface.co
Updated Sep 13, 2024+ more versionsShareFacebookTwitterEmailClick to copy linkLink copiedCiteBigCode (2024). bigcodebench-perf [Dataset]. https://huggingface.co/datasets/bigcode/bigcodebench-perfDataset updatedSep 13, 2024Dataset authored and provided byBigCodeDescriptionbigcode/bigcodebench-perf dataset hosted on Hugging Face and contributed by the HF Datasets community
- h
bigcodebench-hard-perf
- huggingface.co
Updated Jul 26, 2024ShareFacebookTwitterEmailClick to copy linkLink copiedCitebigcodebench-hard-perf [Dataset]. https://huggingface.co/datasets/bigcode/bigcodebench-hard-perfDataset updatedJul 26, 2024Dataset authored and provided byBigCodeDescriptionDataset Card for "bigcodebench-hard-perf"
More Information needed
- h
bigcodebench-hard-solve-rate
- huggingface.co
Updated Jul 12, 2024+ more versionsShareFacebookTwitterEmailClick to copy linkLink copiedCiteBigCode (2024). bigcodebench-hard-solve-rate [Dataset]. https://huggingface.co/datasets/bigcode/bigcodebench-hard-solve-rateDataset updatedJul 12, 2024Dataset authored and provided byBigCodeDescriptionbigcode/bigcodebench-hard-solve-rate dataset hosted on Hugging Face and contributed by the HF Datasets community
bigcodebench-lite-pro
- huggingface.co
Updated Jan 3, 2023ShareFacebookTwitterEmailClick to copy linkLink copiedCitebigcodebench-lite-pro [Dataset]. https://huggingface.co/datasets/CodeEval-Pro/bigcodebench-lite-proCroissantCroissant is a format for machine-learning datasets. Learn more about this at mlcommons.org/croissant.Dataset updatedJan 3, 2023Dataset provided byCodeEval, Inc.AuthorsCodeEval-ProDescriptionEvaluation dataset for umanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation Task (arxiv.org/abs/2412.21199).
Not seeing a result you expected?
Learn how you can add new datasets to our index.
FacebookTwitterEmailBigCodeBench is an easy-to-use benchmark for code generation with practical and challenging programming tasks¹. It aims to evaluate the true programming capabilities of large language models (LLMs) in a more realistic setting¹. The benchmark is designed for HumanEval-like function-level code generation tasks, but with much more complex instructions and diverse function calls¹.
Here are some key features of BigCodeBench: - Precise evaluation & ranking: It provides a leaderboard for latest LLM rankings before & after rigorous evaluation¹. - Pre-generated samples: BigCodeBench accelerates code intelligence research by open-sourcing LLM-generated samples for various models¹. - Execution Environment: The execution environment in BigCodeBench is less bounded than EvalPlus to support tasks with diverse library dependencies¹. - Test Evaluation: BigCodeBench relies on unittest for evaluating the generated code¹.
(1) GitHub - bigcode-project/bigcodebench: BigCodeBench: The Next .... https://github.com/bigcode-project/bigcodebench/.