Description
Metrics for Assessing the Quality of Generated Text.
Description
Implementation of the BLEU-Score in 'C++' to evaluate the quality of generated text. The BLEU-Score, introduced by Papineni et al. (2002) <doi:10.3115/1073083.1073135>, is a metric for evaluating the quality of generated text. It is based on the n-gram overlap between the generated text and reference texts. Additionally, the package provides some smoothing methods as described in Chen and Cherry (2014) <doi:10.3115/v1/W14-3346>.
README.md
sacRebleu
The goal of sacRebleu is to provide a simple interface to the BLEU score, a metric for evaluating the quality of generated text. This package is inspired by the NLTK and sacrebleu implementations of the BLEU score, and is implemented in C++ for the R programming language.
Installation
You can install the development version of sacRebleu from GitHub with:
Linux and MacOS
# install.packages("devtools")
devtools::install_github("LazerLambda/sacRebleu")
Windows
This package builds upon the tok package, which requires the Rust toolchain to be installed. To install the rustup, follow the instructions at https://www.rust-lang.org/tools/install and at tok. After installing the Rust toolchain, you can install the development version of sacRebleu as described above.
Example
library(sacRebleu)
cand_corpus <- list("This is good", "This is not good")
ref_corpus <- list(list("Perfect outcome!", "Excellent!"), list("Not sufficient.", "Horrible."))
bleu_corpus <- bleu_corpus(ref_corpus, cand_corpus)