ROUGE, or Recall-Oriented Understudy for Gisting Evaluation, is a set of metrics and a software package used for evaluating automatic summarization and machine translation software in natural language processing. The metrics compare an automatically produced summary or translation against a reference or a set of references (human-produced) summary or translation. ROUGE metrics range between 0 and 1, with higher scores indicating higher similarity between the automatically produced summary and the reference.
Metrics The following five evaluation metrics are available.
ROUGE-N: Overlap of n-grams between the system and reference summaries. ROUGE-1 refers to the overlap of unigrams (each word) between the system and reference summaries. ROUGE-2 refers to the overlap of bigrams between the system and reference summaries. ROUGE-L: Longest Common Subsequence (LCS) based statistics. Longest common subsequence problem takes into account sentence-level structure similarity naturally and identifies longest co-occurring in sequence n-grams automatically. ROUGE-W: Weighted LCS-based statistics that favors consecutive LCSes. ROUGE-S: Skip-bigram based co-occurrence statistics. Skip-bigram is any pair of words in their sentence order. ROUGE-SU: Skip-bigram plus unigram-based co-occurrence statistics.
See also BLEU F-Measure METEOR NIST (metric) Noun-phrase chunking Word error rate (WER)
References
External links ROUGE Usage Tutorial Java Implementation of ROUGE
