Paraphrase or paraphrasing in computational linguistics is the natural language processing task of detecting and generating paraphrases. Applications of paraphrasing are varied including information retrieval, question answering, text summarization, and plagiarism detection. Paraphrasing is also useful in the evaluation of machine translation, as well as semantic parsing and generation of new samples to expand existing corpora.
Paraphrase generation
Multiple sequence alignment Barzilay and Lee proposed a method to generate paraphrases through the usage of monolingual parallel corpora, namely news articles covering the same event on the same day. Training consists of using multi-sequence alignment to generate sentence-level paraphrases from an unannotated corpus. This is done by
finding recurring patterns in each individual corpus, i.e. "X (injured/wounded) Y people, Z seriously" where X, Y, Z are variables finding pairings between such patterns the represent paraphrases, i.e. "X (injured/wounded) Y people, Z seriously" and "Y were (wounded/hurt) by X, among them Z were in serious condition" This is achieved by first clustering similar sentences together using n-gram overlap. Recurring patterns are found within clusters by using multi-sequence alignment. Then the position of argument words is determined by finding areas of high variability within each cluster, aka between words shared by more than 50% of a cluster's sentences. Pairings between patterns are then found by comparing similar variable words between different corpora. Finally, new paraphrases can be generated by choosing a matching cluster for a source sentence, then substituting the source sentence's argument into any number of patterns in the cluster.
Phrase-based machine translation Paraphrase can also be generated through the use of phrase-based translation as proposed by Bannard and Callison-Burch. The chief concept consists of aligning phrases in a pivot language to produce potential paraphrases in the original language. For example, the phrase "under control" in an English sentence is aligned with the phrase "unter kontrolle" in its German counterpart. The phrase "unter kontrolle" is then found in another German sentence with the aligned English phrase being "in check," a paraphrase of "under control." The probability distribution can be modeled as Pr ( e 2 | e 1 ) {\displaystyle \Pr(e_{2}|e_{1})} , the probability phrase e 2 {\displaystyle e_{2}} is a paraphrase of e 1 {\displaystyle e_{1}} , which is equivalent to Pr ( e 2 | f ) Pr ( f | e 1 ) {\displaystyle \Pr(e_{2}|f)\Pr(f|e_{1})} summed over all f {\displaystyle f} , a potential phrase translation in the pivot language. Additionally, the sentence e 1 {\displaystyle e_{1}} is added as a prior to add context to the paraphrase. Thus the optimal paraphrase, e 2 ^ {\displaystyle {\hat {e_{2}}}} can be modeled as:
e 2 ^ = arg max e 2 ≠ e 1 Pr ( e 2 | e 1 , S ) = arg max e 2 ≠ e 1 ∑ f Pr ( e 2 | f , S ) Pr ( f | e 1 , S ) {\displaystyle {\hat {e_{2}}}={\text{arg}}\max _{e_{2}\neq e_{1}}\Pr(e_{2}|e_{1},S)={\text{arg}}\max _{e_{2}\neq e_{1}}\sum _{f}\Pr(e_{2}|f,S)\Pr(f|e_{1},S)}
Pr ( e 2 | f ) {\displaystyle \Pr(e_{2}|f)} and Pr ( f | e 1 ) {\displaystyle \Pr(f|e_{1})} can be approximated by simply taking their frequencies. Adding S {\displaystyle S} as a prior is modeled by calculating the probability of forming the S {\displaystyle S} when e 1 {\displaystyle e_{1}} is substituted with e 2 {\displaystyle e_{2}} .
… excerpt ends here. Continue reading the full article.
