The peculiar complexities of the Hungarian language presented a unique challenge to spell checker programmers in the early days of computer programming.
Background In the early decades of computer technology, creating spell-checking programs for some languages – such as Hungarian – encountered particular difficulties: The English language usually has just two alphabetical variants of a word (its singular and plural forms), so a spellchecker with a database of 50,000 words need only recognise 100,000 words. However, the Hungarian language is completely different as it is agglutinative. This means that basic words have several alphabetical variants depending on the former, conjugations and suffixes used. So a spellchecker with a database of 50,000 words would need to grow by a factor of two to three thousand to recognise all variants of these words.
NyelvÉsz NyelvÉsz was a Hungarian spellchecker and corrector programme created in the late 1980s by computer engineer and programmer Tibor Béres, linguist Lajos Seregy, and co-programmers József Vanczák and Miklós Hámori. The name of the program was a play on words: the combination of the Hungarian words for language (nyelv) and mind (ész) literally means LinguIst ("NyelvÉsz"). The inherent complexity of the Hungarian language lead to the creation of a new programming model that could subsequently be universally applied to languages: the generative model. The novelty of the approach lay in the fact that the programme was not a parser, but an assembler, a constructor, a generator of words: The word database of the programme only contained the basic words, together with the information about all possible formers, bundles, conjugations and suffixes that each word could have. It also grouped words with the same substitutability together, and included an algorithm describing how to perform substitutions for each group. In addition, the programming required an unprecedented level of data compression at the time. NyelvÉsz only required 300 KB of memory, which one of its authors visualised by imagining pouring 400 hectolitres of wine into one bottle. In the years after the regime change in Hungary (1989), entrepreneurship, companies and copyright were still in their infancy. The country's isolation from international academic and entrepreneurial life also resulted in a paucity of reliable English-language sources on the subject. Yet, a reliable, independent and in-depth source document from the period: a detailed programme testing supplemented with interviews in the leading computer journal at the time Computerworld Hungary, acknowledged it as the first Hungarian spellchecker, despite demonstrating its limitations and calling for further improvements. One of Nyelvész's authors, linguist Lajos Seregy, consequently referred to it as the first Hungarian spellchecker in the first series of official Hungarian conferences in the field of applied linguistics, where he lectured on the topic for years; transcripts of all lectures presented at these conferences have been subsequently published in reviewed and edited volumes. From a cultural and linguistic perspective, the creation of the first Hungarian spellchecker required basic linguistic research and therefore resulted in a mathematically precise description of the Hungarian language. For example, the number of conjugation paradigms existing within each word category became precisely definable, since the program had to contain all existing algorithms. It has become possible to precisely determine that the number of declensions of nouns in Hungarian is: 248. The number of conjugations, i.e. verbalisation paradigms, is close to 600. From the perspective of programming history (especially in the field of language processing tools), NyelvÉsz proved that the task of creating a Hungarian-language spellchecker was feasible using the tools available at the time. It was presented the IFABO IT-event in 1991, at a time when the general opinion was that the Hungarian "spell checker" was impossible to make – an opinion expressed in the May 1991 issue of the computer magazine Alaplap was still qualifying the task as "very complex", "mathematically ill-defined", and therefore "practically impossible to implement".
LEKTOR, Helyes-e?, Helyeske, Hunspell
One of the NyelvÉsz authors, Lajos Seregy, participated in the development of improved versions under the name LEKTOR. In the rapidly developing field of language technology, alongside LEKTOR, several other spell checkers quickly appeared: Helyes-e? created by Gábor Prószéky, Miklós Pál, László Tihanyi, further developed by Attila Novák; Helyeske created by Mátyás Naszódi, Ernő Farkas and László Elekfi; and Hunspell created by László Németh. The operating principle of these spellcheckers is similar. The difference between them lies in whether the emphasis is placed on the basic vocabulary, the affix accuracy, or the most effective handling of word combinations. While the automatic classification method used for word expansion is pattern-based in the case of Helyes-e, the word classification algorithm also relies on statistical tools in the case of Hunspell.
… excerpt ends here. Continue reading the full article.
