computer science

Never-Ending Language Learning

Never-Ending Language Learning is a computer science topic covered in the lgStudy science library. This page brings together a partial reference excerpt, illustrations, worked examples, real-world applications and a short study plan, so you can understand Never-Ending Language Learning rather than just read about it. In short: Never-Ending Language Learning (NELL) system is a semantic machine learning system that as of 2010 was being developed by a research team at Carnegie Mellon University, and supported by grants from DARPA, Google, NSF, and CNPq with portions of the system running on a supercomputing cluster provided by Yahoo!. Process and goals NELL was programmed by its developers to be able to identify a basic set of fundamental se…

Key takeaways

Never-Ending Language Learning belongs to computer science; place it in that map before memorising details.
Learn the definition first, then one example that makes the definition concrete.
Connect Never-Ending Language Learning to a quantity you can measure, compute or draw — that is where exam questions come from.
Reproduce the core statement of Never-Ending Language Learning from memory before moving on to harder problems.

Reference excerpt

Never-Ending Language Learning (NELL) system is a semantic machine learning system that as of 2010 was being developed by a research team at Carnegie Mellon University, and supported by grants from DARPA, Google, NSF, and CNPq with portions of the system running on a supercomputing cluster provided by Yahoo!.

Process and goals

NELL was programmed by its developers to be able to identify a basic set of fundamental semantic relationships between a few hundred predefined categories of data, such as cities, companies, emotions and sports teams. Since the beginning of 2010, the Carnegie Mellon research team has been running NELL around the clock, sifting through hundreds of millions of web pages looking for connections between the information it already knows and what it finds through its search process – to make new connections in a manner that is intended to mimic the way humans learn new information. For example, in encountering the word pair "Pikes Peak", NELL would notice that both words are capitalized and deduce from the second word that it was the name of a mountain, and then build on the relationship of words surrounding those two words to deduce other connections. The goal of NELL and other semantic learning systems, such as IBM's Watson system, is to be able to develop means of answering questions posed by users in natural language with no human intervention in the process. Oren Etzioni of the University of Washington lauded the system's "continuous learning, as if NELL is exercising curiosity on its own, with little human help". By October 2010, NELL has doubled the number of relationships it has available in its knowledge base and has learned 440,000 new facts, with an accuracy of 87%. Team leader Tom M. Mitchell, chairman of the machine learning department at Carnegie Mellon described how NELL "self-corrects when it has more information, as it learns more", though it does sometimes arrive at incorrect conclusions. Accumulated errors, such as the deduction that Internet cookies were a kind of baked good, led NELL to deduce from the phrases "I deleted my Internet cookies" and "I deleted my files" that "computer files" also belonged in the baked goods category. Clear errors like these are corrected every few weeks by the members of the research team and the system is allowed to continue its learning process. By 2018, NELL had "acquired a knowledge base with 120mn diverse, confidence-weighted beliefs (e.g., servedWith(tea,biscuits)), while learning thousands of interrelated functions that continually improve its reading competence over time." As of September 2023, the project's most recently gathered facts dated from February 2019 (according to its Twitter feed) or September 2018 (according to its home page).

Reception In his 2019 book "Human Compatible", Stuart Russell commented that 'Unfortunately NELL has confidence in only 3 percent of its beliefs and relies on human experts to clean out false or meaningless beliefs on a regular basis—such as its beliefs that “Nepal is a country also known as United States” and "value is an agricultural product that is usually cut into basis."' A 2023 paper commented that "While the never-ending part seems like the right approach, NELL still had the drawback that its focus remained much too grounded on object-language descriptions, and relied on web pages as its only source, which significantly influenced the type of grammar, symbolism, slang, etc. analysed."

See also Cognitive architecture Computational models of language acquisition Cyc Darwin among the Machines The Adolescence of P-1

References

External links Project homepage

Worked examples

Example 1 — a first encounter with Never-Ending Language Learning

Start with the simplest possible case. Write down what Never-Ending Language Learning claims or describes in one sentence, then invent the smallest concrete situation in which that sentence is true. In computer science, the smallest case is usually a single object, a single equation or a single measurement. Check that every symbol or term in your sentence has a meaning in that case.

Example 2 — changing one variable

Take the situation from Example 1 and change exactly one quantity: double it, halve it, or set it to zero. Predict what should happen to Never-Ending Language Learning before you calculate. Comparing your prediction with the result is the fastest way to find out whether you understand the idea or only the words.

Example 3 — an exam-style question

Typical questions about Never-Ending Language Learning ask you to (a) state it precisely, (b) apply it to given data, and (c) explain a limitation. Practise writing all three answers in under five minutes; the third part is what separates a full-mark answer from an average one.

Applications of Never-Ending Language Learning

In research: Never-Ending Language Learning appears in computer science research whenever the underlying quantities have to be modelled precisely. Papers usually cite it as a starting assumption and then explore where it breaks down.
In technology and industry: Engineering practice reuses Never-Ending Language Learning in design rules, simulations and safety margins. Knowing the idea lets you read a specification sheet and understand why the numbers look the way they do.
In the classroom: Never-Ending Language Learning is common in secondary-school and first-year university syllabi. It links to neighbouring topics Data mining and machine learning software, Natural language processing software, so understanding it makes those chapters shorter.
In everyday life: Look for Never-Ending Language Learning outside the textbook — in sport, cooking, traffic, electronics or the sky above you. An example you found yourself is remembered far longer than one you were given.

Ask Teacher Smith questions about this articleOpens your AI tutor with a question about “Never-Ending Language Learning” →

Affiliate

Preply — study more efficiently by working with a personal tutor. 50% off.

How to study Never-Ending Language Learning in 20 minutes

Read the reference excerpt below once, without taking notes.
Close the page and write down what Never-Ending Language Learning means in your own words.
Compare your version with the excerpt and mark what you missed.
Work through the three examples above with pen and paper.
Explain Never-Ending Language Learning out loud to somebody else — or to Teacher Smith in the lgStudy chat.

Frequently asked questions

What is Never-Ending Language Learning in simple terms?

Why does Never-Ending Language Learning matter?

Because it connects several computer science ideas at once: it gives you a definition you can apply, a quantity you can calculate, and a way to check whether a result is plausible.

How should I study Never-Ending Language Learning?

Read the excerpt, restate it from memory, then work through the examples and applications listed on this page. The five-step study plan above takes about twenty minutes.

What does this page cover?

It gives you a compact reference excerpt plus original lgStudy explanations, examples, applications and study material on Never-Ending Language Learning.