computer science

Neocognitron

Neocognitron is a computer science topic covered in the lgStudy science library. This page brings together a partial reference excerpt, illustrations, worked examples, real-world applications and a short study plan, so you can understand Neocognitron rather than just read about it. In short: The neocognitron is a hierarchical, multilayered artificial neural network proposed by Kunihiko Fukushima in 1979. It has been used for Japanese handwritten character recognition and other pattern recognition tasks, and served as the inspiration for convolutional neural networks.

Key takeaways

Neocognitron belongs to computer science; place it in that map before memorising details.
Learn the definition first, then one example that makes the definition concrete.
Connect Neocognitron to a quantity you can measure, compute or draw — that is where exam questions come from.
Reproduce the core statement of Neocognitron from memory before moving on to harder problems.

Reference excerpt

The neocognitron is a hierarchical, multilayered artificial neural network proposed by Kunihiko Fukushima in 1979. It has been used for Japanese handwritten character recognition and other pattern recognition tasks, and served as the inspiration for convolutional neural networks. Previously in 1969, he published a similar architecture, but with hand-designed kernels inspired by convolutions in mammalian vision. In 1975 he improved it to the Cognitron, and in 1979 he improved it to the neocognitron, which learns all convolutional kernels by unsupervised learning (in his terminology, "self-organized by 'learning without a teacher'"). The neocognitron was inspired by the model proposed by Hubel & Wiesel in 1959. They found two types of cells in the visual primary cortex called simple cell and complex cell, and also proposed a cascading model of these two types of cells for use in pattern recognition tasks. The neocognitron is a natural extension of these cascading models. The neocognitron consists of multiple types of cells, the most important of which are called S-cells and C-cells. The local features are extracted by S-cells, and these features' deformation, such as local shifts, are tolerated by C-cells. Local features in the input are integrated gradually and classified in the higher layers. The idea of local feature integration is found in several other models, such as the Convolutional Neural Network model, the SIFT method, and the HoG method. There are various kinds of neocognitron. For example, some types of neocognitron can detect multiple patterns in the same input by using backward signals to achieve selective attention.

See also Artificial neural network Deep learning Pattern recognition Receptive field Self-organizing map Unsupervised learning

Notes

References Fukushima, Kunihiko (April 1980). "Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position". Biological Cybernetics. 36 (4): 193–202. doi:10.1007/bf00344251. PMID 7370364. S2CID 206775608. Fukushima, Kunihiko; Miyake, S.; Ito, T. (1983). "Neocognitron: a neural network model for a mechanism of visual pattern recognition". IEEE Transactions on Systems, Man, and Cybernetics. SMC-13 (3): 826–834. doi:10.1109/TSMC.1983.6313076. S2CID 8235461. Fukushima, Kunihiko (1987). "A hierarchical neural network model for selective attention". In Eckmiller, R.; Von der Malsburg, C. (eds.). Neural computers. Springer-Verlag. pp. 81–90. Fukushima, Kunihiko (2007). "Neocognitron". Scholarpedia. 2 (1): 1717. Bibcode:2007SchpJ...2.1717F. doi:10.4249/scholarpedia.1717. Hubel, D.H.; Wiesel, T.N. (1959). "Receptive fields of single neurones in the cat's striate cortex". J Physiol. 148 (3): 574–591. doi:10.1113/jphysiol.1959.sp006308. PMC 1363130. PMID 14403679.

External links Neocognitron on Scholarpedia NeoCognitron by Ing. Gabriel Minarik - application (C#) and video Neocognitron resources at Visiome Platform - includes MATLAB environment Beholder - a Neocognitron simulator

Worked examples

Example 1 — a first encounter with Neocognitron

Start with the simplest possible case. Write down what Neocognitron claims or describes in one sentence, then invent the smallest concrete situation in which that sentence is true. In computer science, the smallest case is usually a single object, a single equation or a single measurement. Check that every symbol or term in your sentence has a meaning in that case.

Example 2 — changing one variable

Take the situation from Example 1 and change exactly one quantity: double it, halve it, or set it to zero. Predict what should happen to Neocognitron before you calculate. Comparing your prediction with the result is the fastest way to find out whether you understand the idea or only the words.

Example 3 — an exam-style question

Typical questions about Neocognitron ask you to (a) state it precisely, (b) apply it to given data, and (c) explain a limitation. Practise writing all three answers in under five minutes; the third part is what separates a full-mark answer from an average one.

Applications of Neocognitron

In research: Neocognitron appears in computer science research whenever the underlying quantities have to be modelled precisely. Papers usually cite it as a starting assumption and then explore where it breaks down.
In technology and industry: Engineering practice reuses Neocognitron in design rules, simulations and safety margins. Knowing the idea lets you read a specification sheet and understand why the numbers look the way they do.
In the classroom: Neocognitron is common in secondary-school and first-year university syllabi. It links to neighbouring topics Artificial neural networks, Computational neuroscience stubs, Machine learning stubs, so understanding it makes those chapters shorter.
In everyday life: Look for Neocognitron outside the textbook — in sport, cooking, traffic, electronics or the sky above you. An example you found yourself is remembered far longer than one you were given.

Affiliate

Preply — study more efficiently by working with a personal tutor. 50% off.

How to study Neocognitron in 20 minutes

Read the reference excerpt below once, without taking notes.
Close the page and write down what Neocognitron means in your own words.
Compare your version with the excerpt and mark what you missed.
Work through the three examples above with pen and paper.
Explain Neocognitron out loud to somebody else — or to Teacher Smith in the lgStudy chat.

Frequently asked questions

What is Neocognitron in simple terms?

Why does Neocognitron matter?

Because it connects several computer science ideas at once: it gives you a definition you can apply, a quantity you can calculate, and a way to check whether a result is plausible.

How should I study Neocognitron?

Read the excerpt, restate it from memory, then work through the examples and applications listed on this page. The five-step study plan above takes about twenty minutes.

What does this page cover?

It gives you a compact reference excerpt plus original lgStudy explanations, examples, applications and study material on Neocognitron.