computer science

Weighted majority algorithm (machine learning)

Weighted majority algorithm (machine learning) is a computer science topic covered in the lgStudy science library. This page brings together a partial reference excerpt, illustrations, worked examples, real-world applications and a short study plan, so you can understand Weighted majority algorithm (machine learning) rather than just read about it. In short: In machine learning, weighted majority algorithm (WMA) is a meta learning algorithm used to construct a compound algorithm from a pool of prediction algorithms, which could be any type of learning algorithms, classifiers, or even real human experts. The algorithm assumes that we have no prior knowledge about the accuracy of the algorithms in the pool, but there are sufficient reasons to believe that one or more will…

Key takeaways

Weighted majority algorithm (machine learning) belongs to computer science; place it in that map before memorising details.
Learn the definition first, then one example that makes the definition concrete.
Connect Weighted majority algorithm (machine learning) to a quantity you can measure, compute or draw — that is where exam questions come from.
Reproduce the core statement of Weighted majority algorithm (machine learning) from memory before moving on to harder problems.

Reference excerpt

In machine learning, weighted majority algorithm (WMA) is a meta learning algorithm used to construct a compound algorithm from a pool of prediction algorithms, which could be any type of learning algorithms, classifiers, or even real human experts. The algorithm assumes that we have no prior knowledge about the accuracy of the algorithms in the pool, but there are sufficient reasons to believe that one or more will perform well. Assume that the problem is a binary decision problem. To construct the compound algorithm, a positive weight is given to each of the algorithms in the pool. The compound algorithm then collects weighted votes from all the algorithms in the pool, and gives the prediction that has a higher vote. If the compound algorithm makes a mistake, the algorithms in the pool that contributed to the wrong predicting will be discounted by a certain ratio β where 0<β<1. It can be shown that the upper bounds on the number of mistakes made in a given sequence of predictions from a pool of algorithms A {\displaystyle \mathbf {A} } is

O ( l o g | A | + m ) {\displaystyle \mathbf {O(log|A|+m)} }

if one algorithm in x i {\displaystyle \mathbf {x} _{i}} makes at most m {\displaystyle \mathbf {m} } mistakes. There are many variations of the weighted majority algorithm to handle different situations, like shifting targets, infinite pools, or randomized predictions. The core mechanism remains similar, with the final performances of the compound algorithm bounded by a function of the performance of the specialist (best performing algorithm) in the pool.

See also Randomized weighted majority algorithm

References

Worked examples

Example 1 — a first encounter with Weighted majority algorithm (machine learning)

Start with the simplest possible case. Write down what Weighted majority algorithm (machine learning) claims or describes in one sentence, then invent the smallest concrete situation in which that sentence is true. In computer science, the smallest case is usually a single object, a single equation or a single measurement. Check that every symbol or term in your sentence has a meaning in that case.

Example 2 — changing one variable

Take the situation from Example 1 and change exactly one quantity: double it, halve it, or set it to zero. Predict what should happen to Weighted majority algorithm (machine learning) before you calculate. Comparing your prediction with the result is the fastest way to find out whether you understand the idea or only the words.

Example 3 — an exam-style question

Typical questions about Weighted majority algorithm (machine learning) ask you to (a) state it precisely, (b) apply it to given data, and (c) explain a limitation. Practise writing all three answers in under five minutes; the third part is what separates a full-mark answer from an average one.

Applications of Weighted majority algorithm (machine learning)

In research: Weighted majority algorithm (machine learning) appears in computer science research whenever the underlying quantities have to be modelled precisely. Papers usually cite it as a starting assumption and then explore where it breaks down.
In technology and industry: Engineering practice reuses Weighted majority algorithm (machine learning) in design rules, simulations and safety margins. Knowing the idea lets you read a specification sheet and understand why the numbers look the way they do.
In the classroom: Weighted majority algorithm (machine learning) is common in secondary-school and first-year university syllabi. It links to neighbouring topics Machine learning algorithms, so understanding it makes those chapters shorter.
In everyday life: Look for Weighted majority algorithm (machine learning) outside the textbook — in sport, cooking, traffic, electronics or the sky above you. An example you found yourself is remembered far longer than one you were given.

Affiliate

Preply — study more efficiently by working with a personal tutor. 50% off.

How to study Weighted majority algorithm (machine learning) in 20 minutes

Read the reference excerpt below once, without taking notes.
Close the page and write down what Weighted majority algorithm (machine learning) means in your own words.
Compare your version with the excerpt and mark what you missed.
Work through the three examples above with pen and paper.
Explain Weighted majority algorithm (machine learning) out loud to somebody else — or to Teacher Smith in the lgStudy chat.

Frequently asked questions

What is Weighted majority algorithm (machine learning) in simple terms?

Why does Weighted majority algorithm (machine learning) matter?

Because it connects several computer science ideas at once: it gives you a definition you can apply, a quantity you can calculate, and a way to check whether a result is plausible.

How should I study Weighted majority algorithm (machine learning)?

Read the excerpt, restate it from memory, then work through the examples and applications listed on this page. The five-step study plan above takes about twenty minutes.

What does this page cover?

It gives you a compact reference excerpt plus original lgStudy explanations, examples, applications and study material on Weighted majority algorithm (machine learning).