ArticleslgStudy

computer science

Newton's method

Newton's method is a computer science topic covered in the lgStudy science library. This page brings together a partial reference excerpt, illustrations, worked examples, real-world applications and a short study plan, so you can understand Newton's method rather than just read about it. In short: In numerical analysis, the Newton–Raphson method, also known simply as Newton's method, named after Isaac Newton and Joseph Raphson, is a root-finding algorithm which produces successively better approximations to the roots (or zeroes) of a real-valued function. The most basic version starts with a real-valued function f, its derivative f′, and an initial guess x0 for a root of f.

Newton's method — main illustration
Newton's method — illustration

Key takeaways

  • Newton's method belongs to computer science; place it in that map before memorising details.
  • Learn the definition first, then one example that makes the definition concrete.
  • Connect Newton's method to a quantity you can measure, compute or draw — that is where exam questions come from.
  • Reproduce the core statement of Newton's method from memory before moving on to harder problems.

Reference excerpt

In numerical analysis, the Newton–Raphson method, also known simply as Newton's method, named after Isaac Newton and Joseph Raphson, is a root-finding algorithm which produces successively better approximations to the roots (or zeroes) of a real-valued function. The most basic version starts with a real-valued function f, its derivative f′, and an initial guess x0 for a root of f. If f satisfies certain assumptions and the initial guess is close, then

x 1 = x 0 − f ( x 0 ) f ′ ( x 0 ) {\displaystyle x_{1}=x_{0}-{\frac {f(x_{0})}{f'(x_{0})}}}

is a better approximation of the root than x0. Geometrically, (x1, 0) is the x-intercept of the tangent to the graph of f at (x0, f(x0)): that is, the improved guess, x1, is the unique root of the linear approximation of f at the initial guess, x0. The process is repeated as

x n + 1 = x n − f ( x n ) f ′ ( x n ) {\displaystyle x_{n+1}=x_{n}-{\frac {f(x_{n})}{f'(x_{n})}}}

until a sufficiently precise value is reached. The number of correct digits roughly doubles with each step. This algorithm is first in the class of Householder's methods, and was succeeded by Halley's method. The method can also be extended to complex functions and to systems of equations.

Description The purpose of Newton's method is to find a root of a function. The idea is to start with an initial guess near a root, approximate the function by its tangent line near the guess, and then take the root of the linear approximation as a next guess at the function's root. This will typically be closer to the function's root than the previous guess, and the method can be iterated.

The best linear approximation to an arbitrary differentiable function f ( x ) {\displaystyle f(x)} near the point x = x n {\displaystyle x=x_{n}} is the tangent line to the curve, with equation

f ( x ) ≈ f ( x n ) + f ′ ( x n ) ( x − x n ) . {\displaystyle f(x)\approx f(x_{n})+f'(x_{n})(x-x_{n}).}

The root of this linear function, the place where it intercepts the ⁠ x {\displaystyle x} ⁠-axis, can be taken as a closer approximate root ⁠ x n + 1 {\displaystyle x_{n+1}} ⁠ if f ′ ( x n ) ≠ 0 {\displaystyle f'(x_{n})\neq 0} :

x n + 1 = x n − f ( x n ) f ′ ( x n ) . {\displaystyle x_{n+1}=x_{n}-{\frac {f(x_{n})}{f'(x_{n})}}.}

The process can be started with any arbitrary initial guess ⁠ x 0 {\displaystyle x_{0}} ⁠, though it will generally require fewer iterations to converge if the guess is close to one of the function's roots. The method will usually converge if ⁠ f ′ ( x 0 ) ≠ 0 {\displaystyle f'(x_{0})\neq 0} ⁠. Furthermore, for a root of multiplicity 1, the convergence is at least quadratic (see Rate of convergence) in some sufficiently small neighbourhood of the root: the number of correct digits of the approximation roughly doubles with each additional step. More details can be found in § Analysis below. Householder's methods are similar but have higher order for even faster convergence. However, the extra computations required for each step can slow down the overall performance relative to Newton's method, particularly if ⁠ f {\displaystyle f} ⁠ or its derivatives are computationally expensive to evaluate.

… excerpt ends here. Continue reading the full article.

Illustrations

Newton's method: An illustration of Newton's method
An illustration of Newton's method
Newton's method: xn+1 is a better approximation than xn for the root x of the function f (blue curve).
xn+1 is a better approximation than xn for the root x of the function f (blue curve).
Newton's method: Iteration typically improves the approximation.
Iteration typically improves the approximation.
Newton's method: The tangent lines of x3 − 2x + 2 at 0 and 1 intersect the x-axis at 1 and 0 respectively, illustrating why Newton's method oscillates between these values for some starting points.
The tangent lines of x3 − 2x + 2 at 0 and 1 intersect the x-axis at 1 and 0 respectively, illustrating why Newton's method oscillates between these values for some starting points.
Newton's method: The milk carton to be constructed
The milk carton to be constructed

Worked examples

Example 1 — a first encounter with Newton's method

Start with the simplest possible case. Write down what Newton's method claims or describes in one sentence, then invent the smallest concrete situation in which that sentence is true. In computer science, the smallest case is usually a single object, a single equation or a single measurement. Check that every symbol or term in your sentence has a meaning in that case.

Example 2 — changing one variable

Take the situation from Example 1 and change exactly one quantity: double it, halve it, or set it to zero. Predict what should happen to Newton's method before you calculate. Comparing your prediction with the result is the fastest way to find out whether you understand the idea or only the words.

Example 3 — an exam-style question

Typical questions about Newton's method ask you to (a) state it precisely, (b) apply it to given data, and (c) explain a limitation. Practise writing all three answers in under five minutes; the third part is what separates a full-mark answer from an average one.

Applications of Newton's method

In research
Newton's method appears in computer science research whenever the underlying quantities have to be modelled precisely. Papers usually cite it as a starting assumption and then explore where it breaks down.
In technology and industry
Engineering practice reuses Newton's method in design rules, simulations and safety margins. Knowing the idea lets you read a specification sheet and understand why the numbers look the way they do.
In the classroom
Newton's method is common in secondary-school and first-year university syllabi. It links to neighbouring topics Isaac Newton, Optimization algorithms and methods, Root-finding algorithms, so understanding it makes those chapters shorter.
In everyday life
Look for Newton's method outside the textbook — in sport, cooking, traffic, electronics or the sky above you. An example you found yourself is remembered far longer than one you were given.

Affiliate

Preply — study more efficiently by working with a personal tutor. 50% off.

How to study Newton's method in 20 minutes

  1. Read the reference excerpt below once, without taking notes.
  2. Close the page and write down what Newton's method means in your own words.
  3. Compare your version with the excerpt and mark what you missed.
  4. Work through the three examples above with pen and paper.
  5. Explain Newton's method out loud to somebody else — or to Teacher Smith in the lgStudy chat.

Frequently asked questions

What is Newton's method in simple terms?

In numerical analysis, the Newton–Raphson method, also known simply as Newton's method, named after Isaac Newton and Joseph Raphson, is a root-finding algorithm which produces successively better approximations to the roots (or zeroes) of a real-valued function. The most basic version starts with a…

Why does Newton's method matter?

Because it connects several computer science ideas at once: it gives you a definition you can apply, a quantity you can calculate, and a way to check whether a result is plausible.

How should I study Newton's method?

Read the excerpt, restate it from memory, then work through the examples and applications listed on this page. The five-step study plan above takes about twenty minutes.

What does this page cover?

It gives you a compact reference excerpt plus original lgStudy explanations, examples, applications and study material on Newton's method.

Tags

  • Isaac Newton
  • Optimization algorithms and methods
  • Root-finding algorithms

Keep exploring