S-estimator

Key takeaways

S-estimator belongs to science; place it in that map before memorising details.
Learn the definition first, then one example that makes the definition concrete.
Connect S-estimator to a quantity you can measure, compute or draw — that is where exam questions come from.
Reproduce the core statement of S-estimator from memory before moving on to harder problems.

Reference excerpt

The goal of S-estimators is to have a simple high-breakdown regression estimator, which share the flexibility and nice asymptotic properties of M-estimators. The name "S-estimators" was chosen as they are based on estimators of scale. We will consider estimators of scale defined by a function ρ {\displaystyle \rho } , which satisfy

R1 – ρ {\displaystyle \rho } is symmetric, continuously differentiable and ρ ( 0 ) = 0 {\displaystyle \rho (0)=0} . R2 – there exists c > 0 {\displaystyle c>0} such that ρ {\displaystyle \rho } is strictly increasing on [ c , ∞ ] {\displaystyle [c,\infty ]}

For any sample { r 1 , . . . , r n } {\displaystyle \{r_{1},...,r_{n}\}} of real numbers, we define the scale estimate s ( r 1 , . . . , r n ) {\displaystyle s(r_{1},...,r_{n})} as the solution of

1 n ∑ i = 1 n ρ ( r i / s ) = K {\textstyle {\frac {1}{n}}\sum _{i=1}^{n}\rho (r_{i}/s)=K} , where K {\displaystyle K} is the expectation value of ρ {\displaystyle \rho } for a standard normal distribution. (If there are more solutions to the above equation, then we take the one with the smallest solution for s; if there is no solution, then we put s ( r 1 , . . . , r n ) = 0 {\displaystyle s(r_{1},...,r_{n})=0} .) Definition: Let ( x 1 , y 1 ) , . . . , ( x n , y n ) {\displaystyle (x_{1},y_{1}),...,(x_{n},y_{n})} be a sample of regression data with p-dimensional x i {\displaystyle x_{i}} . For each vector θ {\displaystyle \theta } , we obtain residuals s ( r 1 ( θ ) , . . . , r n ( θ ) ) {\displaystyle s(r_{1}(\theta ),...,r_{n}(\theta ))} by solving the equation of scale above, where ρ {\displaystyle \rho } satisfy R1 and R2. The S-estimator θ ^ {\displaystyle {\hat {\theta }}} is defined by

θ ^ = min θ s ( r 1 ( θ ) , . . . , r n ( θ ) ) {\displaystyle {\hat {\theta }}=\min _{\theta }\,s(r_{1}(\theta ),...,r_{n}(\theta ))}

and the final scale estimator σ ^ {\displaystyle {\hat {\sigma }}} is then

σ ^ = s ( r 1 ( θ ^ ) , . . . , r n ( θ ^ ) ) {\displaystyle {\hat {\sigma }}=s(r_{1}({\hat {\theta }}),...,r_{n}({\hat {\theta }}))} .

References

Worked examples

Example 1 — a first encounter with S-estimator

Start with the simplest possible case. Write down what S-estimator claims or describes in one sentence, then invent the smallest concrete situation in which that sentence is true. In science, the smallest case is usually a single object, a single equation or a single measurement. Check that every symbol or term in your sentence has a meaning in that case.

Example 2 — changing one variable

Take the situation from Example 1 and change exactly one quantity: double it, halve it, or set it to zero. Predict what should happen to S-estimator before you calculate. Comparing your prediction with the result is the fastest way to find out whether you understand the idea or only the words.

Example 3 — an exam-style question

Typical questions about S-estimator ask you to (a) state it precisely, (b) apply it to given data, and (c) explain a limitation. Practise writing all three answers in under five minutes; the third part is what separates a full-mark answer from an average one.

Applications of S-estimator

In research: S-estimator appears in science research whenever the underlying quantities have to be modelled precisely. Papers usually cite it as a starting assumption and then explore where it breaks down.
In technology and industry: Engineering practice reuses S-estimator in design rules, simulations and safety margins. Knowing the idea lets you read a specification sheet and understand why the numbers look the way they do.
In the classroom: S-estimator is common in secondary-school and first-year university syllabi. It links to neighbouring topics Estimator, Robust regression, so understanding it makes those chapters shorter.
In everyday life: Look for S-estimator outside the textbook — in sport, cooking, traffic, electronics or the sky above you. An example you found yourself is remembered far longer than one you were given.

How to study S-estimator in 20 minutes

Read the reference excerpt below once, without taking notes.
Close the page and write down what S-estimator means in your own words.
Compare your version with the excerpt and mark what you missed.
Work through the three examples above with pen and paper.
Explain S-estimator out loud to somebody else — or to Teacher Smith in the lgStudy chat.

Frequently asked questions

What is S-estimator in simple terms?

The goal of S-estimators is to have a simple high-breakdown regression estimator, which share the flexibility and nice asymptotic properties of M-estimators. The name "S-estimators" was chosen as they are based on estimators of scale.

Why does S-estimator matter?

Because it connects several science ideas at once: it gives you a definition you can apply, a quantity you can calculate, and a way to check whether a result is plausible.

How should I study S-estimator?

Read the excerpt, restate it from memory, then work through the examples and applications listed on this page. The five-step study plan above takes about twenty minutes.

What does this page cover?

It gives you a compact reference excerpt plus original lgStudy explanations, examples, applications and study material on S-estimator.