In information theory, given an unknown stationary source π with alphabet A and a sample w from π, the Krichevsky–Trofimov (KT) estimator produces an estimate pi(w) of the probability of each symbol i ∈ A. This estimator is optimal in the sense that it minimizes the worst-case regret asymptotically. For a binary alphabet and a string w with m zeroes and n ones, the KT estimator pi(w) is defined as:
p 0 ( w ) = m + 1 / 2 m + n + 1 , p 1 ( w ) = n + 1 / 2 m + n + 1 . {\displaystyle {\begin{aligned}p_{0}(w)&={\frac {m+1/2}{m+n+1}},\\[5pt]p_{1}(w)&={\frac {n+1/2}{m+n+1}}.\end{aligned}}}
This corresponds to the posterior mean of a Beta-Bernoulli posterior distribution with prior 1 / 2 {\displaystyle 1/2} . For the general case the estimate is made using a Dirichlet-Categorical distribution.
See also Rule of succession Bayesian inference using conjugate priors for the categorical distribution
References