In analytic number theory, the Petersson trace formula is a kind of orthogonality relation between coefficients of a holomorphic modular form. It is a specialization of the more general Kuznetsov trace formula. In its simplest form the Petersson trace formula is as follows. Let F {\displaystyle {\mathcal {F}}} be an orthonormal basis of S k ( Γ ( 1 ) ) {\displaystyle S_{k}(\Gamma (1))} , the space of cusp forms of weight k > 2 {\displaystyle k>2} on S L 2 ( Z ) {\displaystyle SL_{2}(\mathbb {Z} )} . Then for any positive integers m , n {\displaystyle m,n} we have
Γ ( k − 1 ) ( 4 π m n ) k − 1 ∑ f ∈ F f ^ ¯ ( m ) f ^ ( n ) = δ m n + 2 π i − k ∑ c > 0 S ( m , n ; c ) c J k − 1 ( 4 π m n c ) , {\displaystyle {\frac {\Gamma (k-1)}{(4\pi {\sqrt {mn}})^{k-1}}}\sum _{f\in {\mathcal {F}}}{\bar {\hat {f}}}(m){\hat {f}}(n)=\delta _{mn}+2\pi i^{-k}\sum _{c>0}{\frac {S(m,n;c)}{c}}J_{k-1}\left({\frac {4\pi {\sqrt {mn}}}{c}}\right),}
where δ {\displaystyle \delta } is the Kronecker delta function, S {\displaystyle S} is the Kloosterman sum and J {\displaystyle J} is the Bessel function of the first kind.
References Henryk Iwaniec: Topics in Classical Automorphic Forms. Graduate Studies in Mathematics 17, American Mathematics Society, Providence, RI, 1991. "Petersson and Kuznetsov trace formulas". Lie Groups and Automorphic Forms. AMS/IP Studies in Advanced Mathematics. Vol. 37. 2006. pp. 147–168. doi:10.1090/amsip/037/04. ISBN 9780821841983.