Continuity Corrections for Discrete Distributions Under the Edgeworth Expansion

The Gram–Charlier A series (named in honor of Jørgen Pedersen Gram and Carl Charlier), and the Edgeworth series (named in honor of Francis Ysidro Edgeworth) are series that approximate a probability distribution in terms of its cumulants.[1] The series are the same; but, the arrangement of terms (and thus the accuracy of truncating the series) differ.[2] The key idea of these expansions is to write the characteristic function of the distribution whose probability density function f is to be approximated in terms of the characteristic function of a distribution with known and suitable properties, and to recover f through the inverse Fourier transform.

Gram–Charlier A series [edit]

We examine a continuous random variable. Let f ^ {\displaystyle {\hat {f}}} be the characteristic function of its distribution whose density function is f, and κ r {\displaystyle \kappa _{r}} its cumulants. We expand in terms of a known distribution with probability density function ψ, characteristic function ψ ^ {\displaystyle {\hat {\psi }}} , and cumulants γ r {\displaystyle \gamma _{r}} . The density ψ is generally chosen to be that of the normal distribution, but other choices are possible as well. By the definition of the cumulants, we have (see Wallace, 1958)[3]

f ^ ( t ) = exp [ r = 1 κ r ( i t ) r r ! ] {\displaystyle {\hat {f}}(t)=\exp \left[\sum _{r=1}^{\infty }\kappa _{r}{\frac {(it)^{r}}{r!}}\right]} and
ψ ^ ( t ) = exp [ r = 1 γ r ( i t ) r r ! ] , {\displaystyle {\hat {\psi }}(t)=\exp \left[\sum _{r=1}^{\infty }\gamma _{r}{\frac {(it)^{r}}{r!}}\right],}

which gives the following formal identity:

f ^ ( t ) = exp [ r = 1 ( κ r γ r ) ( i t ) r r ! ] ψ ^ ( t ) . {\displaystyle {\hat {f}}(t)=\exp \left[\sum _{r=1}^{\infty }(\kappa _{r}-\gamma _{r}){\frac {(it)^{r}}{r!}}\right]{\hat {\psi }}(t)\,.}

By the properties of the Fourier transform, ( i t ) r ψ ^ ( t ) {\displaystyle (it)^{r}{\hat {\psi }}(t)} is the Fourier transform of ( 1 ) r [ D r ψ ] ( x ) {\displaystyle (-1)^{r}[D^{r}\psi ](-x)} , where D is the differential operator with respect to x. Thus, after changing x {\displaystyle x} with x {\displaystyle -x} on both sides of the equation, we find for f the formal expansion

f ( x ) = exp [ r = 1 ( κ r γ r ) ( D ) r r ! ] ψ ( x ) . {\displaystyle f(x)=\exp \left[\sum _{r=1}^{\infty }(\kappa _{r}-\gamma _{r}){\frac {(-D)^{r}}{r!}}\right]\psi (x)\,.}

If ψ is chosen as the normal density

ϕ ( x ) = 1 2 π σ exp [ ( x μ ) 2 2 σ 2 ] {\displaystyle \phi (x)={\frac {1}{{\sqrt {2\pi }}\sigma }}\exp \left[-{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}\right]}

with mean and variance as given by f, that is, mean μ = κ 1 {\displaystyle \mu =\kappa _{1}} and variance σ 2 = κ 2 {\displaystyle \sigma ^{2}=\kappa _{2}} , then the expansion becomes

f ( x ) = exp [ r = 3 κ r ( D ) r r ! ] ϕ ( x ) , {\displaystyle f(x)=\exp \left[\sum _{r=3}^{\infty }\kappa _{r}{\frac {(-D)^{r}}{r!}}\right]\phi (x),}

since γ r = 0 {\displaystyle \gamma _{r}=0} for all r > 2, as higher cumulants of the normal distribution are 0. By expanding the exponential and collecting terms according to the order of the derivatives, we arrive at the Gram–Charlier A series. Such an expansion can be written compactly in terms of Bell polynomials as

exp [ r = 3 κ r ( D ) r r ! ] = n = 0 B n ( 0 , 0 , κ 3 , , κ n ) ( D ) n n ! . {\displaystyle \exp \left[\sum _{r=3}^{\infty }\kappa _{r}{\frac {(-D)^{r}}{r!}}\right]=\sum _{n=0}^{\infty }B_{n}(0,0,\kappa _{3},\ldots ,\kappa _{n}){\frac {(-D)^{n}}{n!}}.}

Since the n-th derivative of the Gaussian function ϕ {\displaystyle \phi } is given in terms of Hermite polynomial as

ϕ ( n ) ( x ) = ( 1 ) n σ n H e n ( x μ σ ) ϕ ( x ) , {\displaystyle \phi ^{(n)}(x)={\frac {(-1)^{n}}{\sigma ^{n}}}He_{n}\left({\frac {x-\mu }{\sigma }}\right)\phi (x),}

this gives us the final expression of the Gram–Charlier A series as

f ( x ) = ϕ ( x ) n = 0 1 n ! σ n B n ( 0 , 0 , κ 3 , , κ n ) H e n ( x μ σ ) . {\displaystyle f(x)=\phi (x)\sum _{n=0}^{\infty }{\frac {1}{n!\sigma ^{n}}}B_{n}(0,0,\kappa _{3},\ldots ,\kappa _{n})He_{n}\left({\frac {x-\mu }{\sigma }}\right).}

Integrating the series gives us the cumulative distribution function

F ( x ) = x f ( u ) d u = Φ ( x ) ϕ ( x ) n = 3 1 n ! σ n 1 B n ( 0 , 0 , κ 3 , , κ n ) H e n 1 ( x μ σ ) , {\displaystyle F(x)=\int _{-\infty }^{x}f(u)du=\Phi (x)-\phi (x)\sum _{n=3}^{\infty }{\frac {1}{n!\sigma ^{n-1}}}B_{n}(0,0,\kappa _{3},\ldots ,\kappa _{n})He_{n-1}\left({\frac {x-\mu }{\sigma }}\right),}

where Φ {\displaystyle \Phi } is the CDF of the normal distribution.

If we include only the first two correction terms to the normal distribution, we obtain

f ( x ) 1 2 π σ exp [ ( x μ ) 2 2 σ 2 ] [ 1 + κ 3 3 ! σ 3 H e 3 ( x μ σ ) + κ 4 4 ! σ 4 H e 4 ( x μ σ ) ] , {\displaystyle f(x)\approx {\frac {1}{{\sqrt {2\pi }}\sigma }}\exp \left[-{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}\right]\left[1+{\frac {\kappa _{3}}{3!\sigma ^{3}}}He_{3}\left({\frac {x-\mu }{\sigma }}\right)+{\frac {\kappa _{4}}{4!\sigma ^{4}}}He_{4}\left({\frac {x-\mu }{\sigma }}\right)\right]\,,}

with H e 3 ( x ) = x 3 3 x {\displaystyle He_{3}(x)=x^{3}-3x} and H e 4 ( x ) = x 4 6 x 2 + 3 {\displaystyle He_{4}(x)=x^{4}-6x^{2}+3} .

Note that this expression is not guaranteed to be positive, and is therefore not a valid probability distribution. The Gram–Charlier A series diverges in many cases of interest—it converges only if f ( x ) {\displaystyle f(x)} falls off faster than exp ( ( x 2 ) / 4 ) {\displaystyle \exp(-(x^{2})/4)} at infinity (Cramér 1957). When it does not converge, the series is also not a true asymptotic expansion, because it is not possible to estimate the error of the expansion. For this reason, the Edgeworth series (see next section) is generally preferred over the Gram–Charlier A series.

The Edgeworth series [edit]

Edgeworth developed a similar expansion as an improvement to the central limit theorem.[4] The advantage of the Edgeworth series is that the error is controlled, so that it is a true asymptotic expansion.

Let { Z i } {\displaystyle \{Z_{i}\}} be a sequence of independent and identically distributed random variables with mean μ {\displaystyle \mu } and variance σ 2 {\displaystyle \sigma ^{2}} , and let X n {\displaystyle X_{n}} be their standardized sums:

X n = 1 n i = 1 n Z i μ σ . {\displaystyle X_{n}={\frac {1}{\sqrt {n}}}\sum _{i=1}^{n}{\frac {Z_{i}-\mu }{\sigma }}.}

Let F n {\displaystyle F_{n}} denote the cumulative distribution functions of the variables X n {\displaystyle X_{n}} . Then by the central limit theorem,

lim n F n ( x ) = Φ ( x ) x 1 2 π e 1 2 q 2 d q {\displaystyle \lim _{n\to \infty }F_{n}(x)=\Phi (x)\equiv \int _{-\infty }^{x}{\tfrac {1}{\sqrt {2\pi }}}e^{-{\frac {1}{2}}q^{2}}dq}

for every x {\displaystyle x} , as long as the mean and variance are finite.

The standardization of { Z i } {\displaystyle \{Z_{i}\}} ensures that the first two cumulants of X n {\displaystyle X_{n}} are κ 1 F n = 0 {\displaystyle \kappa _{1}^{F_{n}}=0} and κ 2 F n = 1. {\displaystyle \kappa _{2}^{F_{n}}=1.} Now assume that, in addition to having mean μ {\displaystyle \mu } and variance σ 2 {\displaystyle \sigma ^{2}} , the i.i.d. random variables Z i {\displaystyle Z_{i}} have higher cumulants κ r {\displaystyle \kappa _{r}} . From the additivity and homogeneity properties of cumulants, the cumulants of X n {\displaystyle X_{n}} in terms of the cumulants of Z i {\displaystyle Z_{i}} are for r 2 {\displaystyle r\geq 2} ,

κ r F n = n κ r σ r n r / 2 = λ r n r / 2 1 w h e r e λ r = κ r σ r . {\displaystyle \kappa _{r}^{F_{n}}={\frac {n\kappa _{r}}{\sigma ^{r}n^{r/2}}}={\frac {\lambda _{r}}{n^{r/2-1}}}\quad \mathrm {where} \quad \lambda _{r}={\frac {\kappa _{r}}{\sigma ^{r}}}.}

If we expand the formal expression of the characteristic function f ^ n ( t ) {\displaystyle {\hat {f}}_{n}(t)} of F n {\displaystyle F_{n}} in terms of the standard normal distribution, that is, if we set

ϕ ( x ) = 1 2 π exp ( 1 2 x 2 ) , {\displaystyle \phi (x)={\frac {1}{\sqrt {2\pi }}}\exp(-{\tfrac {1}{2}}x^{2}),}

then the cumulant differences in the expansion are

κ 1 F n γ 1 = 0 , {\displaystyle \kappa _{1}^{F_{n}}-\gamma _{1}=0,}
κ 2 F n γ 2 = 0 , {\displaystyle \kappa _{2}^{F_{n}}-\gamma _{2}=0,}
κ r F n γ r = λ r n r / 2 1 ; r 3. {\displaystyle \kappa _{r}^{F_{n}}-\gamma _{r}={\frac {\lambda _{r}}{n^{r/2-1}}};\qquad r\geq 3.}

The Gram–Charlier A series for the density function of X n {\displaystyle X_{n}} is now

f n ( x ) = ϕ ( x ) r = 0 1 r ! B r ( 0 , 0 , λ 3 n 1 / 2 , , λ r n r / 2 1 ) H e r ( x ) . {\displaystyle f_{n}(x)=\phi (x)\sum _{r=0}^{\infty }{\frac {1}{r!}}B_{r}\left(0,0,{\frac {\lambda _{3}}{n^{1/2}}},\ldots ,{\frac {\lambda _{r}}{n^{r/2-1}}}\right)He_{r}(x).}

The Edgeworth series is developed similarly to the Gram–Charlier A series, only that now terms are collected according to powers of n {\displaystyle n} . The coefficients of n -m/2 term can be obtained by collecting the monomials of the Bell polynomials corresponding to the integer partitions of m. Thus, we have the characteristic function as

f ^ n ( t ) = [ 1 + j = 1 P j ( i t ) n j / 2 ] exp ( t 2 / 2 ) , {\displaystyle {\hat {f}}_{n}(t)=\left[1+\sum _{j=1}^{\infty }{\frac {P_{j}(it)}{n^{j/2}}}\right]\exp(-t^{2}/2)\,,}

where P j ( x ) {\displaystyle P_{j}(x)} is a polynomial of degree 3 j {\displaystyle 3j} . Again, after inverse Fourier transform, the density function f n {\displaystyle f_{n}} follows as

f n ( x ) = ϕ ( x ) + j = 1 P j ( D ) n j / 2 ϕ ( x ) . {\displaystyle f_{n}(x)=\phi (x)+\sum _{j=1}^{\infty }{\frac {P_{j}(-D)}{n^{j/2}}}\phi (x)\,.}

Likewise, integrating the series, we obtain the distribution function

F n ( x ) = Φ ( x ) + j = 1 1 n j / 2 P j ( D ) D ϕ ( x ) . {\displaystyle F_{n}(x)=\Phi (x)+\sum _{j=1}^{\infty }{\frac {1}{n^{j/2}}}{\frac {P_{j}(-D)}{D}}\phi (x)\,.}

We can explicitly write the polynomial P m ( D ) {\displaystyle P_{m}(-D)} as

P m ( D ) = i 1 k i ! ( λ l i l i ! ) k i ( D ) s , {\displaystyle P_{m}(-D)=\sum \prod _{i}{\frac {1}{k_{i}!}}\left({\frac {\lambda _{l_{i}}}{l_{i}!}}\right)^{k_{i}}(-D)^{s},}

where the summation is over all the integer partitions of m such that i i k i = m {\displaystyle \sum _{i}ik_{i}=m} and l i = i + 2 {\displaystyle l_{i}=i+2} and s = i k i l i . {\displaystyle s=\sum _{i}k_{i}l_{i}.}

For example, if m = 3, then there are three ways to partition this number: 1 + 1 + 1 = 2 + 1 = 3. As such we need to examine three cases:

  • 1 + 1 + 1 = 1 · k 1, so we have k 1 = 3, l 1 = 3, and s = 9.
  • 1 + 2 = 1 · k 1 + 2 · k 2, so we have k 1 = 1, k 2 = 1, l 1 = 3, l 2 = 4, and s = 7.
  • 3 = 3 · k 3, so we have k 3 = 1, l 3 = 5, and s = 5.

Thus, the required polynomial is

P 3 ( D ) = 1 3 ! ( λ 3 3 ! ) 3 ( D ) 9 + 1 1 ! 1 ! ( λ 3 3 ! ) ( λ 4 4 ! ) ( D ) 7 + 1 1 ! ( λ 5 5 ! ) ( D ) 5 = λ 3 3 1296 ( D ) 9 + λ 3 λ 4 144 ( D ) 7 + λ 5 120 ( D ) 5 . {\displaystyle {\begin{aligned}P_{3}(-D)&={\frac {1}{3!}}\left({\frac {\lambda _{3}}{3!}}\right)^{3}(-D)^{9}+{\frac {1}{1!1!}}\left({\frac {\lambda _{3}}{3!}}\right)\left({\frac {\lambda _{4}}{4!}}\right)(-D)^{7}+{\frac {1}{1!}}\left({\frac {\lambda _{5}}{5!}}\right)(-D)^{5}\\&={\frac {\lambda _{3}^{3}}{1296}}(-D)^{9}+{\frac {\lambda _{3}\lambda _{4}}{144}}(-D)^{7}+{\frac {\lambda _{5}}{120}}(-D)^{5}.\end{aligned}}}

The first five terms of the expansion are[5]

f n ( x ) = ϕ ( x ) 1 n 1 2 ( 1 6 λ 3 ϕ ( 3 ) ( x ) ) + 1 n ( 1 24 λ 4 ϕ ( 4 ) ( x ) + 1 72 λ 3 2 ϕ ( 6 ) ( x ) ) 1 n 3 2 ( 1 120 λ 5 ϕ ( 5 ) ( x ) + 1 144 λ 3 λ 4 ϕ ( 7 ) ( x ) + 1 1296 λ 3 3 ϕ ( 9 ) ( x ) ) + 1 n 2 ( 1 720 λ 6 ϕ ( 6 ) ( x ) + ( 1 1152 λ 4 2 + 1 720 λ 3 λ 5 ) ϕ ( 8 ) ( x ) + 1 1728 λ 3 2 λ 4 ϕ ( 10 ) ( x ) + 1 31104 λ 3 4 ϕ ( 12 ) ( x ) ) + O ( n 5 2 ) . {\displaystyle {\begin{aligned}f_{n}(x)&=\phi (x)\\&\quad -{\frac {1}{n^{\frac {1}{2}}}}\left({\tfrac {1}{6}}\lambda _{3}\,\phi ^{(3)}(x)\right)\\&\quad +{\frac {1}{n}}\left({\tfrac {1}{24}}\lambda _{4}\,\phi ^{(4)}(x)+{\tfrac {1}{72}}\lambda _{3}^{2}\,\phi ^{(6)}(x)\right)\\&\quad -{\frac {1}{n^{\frac {3}{2}}}}\left({\tfrac {1}{120}}\lambda _{5}\,\phi ^{(5)}(x)+{\tfrac {1}{144}}\lambda _{3}\lambda _{4}\,\phi ^{(7)}(x)+{\tfrac {1}{1296}}\lambda _{3}^{3}\,\phi ^{(9)}(x)\right)\\&\quad +{\frac {1}{n^{2}}}\left({\tfrac {1}{720}}\lambda _{6}\,\phi ^{(6)}(x)+\left({\tfrac {1}{1152}}\lambda _{4}^{2}+{\tfrac {1}{720}}\lambda _{3}\lambda _{5}\right)\phi ^{(8)}(x)+{\tfrac {1}{1728}}\lambda _{3}^{2}\lambda _{4}\,\phi ^{(10)}(x)+{\tfrac {1}{31104}}\lambda _{3}^{4}\,\phi ^{(12)}(x)\right)\\&\quad +O\left(n^{-{\frac {5}{2}}}\right).\end{aligned}}}

Here, φ(j)(x) is the j-th derivative of φ(·) at point x. Remembering that the derivatives of the density of the normal distribution are related to the normal density by ϕ ( n ) ( x ) = ( 1 ) n H e n ( x ) ϕ ( x ) {\displaystyle \phi ^{(n)}(x)=(-1)^{n}He_{n}(x)\phi (x)} , (where H e n {\displaystyle He_{n}} is the Hermite polynomial of order n), this explains the alternative representations in terms of the density function. Blinnikov and Moessner (1998) have given a simple algorithm to calculate higher-order terms of the expansion.

Note that in case of a lattice distributions (which have discrete values), the Edgeworth expansion must be adjusted to account for the discontinuous jumps between lattice points.[6]

Illustration: density of the sample mean of three χ² distributions [edit]

Density of the sample mean of three chi2 variables. The chart compares the true density, the normal approximation, and two Edgeworth expansions.

Take X i χ 2 ( k = 2 ) , i = 1 , 2 , 3 ( n = 3 ) {\displaystyle X_{i}\sim \chi ^{2}(k=2),\,i=1,2,3\,(n=3)} and the sample mean X ¯ = 1 3 i = 1 3 X i {\displaystyle {\bar {X}}={\frac {1}{3}}\sum _{i=1}^{3}X_{i}} .

We can use several distributions for X ¯ {\displaystyle {\bar {X}}} :

  • The exact distribution, which follows a gamma distribution: X ¯ G a m m a ( α = n k / 2 , θ = 2 / n ) = G a m m a ( α = 3 , θ = 2 / 3 ) {\displaystyle {\bar {X}}\sim \mathrm {Gamma} \left(\alpha =n\cdot k/2,\theta =2/n\right)=\mathrm {Gamma} \left(\alpha =3,\theta =2/3\right)} .
  • The asymptotic normal distribution: X ¯ n N ( k , 2 k / n ) = N ( 2 , 4 / 3 ) {\displaystyle {\bar {X}}{\xrightarrow {n\to \infty }}N(k,2\cdot k/n)=N(2,4/3)} .
  • Two Edgeworth expansions, of degrees 2 and 3.

Discussion of results [edit]

  • For finite samples, an Edgeworth expansion is not guaranteed to be a proper probability distribution as the CDF values at some points may go beyond [ 0 , 1 ] {\displaystyle [0,1]} .
  • They guarantee (asymptotically) absolute errors, but relative errors can be easily assessed by comparing the leading Edgeworth term in the remainder with the overall leading term. [7]

See also [edit]

  • Cornish–Fisher expansion
  • Edgeworth binomial tree

References [edit]

  1. ^ Stuart, A., & Kendall, M. G. (1968). The advanced theory of statistics. Hafner Publishing Company.
  2. ^ Kolassa, J. E. (2006). Series approximation methods in statistics (Vol. 88). Springer Science & Business Media.
  3. ^ Wallace, D. L. (1958). "Asymptotic Approximations to Distributions". Annals of Mathematical Statistics. 29 (3): 635–654. doi:10.1214/aoms/1177706528. JSTOR 2237255.
  4. ^ Hall, P. (2013). The bootstrap and Edgeworth expansion. Springer Science & Business Media.
  5. ^ Weisstein, Eric W. "Edgeworth Series". MathWorld.
  6. ^ Kolassa, John E.; McCullagh, Peter (1990). "Edgeworth series for lattice distributions". Annals of Statistics. 18 (2): 981–985. doi:10.1214/aos/1176347637. JSTOR 2242145.
  7. ^ Kolassa, John E. (2006). Series approximation methods in statistics (3rd ed.). Springer. ISBN0387322272.

Further reading [edit]

  • H. Cramér. (1957). Mathematical Methods of Statistics. Princeton University Press, Princeton.
  • Wallace, D. L. (1958). "Asymptotic approximations to distributions". Annals of Mathematical Statistics. 29 (3): 635–654. doi:10.1214/aoms/1177706528.
  • M. Kendall & A. Stuart. (1977), The advanced theory of statistics, Vol 1: Distribution theory, 4th Edition, Macmillan, New York.
  • P. McCullagh (1987). Tensor Methods in Statistics. Chapman and Hall, London.
  • D. R. Cox and O. E. Barndorff-Nielsen (1989). Asymptotic Techniques for Use in Statistics. Chapman and Hall, London.
  • P. Hall (1992). The Bootstrap and Edgeworth Expansion. Springer, New York.
  • "Edgeworth series", Encyclopedia of Mathematics, EMS Press, 2001 [1994]
  • Blinnikov, S.; Moessner, R. (1998). "Expansions for nearly Gaussian distributions" (PDF). Astronomy and Astrophysics Supplement Series. 130: 193–205. arXiv:astro-ph/9711239. Bibcode:1998A&AS..130..193B. doi:10.1051/aas:1998221.
  • Martin, Douglas; Arora, Rohit (2017). "Inefficiency and bias of modified value-at-risk and expected shortfall". Journal of Risk. 19 (6): 59–84. doi:10.21314/JOR.2017.365.
  • J. E. Kolassa (2006). Series Approximation Methods in Statistics (3rd ed.). (Lecture Notes in Statistics #88). Springer, New York.

greenberghaptly.blogspot.com

Source: https://en.wikipedia.org/wiki/Edgeworth_series

0 Response to "Continuity Corrections for Discrete Distributions Under the Edgeworth Expansion"

ارسال یک نظر

Iklan Atas Artikel

Iklan Tengah Artikel 1

Iklan Tengah Artikel 2

Iklan Bawah Artikel