Description of methods for estimating Gini indexes, variance estimates and confidence intervals using GiniVarInterval

The aim of this vignette is to describe the various methods for estimating the Gini index, for both infinite and finite populations, as well as the methods for estimating its variance, as implemented in the giniVarCI package. Different confidence intervals for the Gini index are also explained.

To exemplify the use of the different functions, we assume that inequality is measured for a nonnegative and continuous random variable \(Y\). A popular formulation of the Gini index (\(G\)) is defined by (see David, 1968; Kendall and Stuart, 1977; Qin et al., 2010): \[ G = \frac{1}{2 \mu_{Y}} \int_{0}^{+\infty} \int_{0}^{+\infty} |x-y|dF_{Y}(x)dF_{Y}(y), \] where \[\mu_{Y}=E[Y]=\int_{0}^{+\infty}yf(y)dy=\int_{0}^{+\infty}ydF_{Y}(y),\] is the mean of \(Y\), and \(F_{Y}(y)=P(Y\leq y)\) and \(f(y)\) are the cumulative distribution function and the probability density function of \(Y\), respectively.

In practice, the value of \(G\) is estimated by means of a sample \(S\) with size \(n\), which can be selected from either infinite or finite populations (Berger and Gedik Balay, 2020; Muñoz et al., 2023).

Infinite populations

Estimators of the Gini index

For infinite populations, \(\{Y_{i}: i\in S\}\) denotes a sequence, with size \(n\), of nonnegative random variables with the same distribution as the variable of interest \(Y\). The Gini index (\(G\)) is estimated using the observation of individuals selected in the sample, which are denoted as \(\{y_{i}: i\in S\}\). A popular estimator of the Gini index is (see Langel and Tille, 2013; Giorgi and Gigliarano, 2017; Muñoz et al., 2023): \[\widehat{G} = \displaystyle \frac{2}{\overline{y}n^2}\sum_{i \in S}iy_{(i)} - \frac{n+1}{n}, \] where \(\overline{y}=n^{-1}\sum_{i=1}^{n}y_i\), and \(y_{(i)}\) are the ordered values (in non-decreasing order) of the sample observations \(y_i\). This is the expression computed by the functions iginindex() (method = 5) and igini() when bias.correction = FALSE.

The estimator \(\widehat{G}\) can be biased for small sample sizes (Deltas, 2003). The bias corrected (bc) version of \(\widehat{G}\) is: \[\widehat{G}^{bc} = \displaystyle \frac{2}{\overline{y}n(n-1)}\sum_{i \in S}iy_{(i)} - \frac{n+1}{n-1},\] which corresponds to the Gini index bias correction version computed by iginindex() (method = 5) and igini() when bias.correction = TRUE.

In the first example, a sample with size n=100 is generated using the gsample() function from the standard logNormal distribution (distribution = "lognormal") with true Gini index is \(G=0.5\) (gini = 0.5) and the Gini index is estimated using bias correction.

library(giniVarCI)
set.seed(123)
y <- gsample(n = 100, gini = 0.5, distribution = "lognormal")
igini(y)
#> [1] 0.4671929

iginindex() can be used to estimate the Gini index using various estimation methods and both R and C++ codes. See help(iginindex) for a detailed description of the various estimation methods. Efficiency comparisons between both implementations and with other functions available in other packages, such as laeken, DescTools, ineq or REAT, can be made using, for instance, the function microbenchmark():



#Comparing the computation time for the various estimation methods using R 
microbenchmark::microbenchmark(
iginindex(y, method = 1,  useRcpp = FALSE),
iginindex(y, method = 2,  useRcpp = FALSE),
iginindex(y, method = 3,  useRcpp = FALSE),
iginindex(y, method = 4,  useRcpp = FALSE),
iginindex(y, method = 5,  useRcpp = FALSE),
iginindex(y, method = 6,  useRcpp = FALSE),
iginindex(y, method = 7,  useRcpp = FALSE),
iginindex(y, method = 8,  useRcpp = FALSE),
iginindex(y, method = 9,  useRcpp = FALSE),
iginindex(y, method = 10, useRcpp = FALSE) 
)
#> Unit: microseconds
#>                                        expr      min        lq       mean
#>   iginindex(y, method = 1, useRcpp = FALSE)  104.364  125.1570  159.52238
#>   iginindex(y, method = 2, useRcpp = FALSE)    9.044   12.7725   17.64412
#>   iginindex(y, method = 3, useRcpp = FALSE)    7.816   10.2075   13.37108
#>   iginindex(y, method = 4, useRcpp = FALSE)   10.656   15.2075   19.37271
#>   iginindex(y, method = 5, useRcpp = FALSE)    9.300   12.2815   16.89826
#>   iginindex(y, method = 6, useRcpp = FALSE)   17.432   30.5475   51.18394
#>   iginindex(y, method = 7, useRcpp = FALSE)  551.896  594.8245  685.84675
#>   iginindex(y, method = 8, useRcpp = FALSE)  544.003  582.1915  614.24818
#>   iginindex(y, method = 9, useRcpp = FALSE)  402.412  435.3125  487.89893
#>  iginindex(y, method = 10, useRcpp = FALSE) 6745.311 7082.2905 8081.02734
#>     median        uq       max neval
#>   134.5120  146.4090  2385.011   100
#>    15.5015   18.2950    81.803   100
#>    11.8510   14.8190    45.473   100
#>    17.3200   20.5130    65.686   100
#>    14.4855   18.5345    65.126   100
#>    47.3075   59.5480   453.477   100
#>   639.8540  677.6545  2713.238   100
#>   604.3340  637.0965  1029.672   100
#>   455.9635  482.8545  2591.686   100
#>  7462.3395 9173.7075 11702.957   100


# Comparing the computation time for the various estimation methods using Rcpp
microbenchmark::microbenchmark(
iginindex(y, method = 1),
iginindex(y, method = 2),
iginindex(y, method = 3),
iginindex(y, method = 4),
iginindex(y, method = 5),
iginindex(y, method = 6),
iginindex(y, method = 7),
iginindex(y, method = 8),
iginindex(y, method = 9),
iginindex(y, method = 10) )
#> Unit: microseconds
#>                       expr      min        lq       mean    median        uq
#>   iginindex(y, method = 1)   12.780   14.0900   17.10481   15.9325   18.8020
#>   iginindex(y, method = 2)    7.194    9.3275   12.23946   10.7285   12.4990
#>   iginindex(y, method = 3)    6.997    8.9335   11.67463   10.3105   12.4465
#>   iginindex(y, method = 4)    7.791    9.4875   11.62676   10.4260   11.9670
#>   iginindex(y, method = 5)    6.618    7.8210    9.37523    8.6850    9.9295
#>   iginindex(y, method = 6)    6.561    7.9225   10.34663    9.0900   10.9355
#>   iginindex(y, method = 7)   48.311   51.4140   55.54906   54.0220   57.6115
#>   iginindex(y, method = 8)   12.807   14.6485   17.52210   15.7240   17.9405
#>   iginindex(y, method = 9)   30.129   32.1960   36.11237   34.2205   38.5010
#>  iginindex(y, method = 10) 6659.183 7095.9955 7920.68862 7524.1040 8749.7385
#>        max neval
#>     30.480   100
#>     79.367   100
#>     32.969   100
#>     30.433   100
#>     19.576   100
#>     28.442   100
#>     86.818   100
#>     36.805   100
#>     61.282   100
#>  11025.210   100


# Comparing the computation time for estimates of the Gini index in various R packages.

microbenchmark::microbenchmark(
igini(y),
laeken::gini(y),
DescTools::Gini(y),
ineq::Gini(y),
REAT::gini(y))
#> Registered S3 methods overwritten by 'DescTools':
#>   method   from
#>   lines.Lc ineq
#>   plot.Lc  ineq
#> Unit: microseconds
#>                expr    min      lq       mean   median       uq        max
#>            igini(y)  6.200  8.4940   11.03406  10.0270  12.4295     40.659
#>     laeken::gini(y) 23.760 27.9865  131.42674  31.4410  36.3760   9886.628
#>  DescTools::Gini(y) 36.991 41.4380 9823.14083  49.6555  56.0805 977385.551
#>       ineq::Gini(y) 27.796 32.0530   60.36937  35.8390  39.7245   2385.147
#>       REAT::gini(y) 70.992 81.7970  118.05204 105.8505 114.2535   1655.888
#>  neval
#>    100
#>    100
#>    100
#>    100
#>    100

Variance estimation and confidence intervals

Variance estimators and confidence intervals are described using different methods for the estimator of the non-bias corrected version of Gini index \(\widehat{G}\), since as \[\widehat{G}^{bc} = \frac{n}{n-1}\widehat{G},\] the variance estimators and confidence intervals based on \(\widehat{G}^{bc}\) can be straightforwardly derived. In particular, \[\widehat{V}(\widehat{G}^{bc})=\frac{n^2}{(n-1)^2}\widehat{V}(\widehat{G}).\] Let \([L,U]\) the lower and upper limits of a confidence interval for \(G\) based on \(\widehat{G}\). The confidence interval based on \(\widehat{G}^{bc}\) can be computed as: \[ \left[ \frac{n}{n-1}L, \frac{n}{n-1}U\right].\]

Bootstrap

The argument interval = pbootstrap in the function igini() returns the confidence interval for the Gini index using the percentile bootstrap method. Let \(\{y_{1}^{*}(b),\ldots, y_{n}^{*}(b)\}\) be the \(b\)th bootstrap sample taken from the original sample \(\{y_{1},\ldots, y_{n}\}\) by simple random sampling with replacement, and \(\widehat{G}^{*}(b)\) denotes the estimator \(\widehat{G}\) computed from the \(b\)th bootstrap sample, with \(b=\{1,\ldots,B\}\), being \(B\) the total number of bootstrap samples. For a confidence level \(1-\alpha\), the percentile bootstrap confidence interval is defined as (see Qin et al., 2010): \[\left[ \widehat{G}^{*}_{(\alpha/2)}, \widehat{G}^{*}_{(1-\alpha/2)} \right],\] where \(\widehat{G}^{*}_{(a)}\) is the \(a\)th quantile of the bootstrapped coefficients \(\widehat{G}^{*}(b)\). A variance estimator of the Gini index based on bootstrap is defined as \[\widehat{V}_{B}(\widehat{G})= \displaystyle \frac{1}{B-1}\sum_{b=1}^{B}\left(\widehat{G}^{*}(b) - \overline{G}^{*} \right)^2,\] where \[\overline{G}^{*}=\frac{1}{B}\sum_{b=1}^{B}\widehat{G}^{*}(b).\]


# Gini index estimation and confidence interval using 'pbootstrap',

igini(y, interval = "pbootstrap")
#> $Gini
#> [1] 0.4671929
#> 
#> $Interval
#>          lower     upper
#> [1,] 0.4004204 0.5135315
#> 
#> $Variance
#> [1] 0.0008333577

interval = "BCa" computes the bias-corrected and accelerated bootstrap interval (Davison and Hinkley, 1997). The idea of this confidence interval is to correct for bias due to the skewness in the distribution of bootstrap estimates. The "BCa" confidence interval is defined as: \[\left[ \widehat{G}^{*}_{(\alpha_{1})}, \widehat{G}^{*}_{(\alpha_{2})} \right],\] where \[\alpha_{1}=\phi\left( \widehat{Z}_{0} + \frac{\widehat{Z}_{0} + Z_{\alpha}}{1-\widehat{a} (\widehat{Z}_{0} + \widehat{Z}_{\alpha}) } \right),\]

\[\alpha_{2}=\phi\left( \widehat{Z}_{0} + \frac{\widehat{Z}_{0} + Z_{1-\alpha}}{1-\widehat{a} (\widehat{Z}_{0} + \widehat{Z}_{1-\alpha}) } \right),\] \(\phi(\cdot)\) is the cumulative distribution function of the standard Normal distribution, and \(Z_{a}\) is the \(a\)th quantile of the standard Normal distribution. The bias-correction factor is defined as \[\widehat{Z}_{0}=\phi^{-1}\left(\#\frac{\widehat{G}^{*}(b) - \widehat{G}}{B}\right),\] and the acceleration factor is given by \[\widehat{a}=\frac{\sum_{i \in S}\left(\overline{G} -\widehat{G}_{-i} \right)^3}{6\left\{\sum_{i \in S}\left(\overline{G} -\widehat{G}_{-i} \right)^2\right\}^{3/2}},\] where \(\widehat{G}_{-i}\) are the jackknife estimates defined in the following section, and \[\overline{G} = \frac{1}{n}\sum_{i \in S}\widehat{G}_{-i}.\]


# Gini index estimation and confidence interval using 'Bca'.

igini(y, interval = "BCa")
#> $Gini
#> [1] 0.4671929
#> 
#> $Interval
#>          lower     upper
#> [1,] 0.4178437 0.5280127
#> 
#> $Variance
#> [1] 0.0008051247

Jackknife

The "zjackknife" and "tjackknife" methods compute the variance of the Gini index using the Ogwang Jackknife procedure (Ogwang, 2000; Langel and Tille, 2013). This variance si given by \[\widehat{V}_{J}(\widehat{G})= \displaystyle \frac{n-1}{n}\sum_{i \in S}\left(\widehat{G}_{-i}- \overline{G} \right)^2,\] where \[\widehat{G}_{-i}=\widehat{G}+\frac{2}{\sum_{j \in S}y_j - y_{(i)} }\left[ \frac{y_{(i)} \sum_{j \in S}jy_{(j)}}{n\sum_{j \in S}y_j}+\frac{\sum_{j \in S}jy_{(j)}}{n(n-1)} - \frac{\sum_{j \in S}y_j-\sum_{j=1}^{i}y_{(j)} +iy_{(i)} }{n-1} \right]-\frac{1}{n(n-1)}, \] with \(i=\{1,\ldots,n\}\) being the jackknife estimates, i.e., \(\widehat{G}_{-i}\) is the estimation of the Gini index when the unit \(i\) is removed from the sample. For a confidence level \(1-\alpha\), the "zjackknife" confidence interval is defined as \[\left[\widehat{G} - Z_{1-\alpha/2}\sqrt{\widehat{V}_{J}(\widehat{G})}, \widehat{G} + Z_{1-\alpha/2}\sqrt{\widehat{V}_{J}(\widehat{G})} \right],\] where \(Z_{1-\alpha/2}\) is the \((1-\alpha/2)\)th quantile of the standard Normal distribution.


# Gini index estimation and confidence interval using 'zjackknife'.

igini(y, interval = "zjackknife")
#> $Gini
#> [1] 0.4671929
#> 
#> $Interval
#>          lower     upper
#> [1,] 0.4103563 0.5240296
#> 
#> $Variance
#> [1] 0.0008409313

"tjackknife" sustitutes the critical value \(Z_{1-\alpha/2}\) by critical values computed from the studentized bootstrap. This confidence interval is given by
\[\left[\widehat{G} - t_{J;1-\alpha/2}^{*}\sqrt{\widehat{V}_{J}(\widehat{G})}, \widehat{G} - t_{J;\alpha/2}^{*}\sqrt{\widehat{V}_{J}(\widehat{G})} \right],\] where \(t_{J;a}^{*}\) is the \(a\)th quantile of the values \[ t^{*}_{J}(b)=\frac{\widehat{G}^{*}(b) - \widehat{G}}{\sqrt{\widehat{V}_{J}\left[\widehat{G}^{*}(b)\right]}}\] computed using the bootstrap technique, where \(\widehat{V}_{J}\left[\widehat{G}^{*}(b)\right]\) is the estimated Ogwang Jackknife variance of \(\widehat{G}^{*}(b)\) for the \(b\)th bootstrap sample.


# Gini index estimation and confidence interval using 'tjackknife'.

igini(y, interval = "tjackknife")
#> $Gini
#> [1] 0.4671929
#> 
#> $Interval
#>          lower     upper
#> [1,] 0.4108575 0.5349601
#> 
#> $Variance
#> [1] 0.0008409313

Linearization

The linearization technique for variance estimation (Deville, 1999) has been applied to the following estimators of the Gini index: \[\widehat{G}^{a} = \displaystyle \frac{1}{2\overline{y}n^{2}}\sum_{i \in S}\sum_{j\in S} |y_i-y_j|\] and \[\widehat{G}^{b} = \displaystyle \frac{2}{\overline{y}n}\sum_{i \in S}y_{i}\widehat{F}_{n}(y_{i}) - 1,\] where \[\widehat{F}_{n}(y_i)=\frac{1}{n}\sum_{j \in S}\delta(y_j \leq y_i)\] and \(\delta(\cdot)\) is the indicator variable that takes the value 1 when its argument is true and 0 otherwise. For a given estimator \(\widehat{G}\) and a linearizated variable \(z\), the confidence interval, with confidence level \(1-\alpha\), is defined as:
\[\left[\widehat{G} - Z_{1-\alpha/2}\sqrt{\widehat{V}_{L}(\widehat{G})}, \widehat{G} + Z_{1-\alpha/2}\sqrt{\widehat{V}_{L}(\widehat{G})} \right],\]

where the variance estimator of the Gini index is given by: \[\widehat{V}_{L}(\widehat{G})= \displaystyle \frac{1}{n(n-1)}\sum_{i \in S}\left(z_{i} - \overline{z}\right)^2,\] and \[\overline{z}=\frac{1}{n}\sum_{i \in S}z_{i}.\]

On the one hand, interval = "zalinearization" linearizates the estimator \(\widehat{G}^{a}\), and the corresponding pseudo-values are (see Langel anf Tillé 2013):

\[z_{(i)}^{a}=\frac{1}{\overline{y}}\left[ \frac{2i}{n}\left( y_{(i)} - \widehat{\overline{Y}}_{(i)}\right) + \overline{y} - y_{(i)} - \widehat{G}^{a}\left(\overline{y} + y_{(i)}\right) \right],\] where \[\widehat{\overline{Y}}_{(i)} = \displaystyle \frac{1}{i}\sum_{j = 1}^{i}y_{(j)}.\]

On the other hand, interval = "zblinearization" linearizates the estimator \(\widehat{G}^{b}\), and the corresponding pseudo values are (see Berger, 2008):

\[z_i^{b}=\frac{1}{\overline{y}}\left[ 2y_i\widehat{F}_{n}(y_i) - (\widehat{G}^{b}+1)(y_i+\overline{y})+2\frac{\sum_{j \in S}y_j\delta(y_j \geq y_i)}{n} \right].\]


# Gini index estimation and confidence interval using 'zalinearization'.

igini(y, interval = "zalinearization")
#> $Gini
#> [1] 0.4671929
#> 
#> $Interval
#>          lower     upper
#> [1,] 0.4125876 0.5217982
#> 
#> $Variance
#> [1] 0.0007762



# Gini index estimation and confidence interval using 'zblinearization'.

igini(y, interval = "zblinearization")
#> $Gini
#> [1] 0.4671929
#> 
#> $Interval
#>          lower     upper
#> [1,] 0.4107537 0.5236321
#> 
#> $Variance
#> [1] 0.0008292117

Intervals "talinearization" and "tblinearization" substitute the critical value \(Z_{1-\alpha/2}\) by critical values computed from the Studentized bootstrap. This confidence interval is given by
\[\left[\widehat{G} - t_{L;1-\alpha/2}^{*}\sqrt{\widehat{V}_{L}(\widehat{G})}, \widehat{G} - t_{L;\alpha/2}^{*}\sqrt{\widehat{V}_{L}(\widehat{G})} \right],\] where \(t_{L;a}^{*}\) is the \(a\)th quantile of the values \[ t^{*}_{L}(b)=\frac{\widehat{G}^{*}(b) - \widehat{G}}{\sqrt{\widehat{V}_{L}\left[\widehat{G}^{*}(b)\right]}}.\] \(\widehat{V}_{L}(\cdot)\) is computed using the pseudo-values \(z_{(i)}^{a}\) when interval = "zalinearization", and using the pseudo-values \(z_i^{b}\) when interval = "zblinearization".


# Gini index estimation and confidence interval using 'talinearization'.

igini(y, interval = "talinearization")
#> $Gini
#> [1] 0.4671929
#> 
#> $Interval
#>          lower     upper
#> [1,] 0.4195142 0.5253662
#> 
#> $Variance
#> [1] 0.0007762


# Gini index estimation and confidence interval using 'tblinearization'.

igini(y, interval = "tblinearization")
#> $Gini
#> [1] 0.4671929
#> 
#> $Interval
#>          lower     upper
#> [1,] 0.4224278 0.5329734
#> 
#> $Variance
#> [1] 0.0008292117

Empirical likelihood

Intervals "ELchisq" and "ELboot" compute the empirical likelihood (\(EL\)) method, a nonparametric technique that provides desirable inferences under skewed distributions. The shape of the \(EL\) confidence intervals are determined by the data-driven likelihood ratio function (Owen, 2001). interval = "ELchisq" obtains the \(EL\) confidence interval, with confidence level \(1-\alpha\), for the Gini index as defined by Qin et al. (2010): \[\left\{ \theta|-2R(\theta) \leq \frac{\chi^2_{1;1-\alpha}}{k}\right\}\]

where \[R(\theta)= - \sum_{i \in S} log\{1+\lambda Z(y_i,\theta)\}\] is the log-EL ratio statistic for \(\theta = G\), \[Z(y_i,\theta)=\{2\widehat{F}_{n}(y_i)-1\}y_{i} - \theta y_i,\] \(\lambda\) is the solution to \[ \frac{1}{n}\sum_{i \in S}\frac{Z(y_i,\theta)}{1+Z(y_i,\theta)}=0,\] \(k=\widehat{\sigma}_{2}^{2}/\widehat{\sigma}_{1}^{2}\) is the scaling factor, \[ \widehat{\sigma}_{j}^{2}=\frac{1}{n-1}\sum_{i \in S}\left(u_{ji} - \overline{u}_{j} \right)^2,\] with \(j=\{1,2\}\), \[ \overline{u}_{j} = \frac{1}{n}\sum_{i \in S}u_{ji},\] and \(\chi^2_{1;1-\alpha}\) is the \((1-\alpha)\)th quantile of Chi-Squared distribution with one degree of freedom.


# Gini index estimation and confidence interval using 'ELchisq'.

igini(y, interval = "ELchisq")
#> $Gini
#> [1] 0.4671929
#> 
#> $Interval
#>          lower     upper
#> [1,] 0.4216374 0.5319404
#> 
#> $Variance
#> [1] 0.0008292117

interval = "ELboot" substitutes the critical value based on the Chi-Squared distribution by an empirical critical value based on bootstrap. "ELboot" computes the \(EL\) confidence interval (Qin et al., 2010): \[\left\{ \theta|-2R(\theta) \leq C_{1-\alpha}\right\},\] where \(C_{1-\alpha}\) is the \((1-\alpha)\)th quantile of the values \(\{-R_{1}^{*}(\widehat{G}),\ldots, -R_{B}^{*}(\widehat{G})\}\), and where \(R_{b}^{*}(\widehat{G})\) denotes the value of \(R(\theta)\) computed from the \(b\)th bootstrap sample.


# Gini index estimation and confidence interval using 'ELboot'.

igini(y, interval = "ELboot")
#> $Gini
#> [1] 0.4671929
#> 
#> $Interval
#>          lower     upper
#> [1,] 0.4118394 0.5413343
#> 
#> $Variance
#> [1] 0.0008343201

The function icompareCI() compares the various confidence intervals for the scenario of a sample derived from an infinite population. The argument plotCI = TRUE plots the results derived from the various available methods for constructing confidence intervals.


# Comparisons of variance estimators and confidence intervals.

icompareCI(y, plotCI = FALSE)
#>           interval    bc gini lowerlimit upperlimit var.gini
#> 1       zjackknife FALSE 0.46       0.41       0.52    8e-04
#> 2       zjackknife  TRUE 0.47       0.41       0.52    8e-04
#> 3       tjackknife FALSE 0.46       0.41       0.53    8e-04
#> 4       tjackknife  TRUE 0.47       0.42       0.54    8e-04
#> 5  zalinearization FALSE 0.46       0.41       0.52    8e-04
#> 6  zalinearization  TRUE 0.47       0.41       0.52    8e-04
#> 7  talinearization FALSE 0.46       0.41       0.52    8e-04
#> 8  talinearization  TRUE 0.47       0.42       0.52    8e-04
#> 9  zblinearization FALSE 0.46       0.41       0.52    8e-04
#> 10 zblinearization  TRUE 0.47       0.41       0.52    8e-04
#> 11 tblinearization FALSE 0.46       0.42       0.53    8e-04
#> 12 tblinearization  TRUE 0.47       0.42       0.53    8e-04
#> 13      pbootstrap FALSE 0.46       0.40       0.51    8e-04
#> 14      pbootstrap  TRUE 0.47       0.40       0.51    8e-04
#> 15             BCa FALSE 0.46       0.42       0.53    7e-04
#> 16             BCa  TRUE 0.47       0.42       0.53    8e-04
#> 17         ELchisq FALSE 0.46       0.42       0.53    8e-04
#> 18         ELchisq  TRUE 0.47       0.42       0.53    8e-04
#> 19          ELboot FALSE 0.46       0.41       0.53    7e-04
#> 20          ELboot  TRUE 0.47       0.42       0.54    7e-04

Finite populations

Estimators of the Gini index

For a finite population \(U\), \(\{Y_{i}: i\in U\}\) denotes a sequence, with size \(N\), of nonnegative random variables with the same distribution as the variable of interest \(Y\), and \(\{y_{i}: i\in U\}\) are the population values of the variable of interest. A sample \(S\) is selected from \(U\) by using a sampling design with survey weights \(w_i\), with \(i\in S\). For example, the survey weights can be \(w_i = \pi_{i}^{-1}\), where \(\pi_{i}=P(i\in S)\) are the inclusion probabilities (Muñoz et al., 2023). The Gini index (\(G\)) is estimated using the observations of individuals selected in the sample \(\{y_{i}: i\in S\}\), and the corresponding survey weights \(\{w_{i}: i\in S\}\). The different methods for estimating the Gini index are (see also Muñoz et al., 2023):

  • method = 1 (Langel and Tillé, 2013).

\[\widehat{G}_{w1}= \displaystyle \frac{1}{2\widehat{N}^{2}\overline{y}_{w}}\sum_{i \in S}\sum_{j \in S}w_{i}w_{j}|y_{i}-y_{j}|,\] where \(\widehat{N}=\sum_{i \in S}w_i\) and \[\overline{y}_{w}= \frac{1}{\widehat{N}}\sum_{i \in S}w_{i}y_{i}.\]

  • method = 2 (Alfons and Templ, 2012; Langel and Tillé, 2013).

\[\widehat{G}_{w2} =\displaystyle \frac{2\sum_{i \in S}w_{(i)}^{+}\widehat{N}_{(i)}y_{(i)} -\sum_{i \in S}w_{i}^{2}y_{i} }{\widehat{N}^{2}\overline{y}_{w}}-1,\] where \(y_{(i)}\) are the values \(y_i\) sorted in increasing order, \(w_{(i)}^{+}\) are the values \(w_i\) sorted according to the increasing order of the values \(y_i\), and \(\widehat{N}_{(i)}=\sum_{j=1}^{i}w_{(j)}^{+}\). Note that Langel and Tillé (2013) show that \(\widehat{G}_{w1}=\widehat{G}_{w2}\).

  • method = 3 (Berger, 2008).

\[\widehat{G}_{w3} = \displaystyle \frac{2}{\widehat{N}\overline{y}_{w}}\sum_{i \in S}w_{i}y_{i}\widehat{F}_{w}^{\ast}(y_{i})-1, \] where \[\widehat{F}_{w}^{\ast}(t) = \displaystyle \frac{1}{\widehat{N}}\sum_{i \in S}w_{i}[\delta(y_i < t) + 0.5\delta(y_i = t)]\] is the smooth (mid-point) distribution function.

  • method = 4 (Berger and Gedik-Balay, 2020).

\[\widehat{G}_{w4} = 1 - \displaystyle \frac{\overline{v}_{w}}{\overline{y}_{w}},\] where \(\overline{v}_{w}=\widehat{N}^{-1}\sum_{i \in S}w_{i}v_{i}\) and \[v_{i} = \displaystyle \frac{1}{\widehat{N} - w_{i}}\sum_{ \substack{j \in S\\ j\neq i}}\min(y_{i},y_{j}).\]

  • method = 5 (Lerman and Yitzhaki, 1989).

\[\widehat{G}_{w5} = \displaystyle \frac{2}{\widehat{N}\overline{y}_{w}} \sum_{i \in S} w_{(i)}^{+}[y_{(i)} - \overline{y}_{w}]\left[ \widehat{F}_{w}^{LY}(y_{(i)}) - \overline{F}_{w}^{LY} \right], \] where \[\widehat{F}_{w}^{LY}(y_{(i)}) = \displaystyle \frac{1}{\widehat{N}}\left(\widehat{N}_{(i-1)} + \frac{w_{(i)}^{+}}{2} \right)\] and \[\overline{F}_{w}^{LY}=\frac{1}{\widehat{N}}\sum_{i \in S}w_{(i)}^{+}\widehat{F}_{w}^{LY}(y_{(i)}).\]

In the finite population example, income and weights from the 2006 Austrian EU-SILC data set (laeken package) are used to estimate the Gini index in the Austrian region of Burgenland. The Gini index is estimated using fgini() and method = 2 (the default method).

data(eusilc, package="laeken")
y <- eusilc$eqIncome[eusilc$db040 == "Burgenland"]
w <- eusilc$rb050[eusilc$db040 == "Burgenland"]
fgini(y, w)
#> [1] 0.3205489

fginindex() can be used to estimate the Gini index using various estimation methods and both R and C++ codes. Efficiency comparisons between both implementations and with other functions available in other packages, such as laeken, DescTools, ineq or REAT, can be made using, for example, the function microbenchmark():


#Comparing the computation time for the various estimation methods and using R
microbenchmark::microbenchmark(
fginindex(y, w, method = 1,  useRcpp = FALSE),
fginindex(y, w, method = 2,  useRcpp = FALSE),
fginindex(y, w, method = 3,  useRcpp = FALSE),
fginindex(y, w, method = 4,  useRcpp = FALSE),
fginindex(y, w, method = 5,  useRcpp = FALSE)
)
#> Unit: microseconds
#>                                          expr      min        lq       mean
#>  fginindex(y, w, method = 1, useRcpp = FALSE) 1240.426 1294.9225 1605.35156
#>  fginindex(y, w, method = 2, useRcpp = FALSE)   38.583   49.6765   67.02715
#>  fginindex(y, w, method = 3, useRcpp = FALSE) 2812.811 2955.5500 3839.66555
#>  fginindex(y, w, method = 4, useRcpp = FALSE) 5871.412 6252.2365 7424.26916
#>  fginindex(y, w, method = 5, useRcpp = FALSE)   46.498   70.1475   88.27871
#>     median       uq       max neval
#>  1364.7140 1403.529  4821.270   100
#>    65.8910   80.885   125.017   100
#>  3111.4975 3426.038  8633.000   100
#>  6491.2280 9287.608 10027.040   100
#>    90.5245  103.548   212.839   100


# Comparing the computation time for the various estimation methods and using Rcpp
microbenchmark::microbenchmark(
fginindex(y, w, method = 1),
fginindex(y, w, method = 2),
fginindex(y, w, method = 3),
fginindex(y, w, method = 4),
fginindex(y, w, method = 5)
)
#> Unit: microseconds
#>                         expr     min       lq      mean   median       uq
#>  fginindex(y, w, method = 1) 685.758 711.0885 742.65546 713.9560 790.0315
#>  fginindex(y, w, method = 2)  35.376  38.6105  43.06769  41.9620  43.9435
#>  fginindex(y, w, method = 3) 682.163 707.3015 742.63402 712.2700 777.5850
#>  fginindex(y, w, method = 4) 315.312 324.3810 345.47532 332.0755 368.9740
#>  fginindex(y, w, method = 5)  39.487  42.0540  46.19248  44.7155  48.0910
#>      max neval
#>  874.438   100
#>  126.932   100
#>  884.231   100
#>  414.223   100
#>   77.951   100

# Comparing the computation time for estimates of the Gini index in various R packages.

# Comparing 'method = 2', used also by the laeken package. 

microbenchmark::microbenchmark(
fgini(y,w),
laeken::gini(y,w)
)
#> Unit: microseconds
#>                expr    min     lq     mean  median      uq     max neval
#>         fgini(y, w) 28.461 30.055 32.33698 30.8535 31.9885  89.363   100
#>  laeken::gini(y, w) 34.595 36.139 39.01254 37.0945 38.4255 124.845   100


# Comparing 'method = 5', used also by the DescTools and REAT packages. 

microbenchmark::microbenchmark(
fgini(y,w, method = 5),
DescTools::Gini(y,w),
REAT::gini(y, weighting = w)
)
#> Unit: microseconds
#>                          expr     min       lq      mean   median       uq
#>       fgini(y, w, method = 5)  33.072  35.3285  39.74836  37.8885  40.3575
#>         DescTools::Gini(y, w)  57.915  62.7215  68.92670  65.2620  71.1755
#>  REAT::gini(y, weighting = w) 139.679 152.6750 162.90013 159.6395 168.6070
#>      max neval
#>   90.141   100
#>  130.927   100
#>  292.812   100

Variance estimation and confidence intervals

Jackknife and linearization tecniques compute pseudo-values (named as \(z_{i}\), with \(i \in S\)) that require the use of an expression for the variance estimation. The function fgini() can compute the following type variance estimators using the argument varformula:

  1. The Horvitz-Thompson ("HT") type variance estimator (Hortvitz and Thompson, 1952).

\[\widehat{V}_{HT}(\widehat{G}_{w}) = \displaystyle \sum_{i\in S}\sum_{j\in S}\breve{\Delta}_{ij}w_{i}w_{j}z_{i}z_{j},\] which is computed when varformula = "HT", where \[\breve{\Delta}_{ij}=\displaystyle \frac{\pi_{ij}-\pi_{i}\pi_{j}}{\pi_{ij}}.\]

  1. The Sen-Yates-Grundy ("SYG") type variance estimator (Sen, 1953; Yates and Grundy, 1953).

\[\widehat{V}_{SYG}(\widehat{G}_{w}) = - \displaystyle \frac{1}{2}\sum_{i\in S}\sum_{j\in S}\breve{\Delta}_{ij}(w_{i}z_i-w_{j}z_{j})^{2},\] which is computed when varformula = "SYG".

  1. The Hartley-Rao ("HR") type variance estimator (Hartley and Rao, 1962).

\[\widehat{V}_{HR}(\widehat{G}_{w}) = \displaystyle \frac{1}{n-1}\sum_{i\in S}\sum_{\substack{j \in S\\ j < i}}\left(1-\pi_i-\pi_j + \frac{1}{n}\sum_{k\in U}\pi_{k}^{2} \right)(w_{i}z_i-w_{j}z_{j})^{2},\] which is computed when varformula = "HR".

Note that the "HT" variance estimator may give negative values, and the "SYG" variance estimator is suitable for fixed-size sampling designs. This implies that "SYG" should not be used under Poisson sampling. Fortunately, "HT" always give positive values under this sampling design. We observe that both Horvitz-Thompson and Sen-Yates-Grundy variance estimators depend on second (joint) inclusion probabilities (argument Pij). The Hàjek (1964) approximation \[\pi_{ij}\cong \pi_{i}\pi_{j}\left[1- \displaystyle \frac{(1-\pi_{i})(1-\pi_{j})}{\sum_{i \in S}(1-\pi_{i})} \right]\] is used when the second (joint) inclusion probabilities are not available (Pij = NULL). Note that the Hàjek approximation is suggested for large-entropy sampling designs, large samples, and large populations (see Tille 2006; Berger and Tille, 2009; Haziza et al., 2008; Berger, 2011). For instance, this approximation is not recomended for highly-stratified samples (Berger, 2005). The Hartley-Rao variance estimator requires the first inclusion probabilities at the population level (argument PiU).

Bootstrap

For complex sampling designs, the rescaled bootstrap (Rao el al., 1992; Rust and Rao, 1996) can be used for variance estimation and construction of confidence intervals. interval = "pbootstrap" returns the confidence interval for the Gini index using the rescaled bootstrap with confidence limits obtained by the percentile method. For a given estimator \(\widehat{G}_{w}\) and a confidence level \(1-\alpha\), this confidence interval is given by \[\left[ \widehat{G}^{*}_{w;\alpha/2}, \widehat{G}^{*}_{w;1-\alpha/2} \right],\]

where \(\widehat{G}^{*}_{w;a}\) is the \(a\)th quantile of the bootstrapped coefficients \(\widehat{G}^{*}_{w}(b)\), with \(b=\{1,\ldots,B\}\), and which are obtained by using the expression \(\widehat{G}_{w}\) after substituting the original survey weights \(w_{i}\) by the bootstrap weights \[w_{i}^{*}(b)=w_{i}\frac{r_{i}n}{n-1},\] where \(r_{i}\) is the number of times that \(i\)yh unit is selected by the bootstrap procedure. A variance estimator of the Gini index based on the rescaled bootstrap is defined as: \[\widehat{V}_{B}(\widehat{G}_{w})= \displaystyle \frac{1}{B-1}\sum_{b=1}^{B}\left(\widehat{G}^{*}_{w}(b) - \overline{G}^{*}_{w} \right)^2,\] where \[\overline{G}^{*}_{w}=\frac{1}{B}\sum_{b=1}^{B}\widehat{G}^{*}_{w}(b).\]


# Gini index estimation and confidence interval using 'pbootstrap'.

fgini(y, w, interval = "pbootstrap")
#> $Gini
#> [1] 0.3205489
#> 
#> $Interval
#>          lower     upper
#> [1,] 0.2935952 0.3453333
#> 
#> $Variance
#> [1] 0.0001664895

Jackknife

The "zjackknife" method computes the variance of the Gini index using the jackknife technique. For a given estimator \(\widehat{G}_{w}\), the pseudo-values for variance estimation are defined as (see Berger, 2008): \[z_{i}=\displaystyle \frac{1}{w_{i}}\left(1-\frac{w_{i}}{\widehat{N}}\right)\left(\widehat{G}_{w} - \widehat{G}_{w;-i}\right),\]
where \(\widehat{G}_{w;-i}\) denotes the estimator \(\widehat{G}_{w}\) computed from \(S\setminus\{i\}\), i.e., from the sample \(S\) after removing the \(i\)th unit. For a confidence level \(1-\alpha\), the "zjackknife" confidence interval is defined as \[\left[\widehat{G}_{w} - Z_{1-\alpha/2}\sqrt{\widehat{V}(\widehat{G}_{w})}, \widehat{G}_{w} + Z_{1-\alpha/2}\sqrt{\widehat{V}(\widehat{G}_{w})} \right],\] where the variance \(\widehat{V}(\widehat{G}_{w})\) is computed using the pseudo-values \(z_i\) and any of the aforementioned type variance estimators (Horvitz-Thompson; Sen-Yates-Grundy; or Harley-Rao).


# Gini index estimation and confidence interval using 'zjackknife'.

fgini(y, w, interval = "zjackknife")
#> $Gini
#> [1] 0.3205489
#> 
#> $Interval
#>          lower    upper
#> [1,] 0.2945728 0.346525
#> 
#> $Variance
#> [1] 0.0001756514

Linearization

The linearization technique for variance estimation (Deville, 1999) has been applied to the following estimators of the Gini index: \[\widehat{G}_{w}^{a}= \displaystyle \frac{1}{2\widehat{N}^{2}\overline{y}_{w}}\sum_{i \in S}\sum_{j \in S}w_{i}w_{j}|y_{i}-y_{j}|,\]

and \[\widehat{G}_{w}^{b} = \displaystyle \frac{2}{\widehat{N}\overline{y}_{w}}\sum_{i \in S}w_{i}y_{i}\widehat{F}_{w}(y_{i})-1, \]

where \[\widehat{F}_{w}(t)=\frac{1}{\widehat{N}}\sum_{i \in S}w_i\delta(y_i \leq t)\] For a given estimator \(\widehat{G}_w\) and a linearizated variable \(z\), the confidence interval, with confidence level \(1-\alpha\), is defined as
\[\left[\widehat{G}_w - Z_{1-\alpha/2}\sqrt{\widehat{V}(\widehat{G}_w)}, \widehat{G}_w + Z_{1-\alpha/2}\sqrt{\widehat{V}(\widehat{G}_w)} \right],\]

where the variance \(\widehat{V}(\widehat{G}_w)\) is computed using the corresponding pseudo-values and any of the aforementioned type variance estimators (Horvitz-Thompson; Sen-Yates-Grundy, or Harley-Rao).

On the one hand, interval = "zalinearization" linearizates the estimator \(\widehat{G}_{w}^{a}\), and the corresponding pseudo-values are defined as (Langel anf Tillé 2013):

\[z_{(i)}^{a}=\frac{1}{\widehat{N}^{2}\overline{y}_w}\left[ 2\widehat{N}_{(i)}\left( y_{(i)} - \widehat{\overline{Y}}_{(i)}\right) + \widehat{N}\left\{ \overline{y}_{w} - y_{(i)} - \widehat{G}_{w}^{a}\left(\overline{y}_{w} + y_{(i)} \right) \right\} \right],\] where \[\widehat{\overline{Y}}_{(i)} = \displaystyle \frac{1}{\widehat{N}_{(i)}}\sum_{j=1}^{i}w_{(j)}^{+}y_{(j)}.\]

On the other hand, interval = "zblinearization" linearizates the estimator \(\widehat{G}_{w}^{b}\), and the corresponding pseudo values are (see Berger, 2008):

\[z_i^{b}=\frac{1}{\hat{N}\overline{y}_{w}}\left[ 2y_i\widehat{F}_{w}(y_i) - (\widehat{G}_{w}^{b}+1)(y_i+\overline{y}_{w})+\frac{2}{\hat{N}}\sum_{j \in S}w_jy_j\delta(y_j \geq y_i) \right],\] where \[\widehat{F}_{w}(t) = \displaystyle \frac{1}{\widehat{N}}\sum_{i \in S}w_{i}\delta(y_i \leq t).\]


# Gini index estimation and confidence interval using:
 ## a: The method 2 for point estimation. 
 ## b: The method 'zalinearization' for variance estimation. 
 ## c: The Sen-Yates-Grundy type variance estimator. 
 ## d: The Hàjek approximation for the joint inclusion probabilities. 
fgini(y, w, interval = "zalinearization")
#> $Gini
#> [1] 0.3205489
#> 
#> $Interval
#>          lower    upper
#> [1,] 0.2946057 0.346492
#> 
#> $Variance
#> [1] 0.0001752056

# Gini index estimation and confidence interval using:
 ## a: The method 3 for point estimation. 
 ## b: The method 'zblinearization' for variance estimation. 
 ## c: The Sen-Yates-Grundy type variance estimator. 
 ## d: The Hàjek approximation for the joint inclusion probabilities. 
fgini(y, w, method = 3, interval = "zblinearization")
#> $Gini
#> [1] 0.3205489
#> 
#> $Interval
#>          lower     upper
#> [1,] 0.2944802 0.3466175
#> 
#> $Variance
#> [1] 0.0001769051

References

Alfons, A., and Templ, M. (2012). Estimation of social exclusion indicators from complex surveys: The R package laeken. KU Leuven, Faculty of Business and Economics Working Paper.

Berger, Y. G. (2005). Variance estimation with highly stratified sampling designs with unequal probabilities. Australian & New Zealand Journal of Statistics, 47, 365–373.

Berger, Y. G. (2008). A note on the asymptotic equivalence of jackknife and linearization variance estimation for the Gini Coefficient. Journal of Official Statistics, 24(4), 541-555.

Berger, Y. G. (2011). Asymptotic consistency under large entropy sampling designs with unequal probabilities. Pakistan Journal of Statistics, 27, 407–426.

Berger, Y. G. and Tille, Y. (2009). Sampling with unequal probabilities. In Sample Surveys: Design, Methods and Applications (eds. D. Pfeffermann and C. R. Rao), 39–54. Elsevier, Amsterdam

Berger, Y., and Gedik Balay, İ. (2020). Confidence intervals of Gini coefficient under unequal probability sampling. Journal of Official Statistics, 36(2), 237-249.

David, H.A. (1968). Gini’s mean difference rediscovered. Biometrika, 55, 573–575.

Davison, A. C., and Hinkley, D. V. (1997). Bootstrap Methods and Their Application (Cambridge Series in Statistical and Probabilistic Mathematics, No 1)–Cambridge University Press.

Deville, J.C. (1999). Variance Estimation for Complex Statistics and Estimators: Linearization and Residual Techniques. Survey Methodology, 25, 193–203.

Deltas, G. (2003). The small-sample bias of the Gini coefficient: results and implications for empirical research. Review of Economics and Statistics, 85(1), 226-234.

Giorgi, G. M., and Gigliarano, C. (2017). The Gini concentration index: a review of the inference literature. Journal of Economic Surveys, 31(4), 1130-1148.

Hàjek, J. (1964). Asymptotic theory of rejective sampling with varying probabilities from a finite population. The Annals of Mathematical Statistics, 35, 4, 1491–1523.

Hartley, H. O., and Rao, J. N. K. (1962). Sampling with unequal probabilities and without replacement. The Annals of Mathematical Statistics, 350-374.

Haziza, D., Mecatti, F. and Rao, J. N. K. (2008). Evaluation of some approximate variance estimators under the Rao-Sampford unequal probability sampling design. Metron, LXVI, 91–108.

Horvitz, D. G. and Thompson, D. J. (1952). A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association, 47, 663–685.

Kendall, M., and Stuart, A. (1977). The advanced theory of statistics. Vol. 1: Distribution Theory. London: Griffin.

Langel, M., and Tillé, Y. (2013). Variance estimation of the Gini index: revisiting a result several times published. Journal of the Royal Statistical Society: Series A (Statistics in Society), 176(2), 521-540.

Lerman, R. I., and Yitzhaki, S. (1989). Improving the accuracy of estimates of Gini coefficients. Journal of econometrics, 42(1), 43-47.

Muñoz, J. F., Moya-Fernández, P. J., and Álvarez-Verdejo, E. (2023). Exploring and Correcting the Bias in the Estimation of the Gini Measure of Inequality. Sociological Methods & Research. https://doi.org/10.1177/00491241231176847

Ogwang, T. (2000). A convenient method of computing the Gini index and its standard error. Oxford Bulletin of Economics and Statistics, 62(1), 123-123.

Owen, A. B. (2001). Empirical likelihood. CRC press.

Qin, Y., Rao, J. N. K., and Wu, C. (2010). Empirical likelihood confidence intervals for the Gini measure of income inequality. Economic Modelling, 27(6), 1429-1435.

Rao, J. N. K., Wu, C. F. J., and Yue, K. (1992). Some recent work on resampling methods for complex surveys. Survey methodology, 18(2), 209-217.

Rust, K. F., and Rao, J. N. K. (1996). Variance estimation for complex surveys using replication techniques. Statistical methods in medical research, 5(3), 283-310.

Sen, A. R. (1953). On the estimate of the variance in sampling with varying probabilities. Journal of the Indian Society of Agricultural Statistics, 5, 119–127.

Tillé, Y. (2006). Sampling Algorithms. Springer, New York.

Yates, F., and Grundy, P. M. (1953). Selection without replacement from within strata with probability proportional to size. Journal of the Royal Statistical Society B, 15, 253–261.