Statistics PhD Defense - 03/02/26

Abstract:

This dissertation analyzes the bias and variance of the plug-in estimator for generalized Shannon’s entropy. The setting is discrete data where observations take categorical values and the statistical object of interest is a probability distribution on an alphabet of possible outcomes. In such problems, entropy provides a relabeling-invariant summary of dispersion, but estimation can be challenging when the number of possible outcomes is large relative to the sample size. In addition, on some countably infinite alphabets, the classical Shannon entropy may diverge, motivating alternative entropy-like targets.

Generalized Shannon’s entropy is defined by applying Shannon entropy to an escort reweighting of the underlying probability mass function. An order parameter controls the reweighting and changes how probability mass is emphasized across outcomes. Estimation is carried out using the plug-in principle: empirical frequencies replace unknown probabilities, the empirical escort distribution is formed, and the generalized entropy functional is evaluated at the resulting empirical distribution.

Under a multinomial sampling model on a finite alphabet, explicit finite-sample approximations are derived for the bias and variance of the generalized plug-in estimator, including leading terms and higher-order corrections. These expressions are combined to obtain mean squared error approximations and to motivate analytic bias adjustments and standard-error formulas. Simulation studies validate the theoretical calculations and illustrate how estimation accuracy depends on sample size, distribution shape, alphabet size, and the escort order parameter.

PhD Advisor:

Dr. Jialin Zhang