Entropy and Extropy Measures for Probability Distributions.
RenyiExtropy 
RenyiExtropy provides a comprehensive collection of entropy and extropy measures for discrete probability distributions. All measures use the natural logarithm (results in nats).
Functions
| Function | Description |
|---|---|
shannon_entropy() | Shannon entropy H(P) |
renyi_entropy() | Rényi entropy H_q(P) of order q |
tsallis_entropy() | Tsallis entropy S_q(P) of order q |
normalized_entropy() | Shannon entropy normalised to [0, 1] |
extropy() / shannon_extropy() | Classical (Shannon) extropy J(P) |
renyi_extropy() | Rényi extropy J_q(P) of order q |
max_renyi_extropy() | Maximum Rényi extropy for n outcomes |
conditional_renyi_extropy() | Conditional Rényi extropy J_q(Y|X) |
joint_entropy() | Joint entropy H(X, Y) |
conditional_entropy() | Conditional entropy H(Y|X) |
kl_divergence() | Kullback–Leibler divergence D_KL(P ‖ Q) |
js_divergence() | Jensen–Shannon divergence |
cross_entropy() | Cross-entropy H(P, Q) |
Installation
Install the released version from CRAN:
install.packages("RenyiExtropy")
Or install the development version from GitHub:
# install.packages("remotes")
remotes::install_github("itsmdivakaran/RenyiExtropy")
Quick start
library(RenyiExtropy)
p <- c(0.2, 0.5, 0.3)
# Entropy measures
shannon_entropy(p) # 1.030 nats
renyi_entropy(p, q = 2) # 0.968 nats
tsallis_entropy(p, q = 2) # 0.620
normalized_entropy(p) # 0.937
# Extropy measures
extropy(p) # 0.775 nats
renyi_extropy(p, q = 2) # 0.742 nats
max_renyi_extropy(3) # maximum over uniform 3-outcome distribution
# Divergences
kl_divergence(p, c(0.3, 0.4, 0.3)) # KL(P || Q)
js_divergence(p, c(0.3, 0.4, 0.3)) # symmetric, bounded by log(2)
cross_entropy(p, c(0.3, 0.4, 0.3)) # H(P) + KL(P || Q)
# Joint / conditional measures
Pxy <- matrix(c(0.2, 0.3, 0.1, 0.4), nrow = 2, byrow = TRUE)
joint_entropy(Pxy) # H(X, Y)
conditional_entropy(Pxy) # H(Y | X)
conditional_renyi_extropy(Pxy, q = 2) # J_q(Y | X)
Limit / continuity behaviour
Functions with order parameter q automatically return their q → 1 limit (Shannon entropy or classical extropy) when |q − 1| < 1e-8:
renyi_entropy(p, 1) # equals shannon_entropy(p)
renyi_extropy(p, 1) # equals extropy(p)
tsallis_entropy(p, 1) # equals shannon_entropy(p)
Citation
If you use RenyiExtropy in your research, please cite:
Mahesh Divakaran, G Rajesh, Sreekumar Jayalekshmi (2024). RenyiExtropy: Entropy and Extropy Measures for Probability Distributions. R package version 0.4.0. https://CRAN.R-project.org/package=RenyiExtropy
License
MIT © Divakaran Mahesh, G Rajesh, Sreekumar Jayalekshmi.