Shannon Entropy

Shipping

array_entropy

Shannon entropy of an array treated as a probability distribution

Signature

Inputs

dataArrayrequired— Non-negative values interpreted as a (normalized) probability distribution.

Outputs

entropyScalar— Shannon entropy $-\sum_i p_i \ln p_i$ (nats).

Description

Shannon Entropy treats the input array as a probability distribution ${p_{i}}$ and returns its Shannon entropy $H = - \sum_{i} p_{i} ln p_{i}$ — a measure of the distribution's uncertainty or spread. A sharply peaked distribution (all mass on one outcome) has entropy near $0$ ; a uniform distribution has the maximum entropy for its support size.

Entropy is a cornerstone of information theory and appears in coding, feature selection, and complexity analysis. The node is stateless; the natural-log convention returns entropy in nats.

Mathematics

H = - i \sum p_{i} ln p_{i}, p_{i} \geq 0

Examples

Peaked vs uniform

A near-degenerate distribution $[1, 0, 0, 0]$ has $H \approx 0$ (no uncertainty). A uniform $[0.25, 0.25, 0.25, 0.25]$ maximizes entropy for four outcomes at $H = ln 4 \approx 1.386$ nats.

Applications

Quantifying disorder / uncertainty in a histogram or probability mass.
Feature-selection scores and information gain.
Complexity and diversity metrics over discrete distributions.

Neat

Entropy is maximized by the uniform distribution and minimized (zero) by a point mass — a clean, monotone read-out of how 'spread' a distribution is.

Pairs naturally with Array Histogram: bin a signal, feed the counts here, and read the empirical entropy of the source.