Renyi Entropy Calculator Online

Explanation

History: Alfred Renyi searched for the most inclusive definition of information measures that would maintain the additivity for independent events and was consistent with the axioms of probability.
He began by applying the Cauchy's functional equation: which states that If p and q are independent, then I(pq)=I(p)+I (q).
With the exception of a normalising constant, this is consistent with Hartley's information content formula I(p)=-log p. Assuming that each event X={x₁,...x_N} has a different probability of {p₁,...p_N} and provides I_k bits of information, the total or overall amount of information for the set is

It is possible to identify this as Shannon's entropy. He argued, however, that there is an implicit presumption in this equation: we take the linear average, even though it is not the only option that may be utilised. For any function g(x) with inverse g^-1, the mean can be calculated using the general theory of means as

When we apply this definition to the I(P), we get

Only two feasible values for g(x) are available when the postulate/axiom of additivity for independent events is applied:

The first form provide the Shannon information and the second form provides

for non negative α different from 1. This results in a parametric family/group of information measures known as Renyi's entropies today.
Shannon is a unique case that can be demonstrated when α → 1

So, Shannon is a special case in Renyi's entropies.

How to use Calculate Renyi Entropy using Calculator?

• You need to enter the value of P - Probabilities in the first input field given in the form. Remember each probability is seperated with comma.
• Then enter the value of α.
• Press the submit button and then the result will automatically appear below.

More Calculation Examples (Renyi Entropy Examples)

• Probabilities - P: 0.1, 0.4, 0.5, 0.7
• α : 0.5
• Renyi Entropy = 2.6351292496666


• Probabilities - P: 0.8, 0.6, 0.3, 0.7
• α : 4.7
• Renyi Entropy = 0.17920169202028


• Probabilities - P: 0.6, 0.3, 0.7, 0.2, 0.4, 0.1
• α : 0.7
• Renyi Entropy = 5.2172698493051


• Probabilities - P: 0.6, 0.3, 0.7, 0.1
• α : 0.2
• Renyi Entropy = 2.126099339802