Gratis ordbok - engelska svenska tyska lexikon

Exempel

• Shannon entropy H is given by the formula H = - ∑_i p_i \log_b p_i where pi is the probability of character number i showing up in a stream of characters of the given "script".
• Consider a simple digital circuit which has a two-bit input and a two-bit output . Assuming that the two input bits X and Y have mutually independent chances of 50% of being HIGH, then the input combinations (0,0), (0,1), (1,0), and (1,1) each have a 1/4 chance of occurring, so the circuit's Shannon entropy on the input side is H(X,Y) = 4\Big(-{1\over 4} \log_2 {1\over 4}\Big) = 2 . Then the possible output combinations are (0,0), (0,1), and (1,1) with respective chances of 1/4, 1/2, and 1/4 of occurring, so the circuit's Shannon entropy on the output side is H(X \text{and} Y, X \text{or} Y) = 2\Big(-{1\over 4} \log_2 {1\over 4}\Big) - {1\over 2} \log_2 {1\over 2} = 1 + {1\over 2} = 1 {1\over 2} , so the circuit reduces (or "orders") the information going through it by half a bit of Shannon entropy due to its logical irreversibility.