1. Machine Learning for SQL Use Cases
  2. Concepts
  3. Machine Learning Algorithms
  4. About MDL
  5. Compression and Entropy
  6. Total Entropy

Total Entropy

For a random variable, X, the total entropy is defined as minus the Probability(X) multiplied by the log to the base 2 of the Probability(X). This can be shown to be the variable's most efficient encoding.