MIStatlE
中文

← Knowledge map∑ Mathematical Foundations

Probability

4 concepts, 1 written; shares concepts or relations with 4 other subjects. Earlier ideas sit to the left, a region holds one chapter, and relations that leave the subject hang on the cards as tags.

12 earlierlater ∑Probability derives Law of LargeNumbers KL Divergence &Cross Entropy TypicalSet & AEP ProbabilityFlow ODE Shannon Entropy → KL Divergence & Cross Entropy · extends to — KL divergence is relative entropy: entropy extended from one distribution to a pair.Information Theory · Shannon E… → KL Divergence & Cross Entropy → VAE & ELBO · applied in — The regularizer in the ELBO is a KL divergence.→ Generative Models · VAE & ELBO Shannon Entropy → Typical Set & AEP · requires — Entropy sets the size of the typical set, about 2^{nH}.Information Theory · Shannon E… → Conditional Technique → Probability Flow ODE · extends to — Once the score is learned, the SDE can be replaced by an equivalent deterministic ODE.Generative Models · Condition… →
Relationsposition is order: read along the rail, left to righta tag on a card leads to another subject
NodesConceptTheoremMethodwrittenplanned1 / 4 written

Select a concept to see its summary and relations1 / 4 written

Probability1/4

Select a node to see its summary and how it relates to the others.

Relations in this subject, in words5

  • Law of Large NumbersderivesTypical Set & AEP

    Applying the law of large numbers to −log p(X) gives the AEP.

  • Shannon EntropyrequiresTypical Set & AEP

    Entropy sets the size of the typical set, about 2^{nH}.

  • Shannon Entropyextends toKL Divergence & Cross Entropy

    KL divergence is relative entropy: entropy extended from one distribution to a pair.

  • KL Divergence & Cross Entropyapplied inVAE & ELBO

    The regularizer in the ELBO is a KL divergence.

  • Conditional Techniqueextends toProbability Flow ODE

    Once the score is learned, the SDE can be replaced by an equivalent deterministic ODE.