MIStatlE
中文

← Knowledge mapψ Learning Systems

Deep Learning

5 concepts, 3 written; shares concepts or relations with 4 other subjects. Earlier ideas sit to the left, a region holds one chapter, and relations that leave the subject hang on the cards as tags.

12 earlierlater ψDeep Learning applied in contrast StochasticGradient Methods Policy Gradient ScaledDot-ProductAttention VAE & ELBO LeWorldModel /SIGReg MDPs & Value Functions → Policy Gradient · requires — The objective J(θ) is defined over the MDP's trajectory distribution.Reinforcement Learning · MDPs… → KL Divergence & Cross Entropy → VAE & ELBO · applied in — The regularizer in the ELBO is a KL divergence.Information Theory · KL Diver… → VAE & ELBO → Score Matching · extends to — From a likelihood bound for latent-variable models to objectives that learn the score directly.→ Generative Models · Score Matc…
Relationsposition is order: read along the rail, left to righta tag on a card leads to another subject
NodesMethodModelPaperwrittenplanned3 / 5 written

Select a concept to see its summary and relations3 / 5 written

Deep Learning3/5

Select a node to see its summary and how it relates to the others.

Relations in this subject, in words5

  • KL Divergence & Cross Entropyapplied inVAE & ELBO

    The regularizer in the ELBO is a KL divergence.

  • MDPs & Value FunctionsrequiresPolicy Gradient

    The objective J(θ) is defined over the MDP's trajectory distribution.

  • Stochastic Gradient Methodsapplied inPolicy Gradient

    The update is stochastic gradient ascent on J(θ); its convergence analysis is that of stochastic nonconvex optimization.

  • VAE & ELBOextends toScore Matching

    From a likelihood bound for latent-variable models to objectives that learn the score directly.

  • VAE & ELBOcontrastLeWorldModel / SIGReg

    Both learn latent representations: the VAE reconstructs observations, LeWorldModel predicts only in latent space and relies on SIGReg against collapse.

Notes

Notes in this subject

Filter the notes page by this subject →
Deep LearningPaper Reading LeWorldModel / SIGReg: Why Doesn't It Collapse? Connecting latent prediction, representation collapse and SIGReg, and identifying the conditions the mechanism actually depends on. Read → Deep LearningFoundation Scaled Dot-Product Attention: Why Divide by √dₖ? Deriving the Transformer's scaling factor from dot-product variance, softmax saturation and gradient stability, with a PyTorch implementation. Read →