TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Imitating Deep Learning Dynamics via Stochastic Differential Equations

122 pointsby jiayaozhangover 3 years ago

2 comments

TTProgramsover 3 years ago
You probably want to compress your figures - I think your line plots are stored in some vector format. The paper is 30MB and rendering chokes on those ultra dense figures (and the data resolution is not buying you any information). If the figures are vector format you should convert to png/jpeg etc.
评论 #28874330 未加载
评论 #28875487 未加载
jiayaozhangover 3 years ago
Hello HackerNews! Author here :)<p>TL;DR: We devise a linear SDE&#x2F;ODE model to imitate per-class feature (thinking logits) dynamics of neural nets training based on local elasticity (LE) [1]. We found the emergence of LE implies linear separation of features from different classes as training progresses.<p>The drift matrix of our model has a relatively simple structure; with that estimated, we can simulate the SDE using the forward Euler method, whose results align reasonably well with genuine dynamics.<p>Local elasticity models the phenomenon observed in DNN training: the effect due to training on a sample is greater for samples from the same class, and smaller for samples from different classes. For example, training an image of cats facilitates the model better learns images of other cats while not so for images of, say, dogs.<p>Any comments&#x2F;thoughts&#x2F;questions are most welcome!<p>[0] <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2110.05960" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2110.05960</a> [1] <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;1910.06943" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;1910.06943</a>
评论 #28875507 未加载
评论 #28873441 未加载
评论 #28873006 未加载
评论 #28873901 未加载
评论 #28873342 未加载