TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Clarifying exceptions and visualizing tensor operations in deep learning code

62 点作者 parrt超过 4 年前

6 条评论

jph00超过 4 年前
Be sure to read the last section - really deep work involved, including writing a Python parser! Here&#x27;s a snippet:<p>&quot;I used the built-in Python tokenizer, but built my own parser for the subset of statements (assignments, return) and operations supported by TensorSensor. There is a built-in Python parser, but it generates a different AST than I wanted; plus, filtering the built-in tree structure for just the parts I care about would be about the same amount of work as writing the parser. I built the parser using simple recursive descent, rather than adding another dependency (my friend antlr) to TensorSensor.<p>Next, we need to augment the AST with the values of all subexpressions so it looks like the AST from the previous section. This is a straightforward bottom-up walk of the AST calling eval() on each subexpression, saving the result in the associated node. In order to execute the various bits of the expression in the proper context after an exception, though, TensorSensor has to walk the call stack back down to the specific context that directly invoked the tensor library. The trick is not chasing the call stack too far, down into the tensor library.<p>Once the AST is augmented with partial values, we need to find the smallest subexpressions that evaluate to tensors for visualization purposes. That corresponds to the deepest subtrees that evaluate to tensors, which is the role of tsensor.analysis.smallest_matrix_subexpr().&quot;<p>(Note: the author is also the creator of ANTLR.)
评论 #24704427 未加载
评论 #24707155 未加载
parrt超过 4 年前
One of the biggest challenges when writing code to implement deep learning networks is getting all of the tensor (matrix and vector) dimensions to line up properly, even when using predefined network layers. This article describes a new library called TensorSensor that clarifies exceptions by augmenting messages and visualizing Python code to indicate the shape of tensor variables. It works with Tensorflow, PyTorch, and Numpy, as well as higher-level libraries like Keras and fastai.
hamolton超过 4 年前
This looks like a neat tool, especially for students taking courses like CS 231N! I imagine this already works in Jupyter Notebook? Perhaps this could be integrated into an extension for the VSCode, maybe in the debugger.
评论 #24704420 未加载
blackbear_超过 4 年前
This looks really neat, but I&#x27;ve only felt the need for such a tool with tensorflow. With pytorch and numpy you can just use ipdb or any other debugger to be parachuted right at the exception and inspect shape and value of all the variables you need.
评论 #24710994 未加载
parrt超过 4 年前
BTW, here&#x27;s a link to the twitter thread.<p><a href="https:&#x2F;&#x2F;twitter.com&#x2F;the_antlr_guy&#x2F;status&#x2F;1313569854918590465" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;the_antlr_guy&#x2F;status&#x2F;1313569854918590465</a>
bwasti超过 4 年前
I believe problems solved by this tool generally disappear when using named tensors
评论 #24714055 未加载