TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Tokenizer UI for Mistral and Claude

2 点作者 henry_pulver大约 1 年前

1 comment

henry_pulver大约 1 年前
I use the OpenAI tokenizer UI a lot when prompt engineering.<p>Token count for inputs allows comparison of different data formats (YAML, JSON, TS) and is a crude measure of prompt importance weighting. For outputs it is a relative measure of output speed between prompts (tok&#x2F;s varies by time of day) and a crude measure of compute used in outputs (why “Think step-by-step” works). Token count also determines the cost of a prompt.<p>Since there’s no equivalent for other providers, I built one for Mistral &amp; Anthropic. If it’s useful, I can add other providers too - let me know which you’d like.
评论 #39691761 未加载