Phonetic Matching

The idea that &quot;shore&quot; and &quot;sure&quot; are pronounced &quot;almost identically&quot; would depend pretty heavily on your accent. The vowel is pretty different to me.<p>Also, the matches for &quot;sorI&quot; and &quot;sorY&quot; would seem to me to misinterpret the words as having a vowel at the end, rather than a silent vowel. If you&#x27;re using data meant for foreign surnames, the rules of which may differ from English and which might have silent vowels be very rare depending on the original language, of course you may mispronounce English words like this, saying both shore and sure as &quot;sore-ee&quot;.<p>I&#x27;m sure there are much better ways to transcribe orthography to phonetics, probably people have published libraries that do it. From some googling, it seems like some people call this type of library a phonemic transcriber or IPA transcriber.

This short epilogue struck me.<p><pre><code> This past Yom Kippur, my wife and I drove two hours to spend the afternoon at my aunt’s house, with my cousins. As the night drew on, conversation roamed from television shows and books to politics and philosophy. The circle grew as we touched on increasingly sensitive and challenging topics, drawing us in. We didn’t agree, per se. We were engaging in debate as often as we were engaging in conversation. But we all love each other deeply, and the amount of care and restraint that went into how each person expressed their disagreement was palpable.</code></pre>

It&#x27;s about someone using Levenshtein distance for phonetic fitting against text learning about soundex.<p>One way to start playing around with it is to put some stuff in a database: <a href="https:&#x2F;&#x2F;dev.mysql.com&#x2F;doc&#x2F;refman&#x2F;8.4&#x2F;en&#x2F;string-functions.html#function_soundex" rel="nofollow">https:&#x2F;&#x2F;dev.mysql.com&#x2F;doc&#x2F;refman&#x2F;8.4&#x2F;en&#x2F;string-functions.htm...</a><p>(or this module, <a href="https:&#x2F;&#x2F;www.postgresql.org&#x2F;docs&#x2F;current&#x2F;fuzzystrmatch.html" rel="nofollow">https:&#x2F;&#x2F;www.postgresql.org&#x2F;docs&#x2F;current&#x2F;fuzzystrmatch.html</a> if you&#x27;re stuck with PG)

This is one of these cases where inheriting hacked-together piece of crap (English spelling) makes a lot of additional work higher up.<p>Another example is poetry. A regex can find rhymes in Polish. Same postfix == it rhymes.<p>In English it&#x27;s a feat of engineering.

Oh, hello, I didn&#x27;t realize this was shared here! I guess let me know if anyone has any questions. I mostly wrote this piece as part of processing some hard feelings I&#x27;ve been having and seeing shared among Jewish folks around me, but I also ended up learning quite a bit about phonetic encoding algorithms, and I&#x27;ve spent several years at this point steeped in forced alignment via Storyteller.

I created this sheet[0] to tech my kid to learn Tamil using Roman letters and in the process figured it could be useful for kids learning other Indian languages as well.<p>With the history of reading and speaking (Indian) phonetic languages, I think, English would&#x27;ve been much nicer and uniform if the vowels sounded right, esp the long forms.<p>Extending the long forms using orthogonal vowels probably made it complex, especially with the lack of ii and uu.<p>Say for instance, to extend the long form of &quot;o&quot;, &quot;a&quot; was used. Eg: boat, goat. The correct spelling could&#x27;ve been boot, with the original boot spelled as buut.<p>With that notion, door is probably the only word that&#x27;s written and pronounced phonetically correct, with two oo.<p>Curious to know how would such correct phonetic translation aid in the encoding, matching and compression.<p>[0] <a href="https:&#x2F;&#x2F;docs.google.com&#x2F;spreadsheets&#x2F;d&#x2F;15hdVh-oBUngTyigqDdjgugEAgn4Odaw-FxK2xp_4EjM&#x2F;edit?gid=589132794#gid=589132794" rel="nofollow">https:&#x2F;&#x2F;docs.google.com&#x2F;spreadsheets&#x2F;d&#x2F;15hdVh-oBUngTyigqDdjg...</a>

I&#x27;m using a library, stable-ts, for a similar issue with short audio clips and it works well: <a href="https:&#x2F;&#x2F;github.com&#x2F;jianfch&#x2F;stable-ts&#x2F;tree&#x2F;main">https:&#x2F;&#x2F;github.com&#x2F;jianfch&#x2F;stable-ts&#x2F;tree&#x2F;main</a><p>Not sure how it will perform on something long like an audiobook.

Highly related to my paper on why tokenization in LLMs is the devil: <a href="https:&#x2F;&#x2F;paperswithcode.com&#x2F;paper&#x2F;most-language-models-can-be-poets-too-an-ai-1" rel="nofollow">https:&#x2F;&#x2F;paperswithcode.com&#x2F;paper&#x2F;most-language-models-can-be...</a>

I also had to do this in my previous work and I took the phonetic embeddings of reference and transcribed text and ran a dynamic time warping with them.

Im intrigued.. Is this not done just with a phonemizer?<p><pre><code> from phonemizer.phonemize import phonemize text = &quot;hello world&quot; variations = [ phonemize(text, backend=&quot;espeak&quot;, language=&quot;en-us&quot;, strip=True), phonemize(text, backend=&quot;espeak&quot;, language=&quot;en-gb&quot;, strip=True), phonemize(text, backend=&quot;espeak&quot;, language=&quot;en-au&quot;, strip=True), ] </code></pre> I mean, espeak isnt the best but a lot of folks in the ASR&#x2F;Speech world still are using this right?<p>(NB: If you are on iOS check out the inbuilt one - Settings -&gt; Accessibility -&gt; Spoken Content -&gt; Pronounciations. Adding one it has the ability to phonemize to IPA your spoken message. If someone can tell me where that SDK&#x2F;API is they use in that I&#x27;d love to know) for i, variation in enumerate(variations, 1): print(f&quot;Variation {i}: {variation}&quot;)