One of my proudest moments was finding a bug in SQLite where a corrupted index caused a select statement to segfault Firefox.<p>I jumped through a <i>lot</i> of hoops to get to the point where I got a backtrace that showed me the SQL statement of a corrupted places.sqlite. I then loaded SQLite on the data file, ran the statement and reproduced the segfault. One of their lead devs then got in contact with me, grabbed the data file and fixed the issue.<p>I suspect that not only did my diagnosis lead to a fix for a LOT of Firefox crashes, but it stopped a lot of frustrating crashes on things like iPhones, etc :-)<p>I may not have done the fix, but I took the time to reproduce the problem. It felt damn good :-)<p>P.S. in case anyone is interested, the bug is <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=581946" rel="nofollow">https://bugzilla.mozilla.org/show_bug.cgi?id=581946</a> on Mozilla, and at SQLite it's at <a href="http://www.sqlite.org/src/ci/83395a3d24" rel="nofollow">http://www.sqlite.org/src/ci/83395a3d24</a>
Unfortunately, 100% testing is only effective if you can detect 100% of the errors generated. The only bug I found in sqlite was an off by one in the btree code that was mostly harmless, unless your memory allocator was particularly fussy.<p>I only found the bug, never quite understood it, and after seeing how disturbing the fix was decided some things were best left unlearned. <a href="http://www2.sqlite.org/cgi/src/fdiff?v1=fa113d624d38bcb36700a0244b47f39d57d34efb&v2=8cab7c66c822ae9c37c59a923ffec81927583ee2" rel="nofollow">http://www2.sqlite.org/cgi/src/fdiff?v1=fa113d624d38bcb36700...</a><p>That said, sqlite is one of the most reliable and better designed libraries I've used. Software is hard.
Is there any more widely deployed software in the world than SQLite? Multiple copies (browsers, language runtimes, embedded in other software) on many computers and built into most smartphones and its probably in quite a few TVs and other devices too.<p>Very successful invisible (to non-developers) software.
What I'm more interested in is <i>how</i> SQLite reached such substantial test coverage. What techniques or tools were used to generate the tests? Did someone (drh?) actually sit down, read through the code, and construct a test case for every branch, or did they use tools to facilitate test case construction? How can I apply the same effort to my own software (once it's ready) to improve its reliability?
You should definitely see drh present it in person. He's giving a talk about it in universities around the world, seen it a couple years back, in Poland. It's incredible what effort goes into reliability of what seems such a small thing.<p>Btw, the very same day he persuaded me to move to Fossil.<p>Edit: just to be clear, you should see it for all the good ideas he's explaining. Not for some marketing of a piece o of software.
Well, I told the author of SQLite that a double inner join took forever to complete. He just told me I was wrong, but this inner join worked fine on PostGres, MySql and so on.<p>It annoys me, because neither PostGres not MySql are a binary file you can just run with a query, unlike SQLite which is very convenient for embedding in a desktop app.
My takeaway from this is high-quality software needs several robust test suites, each written with a different testing methodology.<p>A presentation from the Opus audio codec developers ( <a href="http://www.ietf.org/proceedings/82/slides/codec-4.pdf" rel="nofollow">http://www.ietf.org/proceedings/82/slides/codec-4.pdf</a> ) opened my eyes to just how many overlapping approaches you can use. In addition to listing a dizzying array of software tests (similar to the SQLite article), the Opus presentation summarizes the strengths and weaknesses of each approach, which is great for understanding when each approach is appropriate to use.
This Google Talk starting about 32:35 the author talks a little about SQLite testing:
<a href="http://www.youtube.com/watch?v=f428dSRkTs4" rel="nofollow">http://www.youtube.com/watch?v=f428dSRkTs4</a>
Does anyone know the real reason SQLite was removed form HTML5 specs?<p>I known official Mozilla arguments. Which are quite week IMHO.<p>Is this was MS job? They where afraid that browser apps will make desktop apps obsolete?
Anyone have any thoughts on how they might be doing 'automatic' memory/resource leak detection (Section 6.0)? Seems to me it would be hard to do without generating lots of false positive failures.