Off topic, but because this is becoming a hobbyhorse of mine, from the Cure53 report:<p><i>In conclusion, it is evident that the time between the two rounds of testing and since the
assessment concluded was well-spent by the StartCom maintainers. The overall leap in the
state of security is considerable and very much praiseworthy. At present, the ultimate
improvement stems from solid dedication to fixing the reported problems appropriately and in a
manner that prevents recurrence. As two most important arguments, it can be noted that the
numbers of bugs decrease significantly and that the vast majority of the previously spotted
issues has been addressed correctly. The current tendency towards improvement can be read
as a good sign. With each passing month, dedication to security appears to grow and positively
affect the StartSSL compound.</i><p>This kind of language drives me crazy. I don't want to single out Cure53 here because I think a lot of firms deliver this kind of stuff. I know iSEC and Matasano did. But not only do I not believe that software security firms are really qualified, after spending a few weeks looking at a project, to evaluate the true quality standards of a dev team, but I also think it's an enormous conflict of interest.<p>It's not the assessor's job to determine whether StartCom is "praiseworthy" or whether their time was "well-spent" or even to provide a trend line. Their job is to find bugs, recommend fixes, and verify those fixes.<p>I'll go even further and say, I don't think software security firms should be writing these kinds of reports at all. Rather, they should authorize their clients to publish their technical reports, which should keep the editorializing dialed way down.<p>I did this kind of consulting work for over 10 years and I can confidently report that no matter what your standards and principles are, as an assessor you have <i>a lot</i> of wiggle room to report findings positively or negatively (or not at all). When the only audience for your report is your client, that doesn't matter so much, as long as you (1) found bugs and (2) they got fixed. But when the audience is the broader public, I think it matters a great deal how things are reported, and the safest way to do that is denuded of all subjectivity.