I don't think the data is all public.<p>3-letter government agencies in the United States have a considerable spend w/ companies like BBN and Booz Allen Hamilton.<p>I'm not aware at all of NLP tools that are useful "out of the box" without extensive training and customization; I don't think that's going to change ever, although the training and customization may get a lot easier.