What surprises me is that (BOW model + logistic regression) works just fine in most of the benchmarks(except for Amazon Review), interesting paper anyway. Could it be that because the vocabulary for BOW is limited to 5000, a lot of information is lost?