Great to see more data engineering in the direction of SecOps.<p>We do something similar with VAST at <a href="https://vast.io" rel="nofollow">https://vast.io</a>. We’re still early, but especially live and retro detection of threat intel is what we are focusing on. Essentially operationalizing security content for detection and response, plus acquiring and extracting context of alerts and telemetry.<p>We have an experimental serverless deployment with Lambda and Fargate, but the majority of our users still collocate VAST near network sensors like Zeek and Suricata.<p>We’re running everything on top of Apache Arrow, storage of telemetry is now also Parquet. The idea is to do everything with open standards to minimize vendor lock-in.