TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Ask HN: When do you index your logs?

4 pointsby redcirclealmost 12 years ago
Do you write your log-worthy events to a file, and then parse them when you need to analyze them (e.g., Splunk); or do you write the structured event to a structured data store, such as a SQL database, which maintains indices in real time? Both approaches can be configured to perform the parsing and indexing on a remote node.

1 comment

ScottWhighamalmost 12 years ago
I do it nightly to a SQL database. It's my morning routine to review the logs, add firewall rules based on various "things" I find, or order new/replace equipment based on what I see. I have a heavily partitioned SQL database and so my load times for one day are very fast (< 3 mins typically).