It really depends on what you need to do with the data, but in most cases Python could do this pretty easily with csv.reader (with a \t delimiter for TSV) or xml.etree.ElementTree.iterparse (for XML) in streaming fashion such that you're not loading the whole file at once.
You can leverage ClickHouse to process your music data. ClickHouse supports both TSV[0] and XML[1] data formats.<p>[0] <a href="https://clickhouse.com/docs/en/interfaces/formats#tabseparated" rel="nofollow">https://clickhouse.com/docs/en/interfaces/formats#tabseparat...</a><p>[1] <a href="https://clickhouse.com/docs/en/interfaces/formats#xml" rel="nofollow">https://clickhouse.com/docs/en/interfaces/formats#xml</a>
Great solution is DuckDB:
<a href="https://duckdb.org/docs/data/csv/overview.html" rel="nofollow">https://duckdb.org/docs/data/csv/overview.html</a>
What kind of single music data file is 100gb?<p>Also how is it structured? If it's actually a tab separated value file, consider using something like polars or DuckDB?
For TSV, you might wanna consider importing it into a Sqlite database, then querying it however you please.<p><a href="https://stackoverflow.com/a/35454070/5298150" rel="nofollow">https://stackoverflow.com/a/35454070/5298150</a><p>You can also use datasette & sqlite utils for it<p><a href="https://sqlite-utils.datasette.io/en/stable/cli.html#inserting-csv-or-tsv-data" rel="nofollow">https://sqlite-utils.datasette.io/en/stable/cli.html#inserti...</a>