You Don't Want to Use CSV Files

CSV files mean a lot of work and are expensive… couldn’t agree more. I had the distinct “pleasure” of working with the food plan system for a get of around 60,000 people. Every iteration of every plan option across every demographic of person was controlled by the biggest CSV you’ve seen. And it changed multiple times a year, requiring a series of review meetings among a small group of IT to review and update everything. It was absurd how that entire system ran off a CSV.

I remember not being able to read Pandas parquet files using for example Vaex (<a href="https:&#x2F;&#x2F;pypi.org&#x2F;project&#x2F;vaex&#x2F;" rel="nofollow">https:&#x2F;&#x2F;pypi.org&#x2F;project&#x2F;vaex&#x2F;</a>). And it is a bit disgenuine not to enlist all supported arguments of `pandas.read_parquet`&#x27;s `*kwargs`. But I agree that Parquet is already 100x better than CSV.

Someone is trying to sell something. Fool me twice, shame on me. CSV is a convenient format but omits schema&#x2F;subschema. CSV data are what you make it. With so many implementations so might as well be in twisty tunnels each a little different.

I always thought of CSV as something you import with Excel. I&#x27;ve added export features to plenty of programs, but never really thought about any other possibility for the data besides a spreadsheet app.