TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Ask HN: Good/bad idea – universal API for cloud app datasets?

1 pointsby itsderek23about 6 years ago
My background is as a software engineer at SMBs and startups, which typically lack a data scientist. Because I love numbers, charts, etc. I do a lot of amateur data science. I enjoy the analysis, but I despise the data preprocessing: it&#x27;s common for an SMB to store data across 10+ cloud apps (Salesforce, Google Ads, Hubspot, Mixpanel, etc) but it&#x27;s uncommon for them to store this data in a data warehouse. This means I need to learn each app&#x27;s API and write scripts to fetch data, transpose data, set proper data types, handle schema changes, network errors, etc.<p>I&#x27;d love an API like `sklearn.datasets`, but for cloud apps, so I&#x27;m thinking of creating one. For example, if I wanted to fetch Saleforce contacts, I&#x27;d simply call `petaldata.Salesforce(API_KEY).contacts().to_frame()` to load contacts into a Pandas Dataframe with proper types.<p>Good&#x2F;bad idea? This would be for folks that aren&#x27;t ready for a full ETL pipeline + data warehouse but want to dig through their data and experiment with ML algorithms quickly.

no comments

no comments