TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Show HN: Prophecy Spark IDE public beta

12 pointsby ibainsalmost 5 years ago

6 comments

ibainsalmost 5 years ago
Raj here, I&#x27;m the founder of Prophecy. Previously, I was the product manager for Apache Hive at Hortonworks (and wrote compilers for CUDA at NVIDIA, and Microsoft). The tooling for data engineers is really poor, and I saw many of my customers struggle with it. It&#x27;s 2020, Elon just send people to Space station, and we can&#x27;t build semi-decent data engineering tools?? - we&#x27;ve got to do better. So we&#x27;ll fix it! Prophecy is unique:<p>- Unlike all existing ETL tools, we&#x27;re code-first and Git centric. So we&#x27;re perhaps we are the first Data Engineering tool.<p>- We want to give Apple like high quality design, and it-just-works experience.<p>We&#x27;ve built a unique CODE=VISUAL IDE where you can toggle between visual and code editors instantaneously. You can edit code, and the changes show up in visual graph, and vice-versa. So even if you use visual drag-and-drop view to edit a workflow, you&#x27;re doing Git commits. You can just go to the underlying Git folder and do maven build. Same for unit tests. We&#x27;ve added one-click spin up of Databricks cluster from IDE and step-by-step execution.<p>We want to enable you to:<p>- Develop 10x better code for Spark<p>- Develop 10x faster on Spark<p>- Deploy to production 10x faster with CI&#x2F;CD on Spark&#x2F;Airflow.<p>For the beta, when you submit your e-mail, we create an account and send you login details. You can create Spark workflows, spin up clusters (in-built, on Databricks, free - please be kind with usage) and execute these. Please try out and give feedback, help us make data engineering better!
abhishek_vijalmost 5 years ago
Looking forward to seeing launch of the beta release at #ApaxheSpark conference 2020!
tarunazalmost 5 years ago
This is great! Very cool that you can switch between visual and code environments so seamlessly.
roopsinghalmost 5 years ago
Does it also support managed cloud offerings such as EMR ?
评论 #23619154 未加载
roopsinghalmost 5 years ago
Does it also support managed environments such as EMR ?
nealdaalmost 5 years ago
Awesome - great to hear