How do I schedule my transformations so they happen on a recurring basis?<p>I would like to try the tool but keep getting a "Server Error" message which appears to prevent it from fully loading.
> it could be applied to tasks where Spark is normally used<p>How does it relate to Spark? Is it able to read from HDFS, translate to Spark map-reduce jobs or integrate somehow with Spark jobs?
Data Commandr: <a href="http://conceptoriented.com" rel="nofollow">http://conceptoriented.com</a> (Version 0.6.0, 2017-05-14)<p>Data Commandr (DC) is a web application for working with <i>tables</i>. Its goal is to make complex operations with data tables as easy as working with classical spreadsheets (but it is not a spreadsheet application) for producing reports, data integration, data migration and other <i>data wrangling</i> tasks. Its main distinguishing feature is that it does not use traditional (set-oriented) queries but rather relies on <i>column formulas</i> to derive new data.<p>I would like to get your option about Data Commandr from three points of view:<p>- [As developers] How do you like the current UI and general design? How it can be improved?<p>- [As users] Can such a tool be useful for working with tables?<p>- [As investors] What other tasks can be solved by using this approach? For example, I want to pivot in the direction of stream analytics. Or it could be applied to tasks where Spark is normally used (DC might have much higher performance and much simpler data processing language.) Other applications: data dictionary and data catalogs.<p>Thank you in advance for testing the app and your feedback.<p>COMPARISON:<p>- [Spreadsheets] In spreadsheets, a cell is defined as a <i>function</i> of other cells so we get a functional model where the result of each cell (one value) is computed by evaluating other cell formulas. In contrast, DC uses <i>columns</i> instead of cells so that a model is a number of <i>column formulas</i>.<p>- [Set-oriented approaches] In set-oriented approaches (RM, Hadoop, Spark etc.), new data is derived by applying an operation to input sets and producing an output set. Data transformations are represented as a graph of set operations. In contrast, data transformations in DC are represented as a graph of <i>function operations</i>.<p>- [Research] Formally, Data Commandr relies on the <i>concept-oriented model</i> of data [2]. In particular, DC does not rely on such difficult to use and understand operations like join and group-by. Instead, it uses <i>link columns</i> [4] and <i>accumulate functions</i> [1], respectively.<p>- [Competitors] When I started this project [5] I knew only one similar product which is Power BI and its DAX language. Now there exist such relatively new products like airtable, fieldbook and rowshare.<p>[1] A. Savinov. From Group-By to Accumulation: Data Aggregation Revisited. Proc. IoTBDS 2017, 370-379, 2017. <a href="https://www.researchgate.net/publication/316551218_From_Group-by_to_Accumulation_Data_Aggregation_Revisited" rel="nofollow">https://www.researchgate.net/publication/316551218_From_Grou...</a><p>[2] A. Savinov. Concept-Oriented Model: the Functional View. arXiv:1606.02237 [cs.DB] 2016. <a href="https://arxiv.org/abs/1606.02237" rel="nofollow">https://arxiv.org/abs/1606.02237</a><p>[3] A. Savinov. DataCommandr: Column-Oriented Data Integration, Transformation and Analysis. Proc. IoTBD 2016, 339-347. <a href="https://www.researchgate.net/publication/301764506_DataCommandr_Column-Oriented_Data_Integration_Transformation_and_Analysis" rel="nofollow">https://www.researchgate.net/publication/301764506_DataComma...</a><p>[4] A. Savinov. Joins vs. Links or Relational Join Considered Harmful. Proc. IoTBD 2016, 362-368. <a href="https://www.researchgate.net/publication/301764816_Joins_vs_Links_or_Relational_Join_Considered_Harmful" rel="nofollow">https://www.researchgate.net/publication/301764816_Joins_vs_...</a><p>[5] A. Savinov. ConceptMix: Self-Service Analytical Data Integration Based on the Concept-Oriented Model, Proc. DATA 2014, 78-84 <a href="https://www.researchgate.net/publication/265301356_ConceptMix_Self-Service_Analytical_Data_Integration_based_on_the_Concept-Oriented_Model" rel="nofollow">https://www.researchgate.net/publication/265301356_ConceptMi...</a>