Recutils – Tools and libraries to access plain text databases called Recfiles

At CurrySoftware we use recfiles combined with git for all business-processes (incoming and outgoing invoices, customers, etc). It allows us to automate everything we want with simple bash scripts. But we remain flexible because we can perform non-automated tasks manually.

I have a growing affinity for non-database-databases for personal and low&#x2F;sparse-traffic projects. Lots less hassle.<p>Here&#x27;s one I maintain designed for use with AWS Lambda which uses S3 as a Pythonic data-store: <a href="https:&#x2F;&#x2F;github.com&#x2F;Miserlou&#x2F;NoDB" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;Miserlou&#x2F;NoDB</a>

Apparently you can get output in csv (natively) and json (python one liner) without too much assle (<a href="http:&#x2F;&#x2F;swick.2flub.org&#x2F;recutils_JSON_output.html" rel="nofollow">http:&#x2F;&#x2F;swick.2flub.org&#x2F;recutils_JSON_output.html</a>) which suddently makes it even more interesting.

The link to the newer presentation video on the page is broken but this is probably the same one: <a href="https:&#x2F;&#x2F;fscons.org&#x2F;videos&#x2F;2011&#x2F;gnu-recutils-changed-title-and-subject.webm" rel="nofollow">https:&#x2F;&#x2F;fscons.org&#x2F;videos&#x2F;2011&#x2F;gnu-recutils-changed-title-an...</a>

I really like the idea of plain text data files so am really interested in this.<p>YAML serves this purpose too but I&#x27;m not a huge fan of indenting so recfiles look great. Anyone compared and contrasted?<p>Also are there other resources on this? Would be nice to have Java&#x2F;C++&#x2F;Python libraries. (As well as convert to parquet, arrow etc )

Fro docs:<p>YAML 1 is an example of a hierarchical data storage format which is much more readable than XML. The problem with YAML is that it was designed as a “data serialization language” and thus to map the data constructs usually found in programming languages. That makes it too complex for the simple task of storing plain lists of items.<p>I dont see how this is true. Provided sample with books is almost identical in yaml.<p>The main benefit over yaml looks like more control of individial fields but again, yaml based db app could do that too.

I noticed this is GPLv3, which means if you use this library, all your application will have to be open source, however IANAL.

This reminds me a bit of using CGI.pm&#x27;s &quot;Save&quot; function. I built a pretty decent invoicing app using that and the searches for data in documents saved in that format are pretty fast.<p>I won&#x27;t pretend to know the ins-and-outs of that but was told on a Perl mail list that the server created a &quot;B-Tree&quot; index when an initial search was made and used that afterwards.

I do something similar with toml files for simple stuff. Python for piping around, but maybe this is more convenient on the commandline.

Did this come out of Amazon ? I remember a similar set of tools for passing along &quot;recs&quot; through pipes.

I guess these are quite slow (because no indexing) once you have a serious number of records? That in itself isn&#x27;t a problem as long as you understand the scope of the project. I wonder why they didn&#x27;t use (a well-defined subset of) CSV as the format however.

If you want something lighter to use text files as tables you could try TextQL: <a href="https:&#x2F;&#x2F;github.com&#x2F;dinedal&#x2F;textql" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;dinedal&#x2F;textql</a>.

Erm... am I the only one that&#x27;s a little thrown off by the &quot;mascot&quot; in this project?