This entire graph would have to represent every entity in the building, including the control nodes. It's an IoT graph, where a node may represent the light switch, connected to a particular desk(also a node). This graph should also show directions from one entity to another.
For now, I have a photoshop pipeline which generates a simplified graph from a color coded image, using NetworkX. This is a temporary replacement for when the actual BIM comes along. But the graph remains.
Are there any libraries out there that would help me or should I just roll my own system?
I'm not sure I would like to add Neo4J to the stack. Storing everything to Postgres seems to work fine. It's the in memory representation that I have problems with.
A thing to think about: often there are many different graphs you can create to represent the same thing. Each graph representation can focus on or hide different aspects of the real system.<p>Why are you trying to model the building as a graph? What are the use cases? What operations do you want this data structure to be able to perform efficiently?<p>It might turn out that a single graph (or any graph) is not the most effective way of modelling an approximation of the real system.<p>How many nodes and edges will your graphs typically have? python & networkx work okay for bashing out prototype code and may be good enough for MVP or even a large number of releases if your data size is small and the operations you perform on the graph are linear in the graph size (e.g. connectivity checks, traversals)<p>I've also seen C/C++ codebases get pretty far by rolling their own domain specific graph data structures-- eg define your own node and edge struct types, give each node and edge pointers to the edges/nodes they connect to, hack domain specific fields as necessary onto the structs. Then just implement each graph algorithm as you need it. This may end up in an unmaintainable mess after a few years, but I've seen this work well enough so that the product based on this is worth enough money that there's enough cash to hire software engineers to come clean things up!<p>Another thing to think about: are your graphs dynamic or static? If they are large and static, there's lots in common between graphs and sparse matrices. You can encode your graphs in memory in CSR or CSC like sparse matrix formats-- no objects, just giant arrays full of indices. This isn't a good idea if your want to dynamically add or remove nodes and edges, but it is memory efficient.
I working with big graphs. Over 1 bln entities & 40 bln edges.<p>Best open source graph database is ArrangoDB they have master to master cluster.
Fastest and best technology for graphs have commercial TigerDB, but you must pay >300k annually.<p>Networkx is great but loading full model - but for what you doing should be enough. :)
Have you looked at just using IFC? It’s the canonical “building as a graph” BIM format. Whether the in memory IFC allows querying I suppose is up to the implementation, I have only used xbim
<a href="https://github.com/xBimTeam" rel="nofollow">https://github.com/xBimTeam</a>
Question with graphs is whether you want it to be a representation of a dataset, or persist the graph as the data itself.<p>For persistence, I use Neo4j to represent hundreds of dynamic graph ontologies, and I use the hosted version on graphenedb, which has worked just fine for my purposes.<p>For some views, I just use NetworkX to generate interactive d3.js pages from data I have queried from the graph, or python/flask with py2neo to generate json for d3 visualizations. Some others use cytoscape for visualization, but I find that a bit dramatic for most purposes.<p>Depending on how you would like to represent it, I can also recomment Webprotege and WebVOWL, since the graph you are creating is also in effect an ontology.
Using Neo4j community edition would certainly make your life easier in terms of the queries you might want to express, but for this amount of data you can get away with postgres, as you observe.<p>If you want a simple in memory graph modelling library then check out Apache Tinkerpop. Its great.
I would recommend storing your data using Tree Notation in a Tree Language. That way your data is as simple as possible. You could then compile it to whatever format is needed by your downstream tools. Here's a quick start:<p><a href="http://treenotation.org/sandbox/build/#grammar%0A%20nodeType%20graph%0A%20%20root%0A%20%20description%20A%20new%20language%20for%20graphs%20in%20response%20to%20an%20HN%20comment.%0A%20%20catchAllNodeType%20catchAllError%0A%20%20inScope%20node%0A%20cellType%20keyword%0A%20%20highlightScope%20keyword%0A%20cellType%20intCell%0A%20%20highlightScope%20constant.numeric.integer%0A%20cellType%20deviceTypeCell%0A%20%20enum%20Router%20RaspberryPi%0A%20cellType%20nodeIdCell%0A%20%20highlightScope%20meta.annotation.identifier%0A%20nodeType%20node%0A%20%20cells%20nodeIdCell%0A%20%20firstCellType%20keyword%0A%20%20inScope%20deviceTypeNode%20floor%20connectsTo%0A%20nodeType%20floor%0A%20%20cells%20intCell%0A%20%20firstCellType%20keyword%0A%20nodeType%20connectsTo%0A%20%20cells%20nodeIdCell%0A%20%20firstCellType%20keyword%0A%20nodeType%20deviceTypeNode%0A%20%20match%20type%0A%20%20cells%20deviceTypeCell%0A%20%20firstCellType%20keyword%0A%20nodeType%20catchAllError%0A%20%20baseNodeType%20errorNode%0Asample%0A%20node%20thermometer%0A%20%20floor%201%0A%20%20type%20RaspberryPi%0A%20%20connectsTo%20router1%0A%20node%20router1%0A%20%20type%20Router%0A%20%20floor%201" rel="nofollow">http://treenotation.org/sandbox/build/#grammar%0A%20nodeType...</a>
This reminded me of drafting electric diagrams in plans and elevations. And of course the notions of a 'plan' and and 'elevation' are also front and center.<p>Why were not graphs embraced by other domains? Plan and elevations by definition are constrained view points. (Hint: for the same reason the highly available and highly consistent flavors of graph databases require paid licenses :)<p>Graphs make for difficult decomposable 'unit' assemblies. Plans and elevations are 'standard units'. (Again: it may help to think of as plans and elevations as tables and reverse indexes, respectively.)<p>Note requirements such as e.g. "show directions from one entity to another" are also present for documenting the electrical systems, or HVAC, in a building.<p>A modular system for representing graphs of arbitrary scale is the minimal and trivial 'one node per modular unit'. The alternative is throwing huge amounts of processing power to allow arbitrary views into a graph at any scale.<p>IoT, as an 'integrated component' of building systems, will find a very happy place on plans and elevations.
Postgres works well. Neo4j. Persistence is pretty well solved.<p>So is the in-memory representation! Have you thought about using lazy structures? The graph can be conceptually infinite in size but your program only loads the pieces being used as they are needed and offloads old ones that are not.
May be useful:<p><a href="https://github.com/eBay/beam" rel="nofollow">https://github.com/eBay/beam</a><p><a href="https://github.com/gchq/Gaffer" rel="nofollow">https://github.com/gchq/Gaffer</a>
I do not know if it is right for your use case, but you could have a look at <a href="https://github.com/Tulip-Dev/tulip" rel="nofollow">https://github.com/Tulip-Dev/tulip</a> for visualization.
As you talked about BIM, I guess you are already aware of IFC. The real question is that your use case isn't very clear to me. What are you trying to achieve? Visualize a graph?
As usual, TeX can help. See Tikz and PGF. If you want to represent a building you could need to draw electric circuits. See also circuitikz.<p><a href="http://www.texample.net/tikz/examples/pressurized-water-reactor/" rel="nofollow">http://www.texample.net/tikz/examples/pressurized-water-reac...</a>
For our marketing analytics product, we are using AWS Neptune and very happy with that. First we started out with Azure cosmos graph, but due to incomplete tinkerpop support many queries were not supported. You can give AWS Neptune a try
For an in-memory graph representation, you can have a look at Redis Graph <a href="https://oss.redislabs.com/redisgraph/" rel="nofollow">https://oss.redislabs.com/redisgraph/</a>
I'm currently tinkering with graph using <a href="https://dgraph.io/" rel="nofollow">https://dgraph.io/</a> -- soon it will support GraphQL.