This is a very interesting read, because I took the opposite approach.<p>I started out with the premise that manual logging is very hard to do well. It takes effort on my part (and thus hinges on my discipline), can be inconsistent, and can change depending on what my important metrics are.<p>I automatically collect as much information as possible. I log every time I change a window or tab and what I change it to (extending from native windows on macOS all the way through to virtual windows in gmail.com). I log the same from my iPhone. I throw all of my data in a Postgres data warehouse, and then use a BI tool to normalize it and put some KPIs in a dashboard.<p>This system allows me to ask anything from "what articles did I read last week?" to "what percentage of my time at work did I spend programming?" to "how much did I sleep this week?" to "who do I spend the most time messaging?".<p>It's also an exercise in understanding that metadata is incredibly revealing.
I treat Google Calendar as a time-series database.<p>I have 14 calendars logging approx 500 events per month, and I have a UI for all of this... Google Calendar.<p>I also have API endpoints for all of this... Google Calendar. Which can be triggered by IFTTT for automatic stuff (i.e. did I Instagram something or Tweet?).<p>My email is linked to Gmail, so flights, itineraries, online shopping delivery, restaurant bookings can all flow in.<p>So the only thing I need to do manually are the few things it cannot automatically know, such as what film I just watched or what book I finished reading.<p>I only log public things, that is... data that exists in someone else's database somewhere. I do not log private things, i.e. Signal conversations or anything 100% offline that didn't involve a public database.<p>For me it's a simple thing... this data exists, can I make store it myself in a structure that I can do something interesting with it.<p>I have about a decade of this data.<p>Oh, and Google Calendar is really long in the tooth and is becoming both slow to use, hard to search, and clunky.
I made my side project (<a href="https://socialite.ooo" rel="nofollow">https://socialite.ooo</a>) to do personal logging, but I had a slightly different problem. I would remember small things about people but not their names. I was going to the dog park with my girlfriend a lot and couldn't believe how often we'd remember someone's dog's name, or the walk we met them on, but we wouldn't remember them.<p>My solution was to correlate people to events, and events to locations. This allows you to not only remember what you were doing, but pivot off of any piece of information to better recall an event. Logging into a web app to enter information is a bit of a slog. I think making a mobile app is really crucial. That could even lead to some automation of the logging. For example, "You just called [CONTACT], would you like to create an event for this? ", or at the end of the day, "You texted [CONTACT] today, give a quick summary of what you spoke about". From there you can make the analysis smarter and smarter.<p>It's really strange, all these companies have so much data about us. We don't have access to our own data in a decent format though. My call history is with my telco, my location history is with Google (and probably many others), my conversations are spread out among so many different apps. What happens when I want to query the data about me though?
I remember using AutoHotKey [1] (windows only) to randomly prompt me what I was up to every hour.<p>This was many years ago so I don't recall the details but my intention was to understand where/how I spent all those hours in the day as a student.<p>While there was no formal syntax that I followed that would have helped to analyze the logs programmatically, it did help me get a general understanding of my schedule and identify things I should change.<p>Ref :
[1] <a href="https://autohotkey.com/" rel="nofollow">https://autohotkey.com/</a>
[2] <a href="http://lifehacker.com/5582372/use-a-daily-log-to-keep-yourself-focused-on-productivity" rel="nofollow">http://lifehacker.com/5582372/use-a-daily-log-to-keep-yourse...</a>
In college I had to keep a life log for about a week as part of a psychology class. It was a completely manual process (i.e. pencil and paper; this was quite a while ago), and I was supposed to log in as much detail as possible what I was doing throughout the day. At the end we were supposed to create categories from the log, and add up all the time spent in each activity.<p>The purpose was to find out how much time was <i>really</i> spent on each activity, rather than what we thought we did. It was an eye opening experience.
I've been using RescueTime (<a href="http://www.rescuetime.com" rel="nofollow">http://www.rescuetime.com</a>) for around three years now, which pulls data from all of my workstations as well as my phone, but it lacks the ability to handle soft skill logging like social encounters and dietary habits with the finesse that pcmonk is working towards. Right now, I use it to track things like how much time I spend in IDEs, in StackOverflow, Hacker News, etc.<p>For some previously unexamined reason, I am giddy about tools that enable me to surveil my own productivity and procrastination metrics, but feel slightly uneasy about trying to capture daily intangibles.<p>I'm not sure if I would want to know how much time I've spent with each of my friends; the analytical side of me excises and optimizes, and I worry how that might manifest in negative ways if I start to quantify friendships and phone calls.<p>Anyone identify unexpected truffles and/or landmines in the exploration of your personal logs?
My very first (and maybe not so useful comment): I really love your approach. I was thinking something more automatic (I have some withings stuff) but you convinced me to start small and manual. Really interesting. Thank you for sharing.
How about screen video capture? Rewind at 60x at the end of the day and take notes in whatever structured form you want.<p>It would also help to have an app that reacts to inactivity, that is you stepping away, and ask you what you did.<p>This works well if your day revolves around a desktop or maybe Android, but not so well with iPhone or iPad.
Being able to log and analyze your data is great. It not only provides insight, but also provides motivation ("I'm logging this to improve. Let's not screw it up." etc).<p>I think my three big tools that have helped in this space are: my fitbit (tracking exercising), habitica for recording daily tasks and todos, and private journaling at the end of each day.<p>I've been terrible about routines and such, but all of this has really helped get things back on track.<p>Overall, I think also logging has to feed back into both organization and action, such as new goal setting. Which adds new things to log.
This is great. The structure and simplicity is bomb and the data will be easy to mess around with while still being easy to clean up even if you mess around with verb usage like "scarfed" instead of "ate".
There's some evidence that this kind of behavioral monitoring can be useful in a number of areas, especially weight loss, and other gradual lifestyle shifts. This seems like a particularly smart way to implement that.
Most of the human-subject research done in public health, sociology, and similar domains uses data collected in a similar way. They basically ask people to take notes on specific behaviour or fill short surveys multiple times a day (they call it ecological momentary assessment), then they analyze it in the context of their research.<p>So yes, a tool like this has value in helping with better understanding ourselves, but also can help population research as well.
GUI is not a problem, you can always tweak it or delegate it for little money. The big problem is how to force yourself to do it regularly in long term, at leat in my case.<p>I think that good idea is to integrate it with bot on your IM platform of choice. Messenger bot, in my case, could ask randomly what I'm doing and track it.
I've used Timing.app on my Mac for a loooong time. Love it.<p><a href="http://timingapp.com/" rel="nofollow">http://timingapp.com/</a>
ibGib's current web app used today by my girlfriend and myself for personal logging while our shower is getting remodeled: <a href="https://www.ibgib.com/ibgib/Saturday%20II%5E818E0366E6B009CA0F71083DDD001EC99E5B0E708795B9F7FDBBF43C29225220" rel="nofollow">https://www.ibgib.com/ibgib/Saturday%20II%5E818E0366E6B009CA...</a> _(NB: After a temporary session "login" at the bottom of the page, you will be redirected...also, the links are long because the last part of it is a sha-256 hash that guarantees the integrity of the content of the ibGib data)._<p>I post this because I think personal logging is a perfect fit for ibGib's engine (and it's partly why I've written it), and it may interest you as a back end. It would allow you to basically have a graph database for a back end, while also allowing you to create and evolve your semi-structured data much like evolving an Event-Sourcing event in version control like Git (since each and every datum is versioned and maintains a complete audit history). It also is designed to be Big Data friendly, with the ibgib website's datastore acting with <i>completely</i> Open Data.<p>Also, I wanted to mention that I've actually used it recently in the exact use case given by @TACIXAT where I was walking my dogs in my neighborhood and I wanted to be sure I remembered the neighbors' names down the street when I met them a month or two ago (<a href="https://www.ibgib.com/ibgib/Walk%5EE3EE1755557357849C41C151117424D87EEB806D28216987B0F92F71ED1A2A39" rel="nofollow">https://www.ibgib.com/ibgib/Walk%5EE3EE1755557357849C41C1511...</a> - just doule-click the big yellow circle). I definitely like how the socialite app looks, and it's great in that it helps you remember things <i>in context</i>. This is also what is primary about ibGib, allowing for pictures, text, links, etc., to be nested "inside" other "objects" (so each node acts as both a file with content and a folder with relationships to other nodes).<p>So for your personal logging, you could create a semi-structured "type" with your subject-verb-DO-etc. and then create instances of these. Then you can relate them via any named relation to any other construct (called an ibGib). So it's very much like a graph datastore that also has data hashing for all "children" nodes, which makes it a merkle graph. There's all sorts of neat properties, and like I said, I thought your use case would be a good fit for the engine. (Btw, the structure also lends itself to auto-generating dynamic and collaborative blogging from your data, e.g. <a href="https://www.ibgib.com/ibgib/WaffleGib%20Jr%5E59B6D3828D352091B0251F2A017511C2A681BE300E179398F4D7CB8BC56B650A" rel="nofollow">https://www.ibgib.com/ibgib/WaffleGib%20Jr%5E59B6D3828D35209...</a> or <a href="https://www.ibgib.com/ibgib/Grillin%20burgers%5E9E0151A452C17F5814602D9A119865A60E12E53E8A9819FFB41DE33C0D05CDBB" rel="nofollow">https://www.ibgib.com/ibgib/Grillin%20burgers%5E9E0151A452C1...</a>)