I did this after seeing this before on HN. There were a few processes that were manual that benefitted from the technique in the article.<p>Learn from my folly: I even called them “do nothing scripts,” referencing this article. However; I was judged by peers for not writing the full automation versions as they didn’t appreciate the idea of gradual automation (programmer hubris?). Saying “do nothing scripts” in meetings did catch the awkward attention of leadership.<p>As a description, “do nothing” communicates a lot. As a brand, “do nothing” can use some improvements.<p>My short prescription of turning “do nothings” into “do some things” into “do all the things” didn’t help. We had some new people join the team and they had fun turning the do nothing scripts into a document. * Sigh *<p>I still build these type of process description scripts still. I usually don’t advertise them to peers until they do some of the things nowadays.
My experience in corporate IT convinced me that even if an entire release could be automated down to a single press of a giant red button (and even if undoing the release could be another, single button), then culturally our organization would still figure out a way to turn the pressing of that button into a six hour ordeal requiring the participation of twelve people.
The idea here can be generalized to a primitive programming construct: The magic function "wish", which can do anything you want, just give it a string description.<p>For example:<p><pre><code> wish('copy these files to this host', files, host)
</code></pre>
You can augment this further by allowing the program to specify a return type from the wish, and magically the wish will produce the type you want. So for example:<p><pre><code> data = wish(Path, 'the path of the data files I need')
</code></pre>
Of course, the wish is actually implemented by sending a request to some human user. In my "wish" library for Python <a href="http://rsyscall.org/wish/" rel="nofollow">http://rsyscall.org/wish/</a> you can have a stack of handlers for wishes, just like exception handlers. If later you want to automate some wish, you can specify a handler which intercepts certain wishes and forwards the other ones on (by wishing again, just like an exception handler can re-raise).
Side note: - I don't understand why classes are needed here.
instead of<p><pre><code> procedure = [
CreateSSHKeypairStep(),
GitCommitStep(),
WaitForBuildStep(),
RetrieveUserEmailStep(),
SendPrivateKeyStep(),
]
for step in procedure:
step.run(context)
</code></pre>
Why not make them functions instead of classes. The URLs can be locals instead of class variables and the context argument becomes an argument to the function rather than a run method.<p><pre><code> procedure = [
createSSHKeypairStep,
gitCommitStep,
waitForBuildStep,
retrieveUserEmailStep,
sendPrivateKeyStep,
]
for step in procedure:
step(context)
</code></pre>
The OO is just gratuitous. Personally I probably wouldn't even bother with the for loop/context and would just pass the data that's actually needed by each function into it manually.<p><pre><code> createSSHKeypairStep(username)
gitCommitStep(username)
waitForBuildStep()
email = retrieveUserEmailStep(username)
sendPrivateKeyStep(email)
</code></pre>
I get that it feels more modular - "oh all I need to do to add a step is add it to the list!" But like, all you need to do to add a step is add a line of code. There's no difference.<p>Much easier to see how the data is actually flowing that way. I found the fact that the context was mutated in a function it was passed into kind of gross. It's just globals with extra steps. Besides, this is less LOC anyway.
> Create an SSH key pair for the user.<p>> Send the user their private key via 1Password.<p>Why are you generating *private keys* for users, then sharing them? Not that this impacts the automation bits but IMHO users should known how to generate and maintain a key pair, and send you the public key.
(2019)<p>Discussed at the time: <a href="https://news.ycombinator.com/item?id=20495739" rel="nofollow">https://news.ycombinator.com/item?id=20495739</a>
This was very enlightening for me.<p>But I wonder if it wouldn't be better if it were something like a ncurses list with checkboxable list items which one would progress trough.<p>Take a checklist for a pilot for example. Wouldn't it be not as good if the next item on the list only appeared after an item has been checked? It could be beneficial if you could peek a couple of items down the line in order to group your actions or get a better overview of the overall task.<p>"Do I put the water for the coffee in the microwave now or do I first finish this quick item and let the next long-running item start before I go and make my coffee?"
This approach doesn't work well in the long term. I've worked as a successor in one of the author's previous company. All the "do-nothing scripts" diverges from the reality as time goes by. The scripts became a false prophet, not a single source of truth. When I came in, the fully automated parts are doing fairly well. On the other hands, most the one-off scripts doesn't work.<p>Falsy docs are worse than no documentation. We've tried looking at the scripts, but small divergents in the scripts destroyed our confidence in the scripts. Instead we figure how to do task by looking at the source code and actual deployments.<p>The lesson we learned from this is we need to treat scripts as a real software instead. We make scripts idempotent and run them periodically to see if anything breaks. For infrastructures we make them immutable and embraced the cattle mindset(vs pet mindset).
This is a really nice solution. For the 99% who don't work for a place with a massive DevOps team, this approach is a nice middle ground that actually works. If management doesn't like it they don't have to know. These scripts are for getting through the day with 10% more energy to spare at the end.
Cool. A nice medium on the way between "seat-of-your-pants" manual and completely inscrutable automation!<p>People like to brag that they never do something manually more than once, but in reality it takes time and practice to be able to fully automate any non-trivial process. You usually can't do it in one go. I really like that this article embraces that.<p>I especially like how it's broken down such that any particular function can be swapped out with full automation once the author gets enough confidence to be able to do that.
Hey I independently came to this as well!<p>We were performing some migrations to our servers and we weren't sure we had nailed the process down. Automating the procedure has a problem in that I can't just "reset" the server and start again. I could try to make all my operations idempotent or using a babushka style "Check - Set - Check" methodology, but that would greatly expand the work for a one off procedure that once it's done it's done.<p>I wasn't even sure of the generality of the steps, and they were complicated enough that copy and pasting them with all their parameters was tedious. So I created a structure just like this!<p>I called it a living procedure. An excerpt:<p><pre><code> #!/bin/bash
source ./procedure-lib.sh
question "Which client is this?" old_client
question "What worker is $old_client on?" old_worker
question "What is the targets name going to be? (it's fine if it's still $old_client)" new_client
question "What worker is $new_client going to be on?" new_worker
question "What branch is $new_client going to be on?" new_branch
question "What should we name the migration? (typically something like migraiton-$(date --iso))" migration_name
#================ Initialize the client on the new worker ================
step "Login to $new_worker and Initialize $new_client in legacy mode using run.sh init_legacy, set their branch to $new_branch"\
'eg.
ssh '"$new_worker"'.lightship.works "sudo -i bash -c \"
</code></pre>
These question functions also remembered the answer you put in last time (it was based off what the text of the question was, so questions that depend on the answer of previous questions did not presume anything) so that if you just hit "enter" then it would load the last one. This was vital because when a procedure ended up incorrect or needed to be amended, you could ctrl+c the script, fix it, then spam enter until you reached the point you were last time.<p>I've been thinking about creating a better structure to these scripts that allow you to gradually transform these living procedures into babushka style "Check Set Check" automations.
Usually I write instructions in markdown files and test them by redo all the steps.
This saved me already many times because I will forget how to renew this damn certificate two years later, or setup the logging engine after 18 months ago.
No need for a script.
So naturally, my knee-jerk reaction is to create some kind of checklist language and interpreter. Which is almost wrong, of course.<p>I say "almost", because it occurs to me that many steps are shared individually across many different workflows. If you see a single task as an automation target, then you end up writing simple scripts that perform very focused operations -- a strong advantage over a large complicated script that does many different things, some that are repeated for other complicated scripts.<p>In this setup, you end up with checklists that remain forever, with automation replacing some or all of the print commands.<p>Of course, my spidey sense tells me that I've just pushed the problem down a level, so ¯\_(ツ)_/¯.
I really like this idea. It goes hand in hand with an older post of mine: "no script is too simple" [1].<p>A script is a recipe that your whole team can follow. It's easier to update a recipe than to propagate changes across the whole team.<p>It's also similar to a flight check, which is a fancy name for a reusable to-do list.<p>[1] <a href="https://nicolasbouliane.com/blog/no-script-is-too-simple" rel="nofollow">https://nicolasbouliane.com/blog/no-script-is-too-simple</a>
Braintree created a tool for doing something similar: <a href="https://medium.com/braintree-product-technology/https-medium-com-braintree-product-technology-runbook-be6f072cfc0d" rel="nofollow">https://medium.com/braintree-product-technology/https-medium...</a>
This is basically why I ended up writing Runnable Plans, over at <a href="https://github.com/vatine/runnable-plans/" rel="nofollow">https://github.com/vatine/runnable-plans/</a>, the main differences are "you define the plan in YAML" (yeah, horrible, but better than hand-chasing a parser, that MAY come at a later date) instead of "in the script", "the plan has no inherent order" (to allow for future parallel execution), "saves success/failure to allow for later restart", and "can generate a GraphViz graph of the plan dependency ordering".<p>But, whatever works, works. Start somewhere, get it into a script, plan, whatever. Then, it is easier to identify steps that can be turned to entirely machine-operated.
Nice. A class with a single method can be nicely modelled as a closure.<p>Returning higher order functions and using format’s kwargs is an even more succinct version of this.<p><pre><code> def explode(**k)
def run():
return TheBomb(
’{user} is!’.format(**k))
return run
run(user=‘gorgoiler’)</code></pre>
Another benefit of this interactive to-do list: you'll find out pretty quickly if the steps <i>really</i> are that simple.<p>The test is you.<p>Many "slogs" require too much human decision to be automatable, but too little to be interesting. tedious. tiresome. irksome. In a way, also trite, bromidic/
So, this is a wizard, basically.
<a href="https://microsoft.fandom.com/wiki/Wizard" rel="nofollow">https://microsoft.fandom.com/wiki/Wizard</a>
I like Observable <a href="https://observablehq.com" rel="nofollow">https://observablehq.com</a> notebooks for this kind of thing - quickly building automated scripts that simply accept user input and use it to dynamically construct a copy-and-paste output.<p>You can run and edit them entirely in the browser, they can do anything you can do with JavaScript and they're easy to share with other people and create forks.
Amazingly obvious concept, love it. I feel i could use this in my personal life.<p>Just wondering, i know it isn't the point, but does this exist as a service?
For example is there a web service where my sales dept could add all the steps and it just goes through them one by one, but can include a start/completed webhooks.
I also independently stumbled on this idea. My rational was that it turns the process into more of a checklist, without all of the hassle of handling edge cases in my code. Because usually whenever I try to fully automate I re-learn the wisdom of that xkcd cartoon about "automation expectations versus reality".<p>Another wise thing about it is that it's a relatively low-friction way to start documenting institutional knowledge. Your engineers might hate writing docs, but maybe they won't mind writing a bash script like this. Which is essentially documentation in disguise. And since it's (presumably) living with the rest of your version control system you increase the chances that your engineers will explain changes to the process over time (via commit logs).
In my work, we have develop on Linux, MacOS and Windows. Another advantage to this approach is that it allows you not just to incrementally automate steps in the process, but _steps on platform X_. It eliminates yet another barrier to starting the process of automation.
I do something similar but take it a step further and statefully track the progress of the workflow. The code generates a file that is like a TODO list, and a separate command runs a single step before marking it complete. Great for longer running workflows (days).
Braintree's Runbook seems to regard exactly this case of "Do-nothing scripting" <a href="https://github.com/braintree/runbook" rel="nofollow">https://github.com/braintree/runbook</a>
I have a hobby of printing "three-sided cards" that have a front side (maybe a photograph, art reproduction, or anime character) and a back side that describes the image and has a QR code (and maybe an NFC tag) that points to a "web side".<p>This process has a number of steps, some of which are physical (handling paper), some of which are digital and scriptable, and some of which are digital but not scriptable (serious image modifications with photoshop.)<p>I use a process very similar to what's described in the article to manage the whole thing.
Why don’t you just write this down into a checklist? Why codify it?<p>You need the interpreter to run it.<p>It’s less accessible to non technical people.<p>Actually running the script is an extra step that’s not needed.<p>This feels like an engineer overcomplicating something.
Could crude shell scripts be used for this purpose? For example, in zsh [0]:<p><pre><code> $ echo "Do the first step, then press any key."; read -s -k
</code></pre>
EDIT: Apparently so: [1]<p>[0] Modeled on <a href="https://stackoverflow.com/questions/5215343/how-can-i-pause-in-zsh/14867987" rel="nofollow">https://stackoverflow.com/questions/5215343/how-can-i-pause-...</a><p>[1] <a href="https://news.ycombinator.com/item?id=20495739" rel="nofollow">https://news.ycombinator.com/item?id=20495739</a>
The author has written a Golang version of the same as well [1].<p>1: <a href="https://github.com/danslimmon/donothing" rel="nofollow">https://github.com/danslimmon/donothing</a>
I like the article. This is a great idea. I'm going to implement it over the next week. I have high hopes.<p>That being said, I have absolutely no clue what the advantage is to the python programming style exemplified in the article. Can someone explain why on earth I would ever want to convert:<p><pre><code> def do_something(context):
pass
...
do_something(c)
</code></pre>
into<p><pre><code> class DoSomething(object):
def run(self, context):
pass
...
DoSomething().run(c)
</code></pre>
What is the point here?
The lesson for me here is to actually have scripts written where the business logic is clear to see and change.<p>This is a workflow in a python script but could also be a tool just by adding argument parsing. But also tools can be called from software pipelines and have tools to execute this for us (automation).<p>Look at <a href="https://aosabook.org/en/500L/a-continuous-integration-system.html" rel="nofollow">https://aosabook.org/en/500L/a-continuous-integration-system...</a>
This is a lot like the checklists a lot of industries use, just automated on your shell instead of being on paper.<p>I'm thinking of a pilot's checklist, or an operating theatre checklist.
Combining this technique with an automation website like Jenkins and a "no code" programing language could be an interesting product. I imagine having a UI where prompts can be specified and predefined components like send X an email can be dropped into the workflow. Acknowledgments that steps and complete can be tracked, and long running workflows could be shared between users.
I like to do this with a checklist. In Confluence you can clone the checklist page for the onboarding/release, etc and check off the cloned page.<p>Confluence is wiki-style (edit first, ask questions later), but I could definitely see git-style (pull request first) as an improvement for some situations.
Makefiles can be great for this sort of thing. Over time you build up a really nice library of shell commands and it's trivial to organize them into a DAG of dependent steps. Lack of makefiles is a strong reason to avoid windows dev machines like the plague...(IMHO)
One could pretty easily create a tool that takes Github Flavored Markdown and interactively displays the text and make checkbox items appear one at a time in a manner similar to this. Would be much more useful and easier to configure.
Weirdly I have something like this called SOPys (usual python joke). It is pretty much that for pretty much the same reason only this is waaay better explained.<p>So - got my external validation . Will expand that repo when i get my own time back
As others have stated, "do-nothing" may have some baggage.<p>Consider something like "incrementally inclusive" or "eventually complete" or similar.
Why a class for each step instead of a function? They don't have individual contexts and don't even have any members. They don't even need to persist, it's literally a command line script. Object oriented style for no reason is my pet peeve.
This is the way. I wish this were taught in computer science class, development bootcamps, operations team onboarding, anywhere there is a procedure that is even slightly complicated to automate. It is the absolute best solution there is.<p>* Documentation of the entire procedure is contained in one place. No need to go sifting through 20 different sources of documentation. This lowers the human emotional barrier to "just get it done", as people will always avoid things they aren't comfortable/familiar with, or don't have all the steps to. This central point of documentation also enables rapidly improving the process by letting people see all the steps in one place, which makes it easier to fix/collapse/remove steps.<p>* Automation in small pieces over time avoids the trap of "a project" where one or more engineers have to be dedicated to this one task for a long period of time. Most things shouldn't be automated unless there is demonstrably greater value in the cost of automating them than the cost of not doing so. Automating only the most valuable/costly pieces first gives immediate gains without sinking too much into the entire thing.<p>* One unified "method" to encapsulate any kind of process means your organization can ramp up on processes easier, reducing overall organizational cost.<p>* In the absence of any other similar process, you are <i>guaranteed</i> to save time and money.<p>I would say that the only potential downside is if someone decides to "engineer" this method, making it more and more and more complicated, until it loses its value. KISS is a requirement for it to be sustainable.
Terraform. My god, just use terraform.<p>Stop scripting random stuff to create multiple untracked resources in different places by hand.<p>Slap this[1] on top of this[2] and add whatever other services you want, and boom. Done.<p>1. <a href="https://stackoverflow.com/questions/49743220/how-do-i-create-an-ssh-key-in-terraform" rel="nofollow">https://stackoverflow.com/questions/49743220/how-do-i-create...</a><p>2. <a href="https://registry.terraform.io/providers/1Password/onepassword/latest" rel="nofollow">https://registry.terraform.io/providers/1Password/onepasswor...</a>