> git bisect is the "real" way to do this, but it's not something I've ever needed<p>git bisect is great and worth trying; it does what you're doing in your bash loop, plus faster and with more capabilities such as logging, visualizing, skipping, etc.<p>The syntax is: $ git bisect run <command> [arguments]<p><a href="https://git-scm.com/docs/git-bisect" rel="nofollow">https://git-scm.com/docs/git-bisect</a>
I've always had trouble getting `for` loops to work predictably, so my common loop pattern is this:<p><pre><code> grep -l -r pattern /path/to/files | while read x; do echo $x; done
</code></pre>
or the like.<p>This uses bash read to split the input line into words, then each word can be accessed in the loop with variable `$x`. Pipe friendly and doesn't use a subshell so no unexpected scoping issues. It also doesn't require futzing around with arrays or the like.<p>One place I <i>do</i> use bash for loops is when iterating over args, e.g. if you create a bash function:<p><pre><code> function my_func() {
for arg; do
echo $arg
done
}
</code></pre>
This'll take a list of arguments and echo each on a separate line. Useful if you need a function that does some operation against a list of files, for example.<p>Also, bash expansions (<a href="https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html" rel="nofollow">https://www.gnu.org/software/bash/manual/html_node/Shell-Par...</a>) can save you a ton of time for various common operations on variables.
I have something like this in my bashrc:<p><pre><code> preexec ()
{
# shellcheck disable=2034
_CMD_START="$(date +%s)"
}
trap 'preexec; trap - DEBUG' DEBUG
PROMPT_COMMAND="_CMD_STOP=\$(date +%s)
let _CMD_ELAPSED=_CMD_STOP-_CMD_START
if [ \$_CMD_ELAPSED -gt 5 ]; then
_TIME_STR=\" (\${_CMD_ELAPSED}s)\"
else
_TIME_STR=''
fi; "
PS1="\n\u@\h \w\$_TIME_STR\n\\$ "
PROMPT_COMMAND+="trap 'preexec; trap - DEBUG' DEBUG"
</code></pre>
Whenever a command takes more than 5 s it tells me exactly how long at the next prompt.<p>I didn't know about `$SECONDS` so I'm going to change it to use that.
Installing GNU stuff with the 'g' prefix (gsed instead of sed) means having to remember to include the 'g' when you're on a Mac and leave it off when you're on Linux, or use aliases, or some other confusing and inconvenient thing, and then if you're writing a script meant for multi-platform use, it still won't work. I find it's a much better idea to install the entire GNU suite without the 'g' prefix and use PATH to control which is used. I use MacPorts to do this (/opt/local/libexec/gnubin), and even Homebrew finally supports this, although it does it in a stupid way that requires adding a PATH element for each individual GNU utility (e.g. /usr/local/opt/gnu-sed/libexec/gnubin).
<p><pre><code> > 1. Find and replace a pattern in a codebase with capture groups
> git grep -l pattern | xargs gsed -ri 's|pat(tern)|\1s are birds|g'
</code></pre>
Or, in IDEA, Ctrl-Shift-r, put "pat(tern)" in the first box and "$1s are birds" in the second box, Alt-a, boom. Infinitely easier to remember, and no chance of having to deal with any double escaping.
> git bisect is the "real" way to do this, but it's not something I've ever needed<p>uh, yeah, you did need it, that's why you came up with "2. Track down a commit when a command started failing". Seriously though, git bisect is really useful to track down that bug in O(log n) rather than O(n).
This thread seems like a good place to ask this:<p>When you're running a script, what is the expected behaviour if you just run it with no arguments? I think it shouldn't make any changes to your system, and it should print out a help message with common options. Is there anything else you expect a script to do?<p>Do you prefer a script that has a set of default assumptions about how it's going to work? If you need to modify that, you pass in parameters.<p>Do you expect that a script will lay out the changes it's about to make, then ask for confirmation? Or should it just get out of your way and do what it was written to do?<p>I'm asking all these fairly basic questions because I'm trying to put together a list of things everyone expects from a script. Not exactly patterns per se, more conventions or standard behaviours.
I am confused how this works. I would assume `SECONDS` would just be a shell variable and it was first assigned `0` and then it should stay same, why did it keep counting the seconds?<p><pre><code> > SECONDS
bash: SECONDS: command not found
> SECONDS=0; sleep 5; echo $SECONDS;
5
> echo "Your command completed after $SECONDS seconds";
Your command completed after 41 seconds
> echo "Your command completed after $SECONDS seconds";
Your command completed after 51 seconds
> echo "Your command completed after $SECONDS seconds";
Your command completed after 53 seconds</code></pre>
I really love this style of blog post. Short, practical, no backstory, and not trying to claim one correct way to do anything. Just an unopinionated share that was useful to the author.<p>It seems like a throwback to a previous time, but honestly can't remember when that was. Maybe back to a time when I hoped this was what blogging could be.
We need a simpler regex format. One that allows easy searching and replacing in source code. Of course, some IDE's already to that pretty well, but I'd like to be able to do it from the command line with a stand alone tool I can easily use in scripts.<p>The simplest thing I know that is able to do that is coccinelle, but even coccinelle is not handy enough.
My git-gsr (global search replace) command:<p><a href="https://gist.github.com/jaysoffian/0eda35a6a41f500ba5c458f02a29865d" rel="nofollow">https://gist.github.com/jaysoffian/0eda35a6a41f500ba5c458f02...</a><p>Uses perl instead of gsed, defaults to fixed strings but supports perl regexes, properly handles filenames with whitespace.
To conserve host resources RFC 2616 recommends making multiple HTTP requests over a single TCP connection ("HTTP/1.1 pipelining").<p>The cURL project said it never properly suported HTTP/1.1 pipelining and in 2019 it said it was removed once and for all.<p><a href="https://daniel.haxx.se/blog/2019/04/06/curl-says-bye-bye-to-pipelining/" rel="nofollow">https://daniel.haxx.se/blog/2019/04/06/curl-says-bye-bye-to-...</a><p>Anyway, curl is not needed. One can write a small program in their language of choice to generate HTTP/1.1, but even a simple shell script will work. Even more, we get easy control over SNI which curl binary does have have.<p>There are different and more concise ways, but below is an example, using the IFS technique.<p>This also shows the use of sed's "P" and "D" commands (credit: Eric Pement's sed one-liners).<p>Assumes valid, non-malicious URLs, all with same host.<p>Usage: 1.sh < URLs.txt<p><pre><code> #!/bin/sh
(IFS=/;while read w x y z;do
case $w in http:|https:);;*)exit;esac;
case $x in "");;*)exit;esac;
echo $y > .host
printf '%s\r\n' "GET /$z HTTP/1.1";
printf '%s\r\n' "Host: $y";
# add more headers here if desired;
printf 'Connection: keep-alive\r\n\r\n';done|sed 'N;$!P;$!D;$d';
printf 'Connection: close\r\n\r\n';
) >.http
read x < .host;
# SNI;
#openssl s_client -connect $x:443 -ign_eof -servername $x < .http;
# no SNI;
openssl s_client -connect $x:443 -ign_eof -noservername < .http;
exec rm .host .http;</code></pre>
I want to like this, but the for loop is unnecessarily messy, and not correct.<p><pre><code> for route in foo bar baz do
curl localhost:8080/$route
done
</code></pre>
That's just begging go wonky. Should be<p><pre><code> stuff="foo bar baz"
for route in $stuff; do
echo curl localhost:8080/$route
done
</code></pre>
Some might say that it's not absolutely necessary to abstract the array into a variable and that's true, but it sure does make edits a lot easier. And, the original is missing a semicolon after the 'do'.<p>I think it's one reason I dislike lists like this- a newb might look at these and stuff them into their toolkit without really knowing why they don't work. It slows down learning. Plus, faulty tooling can be unnecessarily destructive.
If I find myself running a set of commands in parallel, I'd keep a cheap Makefile around: individual commands I want to run in parallel will be written as phony targets:<p><pre><code> all: cmd1 cmd2
cmd1:
sleep 5
cmd2:
sleep 10
</code></pre>
And then<p><pre><code> make -j[n]</code></pre>
> Use for to iterate over simple lists<p>I definitely use this all the time. Also, generating the list of things over which to iterate using the output of a command:<p><pre><code> for thing in $(cat file_with_one_thing_per_line) ; do ...</code></pre>
This post and comments section means I no longer wonder why 99% of shell scripts I come across look inept. I'm sorry guys but seriously please actually learn bash (and ideally not from this blog post). There's so many things wrong in the post and the comments that it's difficult to enumerate.<p>To start with, if you ever feel the need to write a O(n) for loop for finding which commit broke your build, you DID need git-bisect.<p>Definitely DONT'T wait for PIDs like that and if you do want to write code like that, maybe actually use the arrays bash provides?<p>If your complaint is that for a in b c d; do ...; done is unclear, then maybe also use lists there, because what's definitely LESS clear is putting things in a variable and relying on splitting.<p>And most importantly, DO quote things (except when it's unnecessary).