A Practical Guide to Watchdogs for Embedded Systems

One neat thing I've seen that doesn't get called out enough, is a high priority timer that has a slightly smaller period than your watchdog. When you let the watchdog, you pet this timer too. Then in the timer ISR, write out the trap frame to brain dead non volatile memory (we had battery backed SRAM, and then MRAM on newer boards). Then when the board reboots, and checks in, you can pull down what it was doing when the watchdog triggered.

I was surprised at the poor state of watchdogs in PC-class Linux systems. I needed one recently and was bummed at the state of the old watchdog daemon &#x2F; softdog kernel module. It works, but it is not nearly as easy to get going (on Ubuntu) as I expected. systemd also has its own watchdog and I can&#x27;t figure it out.<p>Anyway turns out I really needed a full PC hardware watchdog. I ended up buying some $8 anonymous piece of Chinese hardware that&#x27;s USB powered. It hits the motherboard reset switch if the motherboard hard drive activity light hasn&#x27;t flashed in awhile. Dumb thing, but it seems to work.

One scenario missed in the list of causes is a corrupted runtime image.<p>An advanced topic is enabling support for the watchdog in your bootloader and having a defined recovery path when the system fails to load or, worse, the application falls into a boot loop.<p>If you have the space, you can fall back to a recovery image or duplicate of the application. If you don’t have the space, falling into a DFU mode is a good plan.

Watchdogs are one of the more frustrating types of issues to debug. Chris&#x27;s overview of how to implement them properly, and investigate resets is an amazing resource I wish I had earlier in my career.

The watchdog is a nice feature to have a borked system reboot, lifesaver in the field if feces hits the fan.<p>What&#x27;s less fun is if there is too little protection against electrostatic fields&#x2F;EMI on the JTAG clock pin. On the small cortex m-class devices we work with, some of them can&#x27;t shut off the JTAG part of the chip, meaning that when operating, if there are enough (I think 8) logic flips on the TCK pin in _any_ amount of time, the JTAG part wakes up, sets the HALT ON BOOT flag. Next time the device reboots (due to firmware update, or watchdog, ...), it will stop and stay in JTAG debug mode. Not nice. You need to manually power cycle the thing.<p>We detect this by periodically checking the JTAG power domain, and if it is on, tell the server this so that we avoid rebooting it (eg automatically after firmware update). This way we&#x27;ve found poor hw implementations and tough EMI environments by proxy of JTAG power domain :D.

A neat enough article, but surprised it didn&#x27;t talk about an electronic watchdog: basically pulsing a gpio pin to trigger a recharge of a cap which holds a transistor active for a second or so, and that transistor drives eg a relay. An alternate method uses a gpio to reset a 555 timer. This will allow machinery to cut off when the embedded circuit stops looping. That is, any attached machinery would have a guaranteed NO (normally open) circuit and can only be engaged when all the watchdogs are working properly.<p>Some mcu pins also go into an unknown state (neither guaranteed high nor low), so resetting a cpu can have bad consequences if it&#x27;s driving big machinery, if not designed correctly.<p>One project I had a pc sending software watchdog pings to several independent devices and each of those had an <i>actual</i> hardware watchdog (as opposed to the cpu resetting one in the article). I used the watchdog to physically control the power to contactors: no watchdog = no power = nothing activates.<p>The system controlled firing of gas burners and fans etc, but the design was very safe, heaps of redundancy and was guaranteed to fail into a safe mode at any instant.

I would like to find a reliable software watchdog that kills a process when a timer expires (for preventing zombie processes from violating lease timeouts).

Never in my career have I ever heard of any other term used than &quot;kicking&quot; the watchdog. Are the other terms popular in America?

This was an awesome read. Thanks for writing this.