Zen Computer & Zen Administrator

One of the most important books I’ve ever read is Zen Computer by Philip Toshio Sudo. It’s out of print, but you can order it from Amazon resellers.

How to live with constant change, manage it, and deal with the fear it brings -- especially if the change seems overwheling, as it can today -- lies at the heart of zen study, and thus, at the heart of Zen Computer

When changes for the worst occur, the knee-jerk reaction of an administrator is to quickly escalate up to root and begin poking around configuration files and restarting daemons, or even rebooting systems. Every minute that a server or service isn’t functioning properly is money down the drain and a blow to your pride. You’ve tried everything. Nothing has been fixed, and you worry that you might have made things worse than they were before.

It’s exactly at this moment that you need to step back and breathe. Follow your breathing, take deep, long, slow breaths and exhale. Count to ten. Sit up straight, and take in your surroundings.

Now go back and solve the problem.

I nearly forgot these lessons recently when I was troubleshooting a FreeBSD Jail where SSH was not functioning properly. You could SSH into the jail but instead of getting a shell you’d get an error stating that “Warning: Remote host failed or refused to allocate a pseudo tty.” I had run all kinds of diagnostics, including recompiling the base system, and migrating all the client data to a fresh jail.

Nothing was working. Even the fresh jail was getting the same error. I was nearing a full on panic. Then I glanced down at my desk and saw that my copy of Zen Computer was right in front of me. I opened the book and read the chapter on breathing and realized that I had let the stress get to me, and I needed to step back.

After stepping back and thinking for a few minutes, I realized that the ruleset for devfs in the jails was too strict and terminal devices were not being given out to the jails. After minor adjustments to my ruleset and restarting the affected jails, the problem was solved.

The important thing to take away from this experience is that it’s important to keep a cool head when change for the worst occurs.