Friday, October 9, 2009

Sys Admins: What your Developers want you to know

This is my response to Katie Cunningham's post telling Developers what System Administrators want you to know.

Of course, this is mostly for shops with large groups such as our own. But some of this applies anywhere.

Publish your damned system specifications

If rackspace, webfaction, and the rest of the web hosting world does it for $10/month, then why can't well-paid system administrators do it?

Alas, I've yet to go to a shop where I could find on a wiki page or an otherwise easily accessible document an exact specification of the target production environment. Invariably you have to ask in person, go to another department, ask managers, or go searching through word documents. Word documents?!?


As a developer I should be able to pull up for each server a consistently formatted web page that lists the operating system, operating system specifications, patches, database(s), compilers, shell, disk space, languages, hosted applications, and anything else you as a system administrator can think of adding. If your organization doesn't support a wiki or anything like that, then consider sending out a weekly email. Something as simple as this:
  • Production01 ( RHEL Linux, Python 2.4.4, 2.6.1, Java 1.6, Ports Open: 80, 443
  • Staging02 ( RHEL Linux, Python 2.4.3, 2.6.3, Java 1.6, Ports Open: 80
Look - discrepancies between servers! What a surprise! Glad this is documented!

Based off of this we can build our development environments to match production environments. The QA group can also build their staging environments to match as well. Also, we might be able to better exploit a service you are providing for us rather than writing something from scratch, which means we end up saving time, energy, and money. Everybody wins!

Communication is king

A system administrator needs to be accessible via phone. It just kills me that in some shops their phone numbers are only accessible via their boss. So if you have a problem and that boss is not around the rest of us are plain screwed. Can't our boss have the system administration contact information? Or maybe even us lowly developers?

Also, squirting off emails that you are starting/finishing things is really useful. Or send an email every 30-60 minutes with status. Or jump on IRC/IM and let the very tense developer/QA/business teams know you are still working.

Stick to the schedule

When you have 15 people waiting on you, start the deployment on time. Besides the simple expense of keeping that many people on the clock, delaying a start on people who have already been there for 8 hours is a good way to earn developer/QA/management enmity. Then you'll wonder why they always blame you right off the bat.

Follow the deployment instructions

When it comes to deployment, the hard truth is that your job as a system administrator is to follow instructions. Don't improvise. If the developer instructions don't work (for example, changing to a non-existent directory), don't fix the problem right then and there by opening files and twiddling stuff. Instead, it's time to call this a failed deployment.

That might make you look bad in the short run. Developers might howl. Sometimes you just have to stand firm. A good way to address this issue is to ask to look at a project's deployment instructions before the deployment date. Which brings us to the next point...

Review the deployment instructions early

Smart developers run their deployment instructions early past a system administrator. Take a good hard look at what they are trying to do and let them know if they are doing anything wrong. Good developers will really appreciate what you are doing for them.

Clone the production environment daily

Its 2009. Shouldn't we have fresh staging servers every morning for continuous integration? Yes, I know that when you built server X it matched server Y, but that was 6 months ago. Things have changed on both servers. A smart system administrator can script this out, or so they always tell me...

Communication is king II

So the deployment had problems. Or maybe it didn't. Maybe you had to do a tiny tweak on the instructions to turn an understandable developer mistake into a shining success. Here is what you can do to help defend yourself when things go wrong and to provide developers with a window into what you are doing.

Share the bash session history by use of the Tee command, the logs, and everything else. Without us asking for it! Dump the data and make it easy for us to find. We'll appreciate it and so will you.

Wednesday, October 7, 2009

Have you signed up for the Django Master Class?

Django chief maintainer Jacob Kaplan-Moss is teaching a master level Django class in the Washington, DC area (actually in Springfield, VA) this October 16th.

Jacob Kaplan-Moss is one of the chief maintainers (BDFL) of the Django project. Besides being a technical guru and good teacher, Jacob is a great guy. Approachable, funny, and taught me about anchovies.

Django Master Class

Lets go over some of the hidden perks that the class description does not provide:
  • Even if you know Django already and have memorized the documentation you are sure to pick up some choice bits! BDFL FTW!
  • Ask questions at any time!
  • Provided 8 a.m. breakfast (so get there early!)
  • Provided Lunch
  • Dinner afterward with lots of fun local Django/Python people including Steve Holden
  • Accessible by metro, bus, and car
  • Lots of cheap hotels to stay at nearby so you get good sleep before and after class.

I'm signed up. Are you?