Kilala.nl - Personal website of Tess Sluijter

Unimportant background
Login
  RSS feed

About me

Blog archives

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

> Weblog

> Sysadmin articles

> Maths teaching

It's great to know one's work is appreciated

2008-06-03 15:05:00

Fuckin' A, man!

As the title of this post says: it's great to know that at least there's -someone- out there who appreciates the work you do :)

Case in point: I've been putting all my college term papers and summaries online and I've been keeping an extensive Wiki with class notes. From time to time a teacher or classmate will suggest that they've had some use for these sites, which is of course quite nice. But last night the aforementioned guidance counselor told me something that made me really happy ^_^

Next year she'll be teaching the second year's General didactics course. Because the course is currently given by one of our somewhat wishy-washy teachers, she was told that she'd have to hunt around and ask people for all the materials. I guess that most of the stuff was never really put to paper. Luckily Lisette knew about my site, found my summary and class notes and was done within a day. Her supervisor was perplexed! :D

So yeah, it's great to know that you're appreciated ^_^


kilala.nl tags: , , ,

View or add comments (curr. 2)

Cutting down on the use of pipes

2007-04-18 14:38:00

One of the obvious down sides to using a scripting language like ksh as opposed to a "real" programming language like Perl or PHP (or C for that matter) is that, for each command that you string together, you're forking off a new process.

This isn't much of a problem when your script isn't too convoluted or when your dataset isn't too large. However, when you start processing 40-50MB log files with multiple FOR loops containing a few IF statements for each line, then you start running into performance issues.

And as I'm running into just that I'm trying to find ways to cut down on the forking, which means getting rid of as many IFs and pipes as possible. Here's a few examples of what has worked for me so far...

Instead of running:

[ expr1 ] && command1

[ expr2 ] && command1

Run:

[ (expr1) && (expr2) ] && command1

Why? Because if test works the way I expect it to, it'll die if the first expression is untrue, meaning that it won't even try the second expression. If you have multiple commands that complement eachother then you ought to be able to fit them into a set of parentheses after test cutting down on more forks.

Instead of running:

if [ `echo $STRING | grep $QUERY | wc -l` -gt 0 ]; then

Run:

if [ ! -z `echo $STRING | grep $QUERY` ]; then

More ideas to follow soon. Maybe I ought to start learning a "real" programming language? :D

EDIT:

OMG! I can't believe that I've just learnt this now, after eight years in the field! When using the Korn shell use [[ expr ]] for your tests as opposed to [ expr ].

Why? Because the [ expr ] is a throw-back to Bourne shell compatibility that makes use of the external test binary, as opposed to the built-in test function. This should speed up things considerably!


kilala.nl tags: , , , ,

View or add comments (curr. 0)

On commenting and debugging your code

2007-04-16 16:38:00

When writing shell scripts for my customers I always try to be as clear as possible, allowing them to modify my code even long after I'm gone. In order to achieve this I usually provide a rather lengthy piece of opening comments, with comments add throughout the script for each subroutine and for every switch or command that may be unclear to the untrained eye.

In general I've found that it's best to have at least the following information in your opening blurb:

* Who made the program? When was it finalised? Who requested the script to be made? Where can the author be reached for questions?

* A "usage" line that shows the reader how to call the program and which parameters are at his disposal.

* A description of what the program actually does.

* Descriptions for each of the parameters and options that can be passed to the script.

* The limitations imposed upon the script. Which specific software is needed? What other requisites are there? What are the nasty little things that may pop up unexpectedly?

* What are the current bugs and faults? The so-called FIXMEs.

* A description of the input that the program takes.

* A description of the output that the program generates.

Equally important is the inclusion of debugging capabilities. Of course you can start adding "echo" lines at various, strategic points in the script when you run into problems, but it's oh-so-much nicer if they're already in there! Adding those new lines is usually a messy affair that can make your problems even worse :( I usually prepend the debugging commands with "[ $DEBUG -eq 1 ] &&", which allows me to turn the debugging on or off at the top of the script using one variable.

And finally, for the more involved scripts, it's a great idea to write a small test suite. Build a script that actually takes the real script through its loops by automatically generating input and by introducing errors.

Two examples of script where I did all of this are check_suncluster and check_log3 with the new TEC-analysis.sh on its way in a few days.

So far, TEC-analysis.sh checks in at:

* 497 lines in total.

* 306 lines of actual code.

* 136 lines of comments.

* 55 lines of debugging code.

Approximately 39% of this script exists solely for the benefit of the reader and user.


kilala.nl tags: , , , ,

View or add comments (curr. 0)

LPIC-102 summary

2007-04-03 23:41:00

The LPIC-102 summary is done. You can find it over here, or in the menu on the left. Enjoy!


kilala.nl tags: , , , , , ,

View or add comments (curr. 0)

Finally! I'm done!

2007-04-03 23:37:00

Calvin hard at work

Ruddy heck, what a day! All in all it took me around thirteen hours, but I've finally finished my LPIC-102 summary. 41 pages of Linuxy goodness, bound to drag me through the second part of my LPIC-1 exams.

Argh, now I'm off to bed. =_= *cough* Let's hope I don't get called for any stand-by work.


kilala.nl tags: , , , ,

View or add comments (curr. 2)

Accessing your Mac at home, from work

2007-03-12 22:08:00

A screenshot of VNC in action.

For weeks on end I've been dragging my iBook along to the office at $CLIENT, even though I'm not allowed to connect it to their network. My iBook is indispensable to me, because it contains all of my archives and past projects, all my e-mail and my address book and calendar. I even use my iBook to keep track of my working hours (thank you TimeLog 3!).

Unfortunately, dragging my laptop around can get tiresome, especially if I ride my bike to work. Which is why I'm very grateful to one of my colleagues for suggesting the use of VNC or another remote desktop solution. Seriously, the suggestion was so obvious that I'm really ashamed that I didn't think of it. I guess I was just clinging -too- much to my dear, sweet iBook.

Anywho... What I'm about to describe is only one of many ways to implement a remote desktop solution for your Mac. A few other options exist, but this is the one I'm using. What we're going to be building is the following:

* I'm at my desk at work, using one of the PCs over there.

* My iBook, running Mac OS 10.4 is at home, connected to my wifi network.

* I will be using my iBook, from my desk at work :)

What you'll need:

* A VNC server. I chose to use Vine Server, which came recommended.

* A VNC client. For Windows and Linux I chose to use Tight VNC and for OS X I use Chicken of the VNC.

* An SSH server. This comes built in, as part of Mac OS X.

* An SSH client. For Windows I use PuTTY, while Linux and OS X come built in with a client.

* Your home IP address. You can find this by browsing to What is my IP address? at home.

Setting up SSH at home

You can use the basic SSH configuration that comes with OS X, but it's not rock solid. If you'd like to be extra secure, please make the following changes. This will disable remote root access and will force each user to make use of SSH keys. If you didn't, you could log in using your normal password which opens you up to brute force password attacks.

* Open Terminal.app and enter the following commands.

cd /private/etc

sudo vi sshd_config

* Change the following lines, so they read as follows. The last two lines a

PermitRootLogin no

PasswordAuthentication no

UsePAM no

* (Re)start SSH

Open System Preferences.

Go to "Sharing".

(Re)start the "Remote access" server.

Setting up the VNC server at home

Vine Server comes in a .DMG and you can simply copy the binary to its desired location. By starting the application you're presented with the applications configuration options, which has buttons at the bottom to stop and start the VNC server.

* You can leave most settings at their default values, but it's extra safe to change the following:

Connection -> set a password

Sharing -> only allow local connections

This secures your VNC server with a password and prevents people on your local network from connecting to your desktop. You'll only be able to login to VNC after logging in to your system through SSH.

* Press the "Start server" button.

Setting up your router

You will need to make your SSH server accessible from the Internet. Configure your router in such a way that it forwards incoming traffic on port 22, to port 22 on your Mac.

Setting up your SSH client at work

If you forced your SSH server to use public/private keypairs earlier, then you'll need to configure your SSH client to do the same. You can use ssh-keygen (OS X and Linux) or PuTTYGen (Windows) to generate a key pair. Please Google around for instructions on how to use SSH keys.

You will need to tell your SSH client to connect to your SSH server at home and to set up port forwarding for VNC. In both examples $HOME-IP is the IP address of your Internet connection at home.

* On Linux and OS X (from the command line): ssh -L 5900:127.0.0.1:5900 $HOME-IP.

* On Windows (in PuTTY): SSH -> Tunnel -> local port = 5900, remote port = 127:0.0.1:5900

What you're doing here is rerouting any traffic that's coming in at your work PC at port 5900 to port 5900 at your home box.

Setting up your VNC client at work

All of the real work is being done by the SSH session, so you can instruct your VNC client to simply connect to desktop 0 at localhost, or at 127.0.0.1. Enter the password that you set up earlier.

Adding more security

Unfortunately Hot Corners don't work through VNC and FUS kills your VNC session, so we'll need to find another way to lock your OS X desktop. Luckily I've found a way in this article. You can use Keychain Access to add a small button to your menu that will allow you to lock your screen.

And there you have it! A fully working VNC setup that will allow you to use your Mac at home, from work.


kilala.nl tags: , , , ,

View or add comments (curr. 0)

The necessity of good reporting

2007-01-26 13:57:00

Finally, I've finised my fourth article for ;Login magazine. It'll appear in next month's issue, in the sysadmin section.

As is the tradition with my articles, I'll try to entice my fellow folks in IT to improve their "soft skills". In the past I've covered things like personal planning and various communications skills. This time I'll try to convey why good reporting is so important to your work and your projects.

HTML version.

PDF version.


kilala.nl tags: , , ,

View or add comments (curr. 0)

All quiet on the western front. Thomas discusses the necessity of good reporting.

2007-01-26 13:08:00

A PDF version of this document is available. Get it over here.

When I returned to the consulting business back in 2005 I found that a change to my modus operandi would have favorable results to the perceived quality of my work.

Up to that point I had never made a big point of reporting my activities to management, trusting that I'd get the job done and that doing so would make everyone happy. And sure enough, things kept rolling and I indeed got things done. But I won't claim that anyone really knew what I was up to or that they had more than a passing notion of my progress.

2005 provided me with a fresh start and I decided that I'd do things differently this time around. And indeed, as my reports started coming in, my client's response to both my employer and myself seemed to improve.

What's the use of reporting?

"So, Peter, what's happening? Aahh, now, are you going to go ahead and have those TPS reports for us this afternoon?"

From the movie 'Office Space'

Reporting. Reports. Status updates. Words that most people in IT dread and that instill nightmare images of piles of paperwork and endless drudgery with managers. Words that make you shudder at the thought of bosses nagging you about layout, instead of content. Of hours of lost time that could have been better spent doing real work.

But seriously, spreading the word about your work and your projects doesn't have to be a huge undertaking and it will probably even help you in getting things done. By spending just a few hours every month you could save yourself a lot of trouble in the long run.

Benefits for the customer:

Benefits for your employer and yourself:

Your first report: describe your assignment

"A lean agreement is better than a fat lawsuit"

German proverb

It may seem slightly odd, but your first report will be made before you've even done any real work. When you start a new project everyone will have a general idea of what you will be doing, but usually no-one has all the details. In order to prevent delays in the future, you will need to make very specific agreements early on.

To get things started you will need to have a little eye-to-eye with your client to draft your assignment. You will be hashing out every significant detail of what the client expects from you.

The good news is that such a meeting usually doesn't take up more than an hour, maybe two. After that you'll need another hour or so to put it all to paper, provided that you have already created a template of sorts.

By putting as much detail as possible into all of these criteria you are creating many opportunities for yourself. From now on everyone agrees on what you will be doing and outsiders can be quickly brought up to speed on your project. At later points in time you can always go back to your document to check whether you're still on the right track. And at the end of everything you can use your original agreement to grade how successful you were in achieving your goals.

So, what if you will be doing "normal" daily management of the customer's servers and IT infrastructure? Doesn't seem like there's a lot to describe, is there? Well, that's when you put extra focus on how things are done. Mind you, even normal daily management includes some projects that you can work on.

Either way, make sure that all demands have been made "SMART": specific, measurable, ambitious, realistic and time-bound. This means that everything should:

When your document is complete, go over it with your client once more to make sure he agrees on everything you put to paper. Then, get his approval in writing.

Here are two examples from my recent assignments. The first example was part of a real project with specific deliverables, while the second example covers normal systems management.

Requirement 1: improving on the old

Our current Nagios monitoring environment is severely lacking in multiple aspects. These include, but are not limited to:

* Sub-optimal design of the infrastructure involved.

* Many services, components and metrics are effectively not monitored.

* Sub-optimal configuration when it comes to alarming.

* Current configuration is a dirty conglomerate of files and objects.

* There is no proper separation between users. People can see monitors to which they really should have no access.



All of these issues should be fixed in the newly designed Nagios environment.

Thomas will take part in the department's normal schedule. This includes the following duties:

* Stand-by duty (being on call), once every five to six weeks.

* The daily shifts, meaning that he either starts his day at 08:00, OR doesn't leave the office before 18:00.

* The expanded schedule with regards to P1 priority incidents and changes. Thomas is expected to put in overtime in these cases.

* The department's change calendar. This involves regular night shifts to implement changes inside specific service windows.

Expanding your activities

You have done your utmost best to make your project description as comprehensive as possible. You've covered every detail that you could think of and even the customer was completely happy at the time.

Unfortunately happiness never lasts long and your client's bound to think of some other things they want you to do. Maybe there's a hike in your deadline, or maybe they want you to install a hundred servers instead of the original fifty. Who knows? Anything can happen! The only thing that's for certain is that it will happen.

When it does, be sure to document all the changes that are being made to your project. Remember, if your original project description is all you have to show at the end, then you'll be measured by the wrong standards! So be sure to go into all the specifics of the modifications and include them in an updated project description.

And of course, again make sure to get written approval from the client.

Interim reporting

Most people I've worked for were delighted to get detailed status updates in writing. Naturally your client will pick up bits and pieces through the grapevine, but he won't know anything for sure until you provide him with all the details. I've found that it is best to deliver a comprehensive document every six to eight weeks, depending on the duration of your undertaking.

Each report should include the following topics. Examples follow after the list.

A short description of your project

The goal of this project is to improve the monitoring capabilities at $CLIENT by completely redesigning the infrastructure, the software and the configuration of the Nagios environment.

Original tasks and their status

Automated installation of UNIX servers Weeks 26 and 27 (28th of June through the 8th of July) were spent full-time on the implementation of the Jumpstart server. $CLIENT had requested I give this part of the project the highest priority, due to recent discoveries regarding the recoverability of certain servers.



At this point in time the so-called Jumpstart server has the following functionality in place:



[...]



Therefore we can conclude that the Jumpstart server has reached full functionality.

Changes to your project

One of the changes made to the project, based on the new technical requirements, is the switch from NRPE to SNMP as a communications protocol. This choice will allow us greater flexibility in the future and will most likely also save us some effort in managing the Nagios clients.



The downside of this choice is my lack of experience in SNMP. This means that I will need to learn everything about SNMP, before I can start designing and building a project that's based upon it.

A simplified timesheet

An simplified time sheet.

Problems and challenges

On the 17th of July I issued a warning to local management that the project would we delayed due to two factors:

* My unfamiliarity with the SNMP protocol and software.

* The lack of a centralized software (and configuration) distribution tool. This lack means that we shall need to install each client manually.

Suggestions and recommendations

$CLIENT is quite lucky to have a CMDB (Configuration Management Database) that is rather up to date. This database contains detailed information on all of their systems and has proved to be very useful in daily life. However, what is lacking is a bird's eye view of the environment. Meaning: maps and lists and such which describe the environment in less detail, but which show a form of method to the madness.

Predictions regarding the outcome of your project

However, as can be seen from the included project planning, I will most probably not be finished with the project before the contract between Snow and $CLIENT runs out.



The contract's end date is set to the 16th of September, while my current estimates point to a project conclusion around the 1st of October. And that's assuming that there will be no delays in acquiring the backup and monitoring software.

Personal contact

One of the biggest mistakes I've made in my recent past was to assume that my customer was reading every document I'd been giving them. I'd been sending them e-mails about show stoppers and I'd been providing them with those beautiful reports I've been telling you about. But still something went horribly wrong. You see, some managers really don't care about technical background and thus they'll ignore certain parts of your reports. They figure that, since you're not coming to talk to them, everything's hunky-dory.

This is exactly why e-mail and big documents are no substitute for good, old face to face contact.

Make sure that you have regular conversations with your client about your progress and any problems that you've run into. You could even go the extra mile and request a regular, bi-weekly meeting! Talking to the customer in person will give you the change to make sure they know exactly what's going on and that they fully understand everything you've written in your interim report.

Everything comes to an end

"You can spend the rest of your life with me...but I can't spend the rest of mine with you. I have to live on. Alone. That's the curse of the Time Lords."

From 2005's season of 'Doctor Who'

Like all masterpieces your enterprise needs a grand finale!

Now that all the work has been done and your goals have been reached you will need to transfer responsibility for everything that you've made. Cross the Ts and dot the Is and all that. In short, you'll be writing an expanded version of the interim report.

The composition of your document should include the following topics:

On the last page of your document, leave room for notes and signatures from your client and their lead technicians. Go over the document with everyone that'll need to take responsibility for your project. When they agree with everything you've written, have them sign that page. You will end up with a page that contains all the autographs that you'll need.

Task review

Solaris automated installation server



[...]



Current status:

Finished per December of 2005. Unfortunately there are a few small flaws still left in the standard build. These have been documented and will be fixed by $CLIENT.

Project recommendations

A basic list of applications to be entered into phase 2 was delivered a few weeks ago. Now we will need to ascertain all the items that should be monitored on a per-application basis.



Once those requirements have been decided on we can continue with the implementation. This includes expanding the existing Nagios configuration, expanding the Nagios client packages and possibly the writing of additional plugins.

Resource expenditure

A graph detailing how I spent my time

Risks and pitfalls

These are areas identified by Thomas as risk areas that need addressing by the $CLIENT team:



1. Limited knowledge of Nagios's implementation of SNMP. 2. Limited knowledge of Perl and Shell scripting in lab team. 3. Limited knowledge of SDS/SVM volume management in lab team. 4. Limited knowledge of Solaris systems management. 5. Only 1 engineer in lab team able to support all aspects of Unix.

Checklists

An example of a checklist.

In conclusion

I've found that many of my customers were pleasantly surprised to receive detailed project reports. It really is something they're not used to from their IT crowd. So go on and surprise your management! Keep them informed, strengthen your bond with them and at the end of the day take in the compliments at a job well done.


kilala.nl tags: , , ,

View or add comments (curr. 0)

LPIC-101 Summary

2006-12-12 22:38:00

Version 1.0 of my LPIC-101 study notes is available. I bashed it together using the two books mentioned below. A word of caution though: this summary was made with my previous knowledge of Solaris and Linux in mind. This means that I'm skipping over a shitload of stuff that might still be interesting to others. Please only use my summary as something extra when studying for your own exam.

I'm up for my exam next Thursday, at ten in the morning. =_=;

Oh yeah... The books:

Ross Brunson - "Exam cram 2: LPIC 1", 0-7897-3127-4

Roderick W. Smith - "LPIC 1 study guide", 978-0-7821-4425-3


kilala.nl tags: , , , , , ,

View or add comments (curr. 0)

I get published! Again :)

2006-02-06 20:37:00

Pride++

Today I received my copy of February's issue of ;Login: which contains the publication of my most recent article. The one about planning projects and your personal time. I love having stuff published. Once Anime Con 2006 is over and done with I'll probably write some more articles! Speaking of Anime Con, I'd better get to work before going to bed :p


kilala.nl tags: , ,

View or add comments (curr. 0)

Keeping track of time. Thomas discusses the art of planning your workload

2005-11-24 08:30:00

A PDF version of this document is available. Get it over here.

Way back in 1999 when I started my second internship I was told to do something that I had never done before: create and maintain a planning of my activities.

At the time this seemed like a horribly complex thing to do, but my supervisor was adamant. He did not want me to shift one bit of work before I had taken a stab at a rough planning. So I twiddled in Word and I fumbled in Excel and finally, an hour or two later, I had finally finished my first project planning ever. And there was much rejoicing!... Well, not really, but I felt that a Monty Python reference would be welcome right about now...

So now it's six years later and I still benefit from the teachings of my past mentor. However, around me I see people who appear to have trouble keeping track of all of their work. Which is exactly why I was originally asked to write this article.

I have always worked in large corporate environments with several layers of management between the deities and me, which always seems to obfuscate matters to no end. However, the ideas outlined in the next few paragraphs will be applicable to anyone in any situation.

Juggling egg shells

"They [hackers] tend to be careful and orderly in their intellectual lives and chaotic elsewhere. Their code will be beautiful, even if their desks are buried in 3 feet of crap."

From "The new hacker's dictionary"

I realize that keeping a planning is definitely not one of the favorite activities for most people in IT. Most of them seem to abhor the whole task, or fail to see its importance. Also, most are of the opinion that they don't have enough time as it is and that there is absolutely no way that they can fit in the upkeep of a personal planning.

Now here's a little secret: the one thing that can help you keep your workload in check IS a planning! By keeping record of all of your projects and other activities you can show management how heavily you're loaded and at which points in time you will be available for additional duties. By providing management with these details they will be able to make decisions like lowering your workload, or adding more people to the workforce.

In this article I will discuss the aspects of making and maintaining a proper personal planning. I will touch on the following subjects:

Personal time vs. Project time

A personal planning is what dictates your day-to-day activities. You use it to keep track of meetings, miscellaneous smaller tasks and time slots that you have reserved for projects. You could say that it's your daily calendar and most people will actually use one (a calendar that is) for this task. In daily life your colleagues and supervisor can use your personal planning to see when you're available for new tasks.

A project planning on the other hand is an elaborate schedule, which dictates the flow of a large project. Each detail will be described meticulously and will receive its own place in time. Depending on the structure of your organization such a plan will be drafted either by yourself, or by so-called project managers who have been specifically hired for that task.

Tools of the trade

"Life is what happens to you, when you're making other plans."

John Lennon

Keeping your personal calendar

I think it's safe to assume that everyone has the basic tools that are needed to keep track of your personal planning. Just about every workstation comes with at least some form of calendar software, which will be more or less suitable.

Microsoft Outlook and Exchange come with a pretty elaborate calendar, as well as a To Do list. These can share information transparently, so you can easily assign a task a slot in your personal planning. Each event in your calendar can be opened up to add very detailed information regarding each task. Also, you and your colleagues can give each other access to your calendars if your organization has a central Exchange server at its disposal.

One of the down sides to this Exchange is the fact that it isn't very easy to keep track of your spent hours in a transparent manner. It allows you to create a second calendar in a separate window, but that doesn't make for easy comparison. You could also try to double book your whole calendar for this purpose, but that would get downright messy.

Looking at the other camp, all Apple Macintosh systems come supplied with the iCal application. It is not as comprehensive as the calendar functions of Exchange, but it is definitely workable. iCal comes with most of the features you would expect, like a To Do list and the possibility to share you calendar with your colleagues. However, this requires that you set up either a .Mac account, or a local WEBDAV server.

A screenshot of iCal

One of the nice things about iCal is the fact that it allows you to keep multiple calendars in one window, thus making it easier to keep track of time spent on projects. In the example shown above the green calendar contains all the events I'd scheduled and the purple calendar shows how my time was really spent.

Finally, I am told that Mozilla's Sunbird software also comes with a satisfactory Calendar. So that could be a nice alternative for those wishing to stick to Linux, or who just have a dislike for the previously mentioned applications.

Keeping track of spent time

It's one thing to enter all of your planned activities into your calendar. Another thing entirely is to keep track of the time you spent. Keeping tabs on how you spend your days gives you the following advantages:

1. Reporting progress towards management.

2. A clear view of which activities are slipping in your schedule.

3. A clear view of which work needs to get rescheduled or even reassigned to somebody else.

However, for some reason there aren't any tools available that focus on this task, at least I haven't been able to find them. Of course there are CRM tools that allow a person to keep track of time spent on different customers, but invariably these tools don't combine this functionality with the planning possibilities that I described earlier.

As I said earlier it's perfectly possible to cram the time you spend on tasks into the same calendar which was used to keep your personal planning in, but that usually gets a bit messy (unless you use iCal). Also, I haven't found any way to create reports from these calendar tools that provide you with a nice comparison between times planned and spent. So for now the best way to create a management-friendly report is still to muck about in your favorite spreadsheet program.

Regarding project planning tools

Most projects are of a much grander scale than your average workweek. There are multiple people to keep track of and each person gets assigned a number of tasks (which in turn get divided sub-tasks and so on). You can imagine that a simple personal calendar will not do.

That's why there is specialized software like Microsoft Project for Windows or PMX for OS X. Tools like these allow you to divide a project into atomic tasks. You can assign multiple resources to each task and all tasks can be interlinked to form dependencies and such. Most tools provide professional functions like Gantt and PERT charts.

A screenshot of MS Project

Making guestimates

In the next few chapters I will ask you to make estimates regarding the time a certain task will take. Often sysadmins will be much too optimistic in their estimates, figuring that "it will take a few hours of tinkering". And it's just that kind of mindset that is detrimental to a good planning.

When making a guestimate regarding such a time frame, clearly visualize all the steps that come with the task at hand. Imagine how much time you would spend on each step, in real life. So keep in mind that computers may choose not to cooperate, that colleagues may be unavailable at times and that you may actually run into some difficulty while performing each step.

So. Have a good idea of how long the task will take? Good! Now double that amount and put that figure up in your planning. Seriously. One colleague recounts of people who multiply their original estimates by Pi and still find that their guestimates are wrong.

One simple rule applies: it is better to arrange for a lot of additional time, than it is to scramble to make ends meet.

Taking the plunge

"It must be Thursday... I never could get the hang of Thursdays."

Douglas Adams ~ "The hitchhiker's guide to the galaxy"

Every beginning is a hard one and this one will be no exception. Your first task will be to gather all the little tidbits that make up your day and then to bring order to the chaos. Here are the steps you will be going through.

1. Make a list of everything you have been doing, are doing right now and will need to do sometime soon. Keep things on a general level.

2. Divide your list into two categories: projects and tasks. In most cases the difference will be that projects are things that need to be tackled in a structural manner that will take a few weeks to finish, whereas tasks can be handled quite easily.

3. Take your list of tasks and break them down into "genres". Exemplary genres from my planning are "security", "server improvements" and "monitoring wish list". The categorized list you've made will be your To Do list. Enter it into your calendar software.

4. For each task decide when it needs to be done and make a guestimate regarding the required time. Start assigning time slots in your calendar to the execution of these activities. I usually divide my days into two parts, each of which gets completely dedicated to one activity. Be sure that you leave plenty of room in your calendar for your projects. Also leave some empty spots to allow for unforeseen circumstances.

Now proceed with the next paragraph to sort out your projects.

The big stuff: handling projects

"Once I broke my problems into small pieces I was able to carry them, just like those acorns: one at a time. ... Be like the squirrel!"

The white stripes ~ "Little acorns"

For each of your projects go through the following loop:

1. Write a short project overview. What is it that needs to be done? When does it need to be done? Who are you doing it for? Who is helping you out?

2. Make a basic time line that tells you which milestones need to be reached in order to attain your goal. For example: if the goal is to have all your servers backed up to tape, exemplary mile stones could be "Select appropriate software/hardware solution", "Acquire software/hardware solution", "Build basic infrastructure" and "Implement backup solution". For each milestone, decide when it needs to be reached.

3. Work out each defined milestone: which granular tasks are parts of the greater whole? For instance, the phase "Select appropriate software/hardware solution" will include tasks such as "Inventory of available software/hardware", "Initial selection of solution", "Testing of initially selected solution" and so on.

4. For each of these atomic tasks, decide how much time will be needed in order to perform it. Use the tips regarding guestimates to decide on the proper figures.

5. Put all the tasks into the time line. Put them in chronological order and include the time you've estimated for each task. Once you're done you've built a basic Gantt chart.

The process of Gannt chart creation

Once you are done, go over the whole project planning and verify that, given the estimated time for each task, you can still make it on time. Discuss your findings with your management so they are know what you are up to and what they can expect from the future.

Inevitable, like taxes and death

"Hackers are often monumentally disorganized and sloppy about dealing with the physical world. ... [Thus] minor maintenance tasks get deferred indefinitely."

From "The new hacker's dictionary"

One of the vitally important facts about planning is that it's not a goal, but an on-going process. Now that you have made your initial planning, you're going to have to perform upkeep. Ad infinitum. The point is that things change and there's no changing that!

Projects fall behind because of many different reasons. Vendors may not deliver on time, colleagues may fail to keep their promises and even you yourself may err at times. Maybe your original planning was too tight, or maybe a task is a lot more complicated than it seemed at first. All in all, your planning will need to be shifted. Depending on the project it is wise to revisit your planning at least once a week. Mark any finished tasks as such and add any delays. Not only will this help you in your daily work, but it will also give management a good idea about the overall progress of your projects.

The same goes for your personal time. Projects need rescheduling, you need to take some unexpected sick leave or J. Random Manager decides that doing an inventory of mouse mats really does need priority above your projects. It is best to revisit your calendar on a daily basis, so you can keep an eye on your week. What will you be doing during the next few days? What should you have done during the past few days? Are you on track when it comes to your To Do list?

Final thoughts

You may think that all of this planning business seems like an awful lot of work. I would be the first to agree with you, because it is! However, as I mentioned at the start of this article: it will be well worth your time. Not only will you be spending your time in a more ordered fashion, but it will also make you look good in the eyes of management.

Drawing a parallel with the Hitchhiker's Guide to the Galaxy you will be the "really hoopy frood, who really knows where his towel is" because when things get messy you will still be organized.


kilala.nl tags: , , ,

View or add comments (curr. 1)

My article on personal planning

2005-09-07 10:15:00

It has been a very long time in coming, but finally I got around to finishing version 1.0 of my tutorial/article on planning. One thing most people in IT are notoriously bad at is writing and maintaining a project planning (or their personal planning for that matter). I was originally asked by my chef at $PREV-CLIENT to write this article, but I never got around to finishing it before i left.

Aniway... It's still version 1.0, so it's still quite rough around the edges. I hope to get a whole load of reviews from friend/colleagues before submitting it to ;Login: for publication. In the mean time you guys can find it in the Sysadmin section. I hope you enjoy reading it!


kilala.nl tags: , ,

View or add comments (curr. 0)

When disaster strikes! Thomas and Roland discuss crisis management

2005-05-31 22:15:00

A PDF version of this document is available. Get it over here.

We've all experienced that sinking feeling: blurry-eyed and not halfway through your first cup of coffee you're startled by the phone. Something's gone horribly wrong and your customers demand your immediate attention!

From then on things usually only get worse. Everybody's working on the same problem. Nobody keeps track of who's doing what. The problem has more depth to it than you ever imagined and your customers keep on calling back for updates. It doesn't matter whether the company is small or large: we've all been there at some point in time.

The last time we encountered such an incident at our company wasn't too long ago. It wasn't a pretty sight and actually went pretty much as described above. During the final analysis our manager requested that we produce a small checklist which would prevent us from making the same mistakes again. The small checklist finally grew into this article which we thought might be useful for other system administrators as well.

Before we begin we'd like to mention that this article was written with our current employer in mind: large support departments, multiple tiers of management, a few hundred servers and an organization styled after ITIL. Most of the principles that are described in this document also apply to smaller departments and companies albeit in a more streamlined form. Meetings will not be as formal, troubleshooting will be more supple and communication lines between you and the customer will be shorter.

Now, we have been told that ITIL is a mostly European phenomenon and that it is still relatively unknown in the US and Asia. The web site of the British Office of Government Commerce (http://www.itil.co.uk) describes ITIL as follows:

"ITIL (IT Infrastructure Library) is the most widely accepted approach to IT Service Management in the world. ITIL provides a cohesive set of best practice, drawn from the public and private sectors internationally.



ITIL is ... supported by publications, qualifications and an international user group. ITIL is intended to assist organizations to develop a framework for IT Service Management."

Some readers may find our recommendations to be strict, while other may find them completely over the top. It is of course up to your own discretion how you deal with crises.

Now. Enough with the disclaimers. On with the show!

A method to the madness

The following paragraphs outline the phases which one should go through when managing a crisis. The way we see things, phases 1 through 3 and phase 11 are all parts of the normal day to day operations. All steps in between (4 through 10) are steps to be taken by the specially formed crisis team.

1. A fault is detected

2. First analysis

3. First crisis meeting

4. Deciding on a course of action

5. Assigning tasks

6. Troubleshooting

7. Second crisis meeting

8. Fixing the problems

9. Verification of functionality

10. Final analysis

11. Aftercare

1. A fault is detected

"Oh the humanity!..."

Reporter at the crash of the Hindenburg

I really doesn't matter how this happens, but this is naturally the beginning. Either you notice something while V-grepping through a log file, or a customer calls you, or some alarm bell starts going off in your monitoring software. The end result will be the same: something has gone wrong and people complain about it.

In most cases the occurrence will simply continue through the normal incident process since the situation is not of a grand scale. But once every so often something very important breaks and that's when this procedure kicks in.

2. First analysis

"Elementary, dear Watson."

The famous (yet imaginary) detective Sherlock Holmes

To be sure of the scale of the situation you'll have to make a quick inventory:

Once you have collected all of this information you will be able to provide your management with a clear picture of the current situation. It will also form the basis for the crisis meeting, which we will discuss next.

This phase underlines the absolute need for detailed and exhaustive documentation of your systems and applications! Things will go so much smoother if you have all of the required details available.1 If you already have things like Disaster Recovery Plans lying around, gather them now.

If you don't have any centralized documentation yet we'd recommend that you start right now. Start building a CMDB, lists of contacts and so-called build documents describing each server.

3. First crisis meeting

"Emergency family meeting!"

From "Cheaper by the dozen"

Now the time has come to determine how to tackle the problem at hand. In order to do this in an orderly fashion you will need to have a small crisis meeting.

Make sure that you have a whiteboard handy, so you can make a list of all of the detected defects. Later on this will make it easier to keep track of progress with the added benefit that the rest of your department won't have to disturb you for updates.

Gather the following people:

During this meeting the on-call team member brings everybody up to speed. The supervisor is present so that he/she may be prepared for any escalation from above, while the problem manager needs to be able to inform the rest of your company through the ITIL problem process. Of course it is clear why all of the other people are invited.

4. Deciding on a course of action

One of the goals of the first crisis meeting is to determine a course of action. You will need to set out a clear list of things that will be checked and of actions that need to be taken, to prevent confusion along the way.

It is possible that your department already has documents like a Disaster Recovery Plan or notes from a previous comparable crisis that describe how to treat your current situation. If you do, follow them to the letter. If you do not have documents such as these you will need to continue with the rest of our procedure.

5. Assigning tasks

Once a clear list of actions and checks has been created you will have to assign tasks to a number of people. We have determined a number of standard roles:

It is imperative that the spokesperson is not involved with any troubleshooting whatsoever. Should the need arise for the spokesperson to get involved, then somebody else should assume the role of spokesperson in his place. This will ensure that lines of communication don't get muddled and that the real work can go on like it should.

6. Troubleshooting

In this phase the designated troubleshooters go over the list of possible checks that was determined in the first crisis meeting. The results for each check need to be recorded of course.

It might be that they find some obvious mistakes that may have led to the situation at hand. We suggest that you refrain from fixing any of these, unless they are really minor. The point is that it would be wiser to save these errors for the meeting that is discussed next.

This might seem counterintuitive, but it could be that these errors aren't related to the fault or that fixing them might lead to other problems. This is why it's wiser to discuss these findings first.

7. Second crisis meeting

Once the troubleshooters have gathered all of their data the crisis team can enter a second meeting.

At this point in time it is not necessary to have either the supervisor or the problem manager present. The spokesperson and the troubleshooters (perhaps assisted by a specialist who's not on the crisis team) will decide on the new course of action.

Hopefully you have found a number of bugs that are related to the fault. If you haven't, loop back to step 4 to decide on new things to check. If you did, now is the time to decide how to go about fixing things and in which order to tackle them.

Make a list of fixable errors and glance over possible corrections. Don't go into too much detail, since that will take up too much time. Leave the details to the person who's going to fix that particular item. Assign each item on the list to one of the troubleshooters, and decide in which order they should be fixed.

When you're done with that, start thinking about plan B. Yes, it's true that you have already invested a lot of time into troubleshooting your problems, but it might be that you will not be able to fix the problems in time. So decide on a time limit if it hasn't been determined for you and start thinking worst case scenario: "What if we don't make it? How are we going to make sure people can do their work anyway?O.

8. Fixing the problems

Obviously you'll now tackle each error, one by one. Make sure that you take notes of all of the changes that are made. Once more though (I'm starting to feel like the school teacher from The Wall): don't be tempted to do anything you shouldn't be doing.

Don't go fixing other faults you've detected. And absolutely do not use the downtime as a convenient time window to perform an upgrade you'd been planning of doing for a while.

9. Verification of functionality

Once you've gone over the list errors and have fixed everything verify that peace has been brought to the land, so to speak. Also verify that your customers can work again and that they experience no more inconvenience. Strike every fixed item from the whiteboard, so your colleagues are in the know.

If you find that there are still some problems left, or that your fixes broke something else, add them to the board and loop back to phase 3.

10. Final analysis

"Analysis not possible... We are lost in the universe of Olympus."

Shirka the board computer, from "Ulysses31"

Naturally your customers will want some explanation for all of the problems you caused them (so to speak). So gather all people involved with the crisis team and hold one final meeting. Go over all of the things you've discovered and make a neat list. Cover how each error was created and its repercussions. You may also want to explain how you'll prevent these errors from happening again in the future.

What you do with this list depends entirely on the demands set out by your organization. It could be that all your customers want is a simple e-mail, while ITIL-reliant organizations may require a full blown Post Mortem.

11. Aftercare

"I don't think any problem is solved unless at the end of the day you've turned it into a non-issue. I would say you're not doing your job properly if it's possible to have the same crisis twice.O

Salvaico, Sysadmintalk.com forum member

Apart from the Post Mortem which was already mentioned you need to take care of some other things.

Maybe you've discovered that the server in question is under powered or that the faults experienced were fixed in a newer version of the software involved. Things like these warrant the start of a new project at the cost of your customers. Or maybe you've found that your monitoring is lacking when it comes to the resource(s) that failed. This of course will lead to an internal project for your department.

All in all, aftercare covers all of the activities required to make sure that such a crisis never occurs again. And if you cannot prevent such a crisis from happening again you should document it painstakingly so that it may be solved quickly in the future.

Final thoughts

We sincerely hope that our article has provided you with some valuable tips and ideas. Managing crises is hard and confusing work and it's always a good idea to take a structured approach. Keeping a clear and level head will be the biggest help you can find!


kilala.nl tags: , , ,

View or add comments (curr. 0)

Zen and the art of getting what you want

2004-12-03 18:43:00

A PDF version of this document is available. Get it over here.

Throughout the last two years I have written a number of technical proposals for my employer. These usually concerned either the acquisition of new hardware or modifications to our current infrastructure.

Strangely enough my colleagues didn't always get the same amount of success as I did, which got me to thinking about the question "How does one write a proper proposal anyway?".

In this mini-tutorial I'll provide a rough outline of what a proposal should contain, along with a number of examples. Throughout the document you'll also find a number of Do's and Don'ts to point out common mistakes.

You will also notice that I predominantly focus on the acquisition of hardware in my examples. This is due to the nature of my line of work, but let me say that the stuff I'll be explaining applies to many other topics. You may just as well apply them to desired changes to your network, some software that you would like to use and even to some half-assed move which you want to prevent management from making.

Now I've never been a great fan of war, but Sun Tzu really knew his stuff! Even today his philosophies on war and battle tactics are still valid and are regularly applied. And not just in the military, since these days it's not uncommon to see corporate busy-bodies reading "The art of war" while commuting to work. In between my stuff you'll find quotes from Sun Tzu which I thought were applicable to the subject matter.

Zen and the art of 'getting what you want'

"Though we have heard of stupid haste in war, cleverness has never been seen associated with long delays."

Sun Tzu ~ "The art of war"

It happens occasionally that I overhear my colleagues talking about one of their proposals. Sometimes the discussion is about the question of why their idea got shot down and "what the heck was wrong with the proposal?". They had copied a proposal that had worked in the past and replaced some information with their own. When asked to show me said proposal, I'm presented with two sheets of paper, of which one is the quote from our vendor's sales department. The other consists of 30% header/footer, a short blurb on what we want to buy and a big box repeating all of the pricing info.

The problem with such a document of course is the fact that management gets its nose rubbed in the fact that we want to spend their money (and loads of it too). To them such a proposal consists of a lot of indecipherable technical mumbo-jumbo (being the quote and some technical stuff), with the rest of the document taking up dollars-Dollars-DOLLARS. While to you it may seem that the four or five lines of explanation provide enough reason to buy the new hardware, to management this will simply not do.

So? One of the things it takes to write a proper proposal is to write one which keeps your organization's upper echelons in mind. However! Don't forget about your colleagues either. It is more than logical that you run a proposition past your peer to check if they agree with all of the technicalities.

So in order to make sure that both your targets agree on your proposal you will have to:

A) employ tech-speak to reach your peers and

B) explain your reasoning in detail to your management.

In order to craft such a document there are a number of standard pieces to the puzzle which you can put in place. I'll go over them one by one. One thing I want you to realize though, is the fact that this will take time. Expect to spend at least half a day or even more on writing.

Pieces to the puzzle

My proposals tend to consist of a number of sections, some of which are optional as not every type of proposal requires the information contained therein. For instance, not every project will require resources which can be easily expressed in numbers and hence there is no need for a list of costs.

  1. A summary: describes in short your current problem, your solution for this and the estimated costs.
  2. Introducing the scope: gives your audience a clear picture of the troubled environment involved.
  3. Problems and solutions: describes in detail what is wrong, what the repercussions are (and what they may become) and your proposed solution.
  4. What is required? A list of things that you'll need to fix the problem.
  5. Other options: of course management wants the ability to save money. Here's where you give them the option to.
  6. Making it work: describes which departments need to put in resources and what their tasks will be.
  7. A break-down: the costs of the various options, set off against their merits and flaws.
  8. Final words: a least plea to your audience.

1. A summary

"The art of war teaches us to rely not on the likelihood of the enemy's not coming, but on our own readiness to receive him."

Sun Tzu ~ "The art of war"

Keep this part as short and simple as possible. Use one, maybe two short paragraphs to describe the current situation or problem and describe how you'll fix it. Use very general terms and make sure that it is clear which of the reader's needs you are addressing.

Be very careful not to put too much stuff in this section. Its main purpose is to provide the reader with a quick overview on what the the problem is that you're trying to solve and what your final goal is. Not only will this allow the reader to quickly grasp the subject of your proposal, but it will also make sure that it will be more easily found on a cluttered desk. A short summary means quick recognition.

For example:

In the past year UNIX Support have put a big effort into improving the stability and performance capacity of their BoKS and NIS+ infrastructural systems. However, the oldest parts of our infrastructure have always fallen outside the scope of these projects and have thus started showing signs of instability. This in turn may lead to bigger problems, ending in the complete inaccessibility of our UNIX environment.



I propose that we upgrade these aging servers, thus preventing any possible stability issues. The current estimated cost of the project is $16,260.

2. Introducing the scope

Most managers only have a broad view on things that are going on in the levels beneath them when it comes down to the technical nitty-gritty. That's the main reason why you should include a short introduction on the the scope of your proposal.

Give a summary of the services that the infrastructure delivers to the "business". This helps management to form a sense of importance. If a certain service is crucial to your company's day to day operation, make sure that your reader knows this. If it will help paint a clearer picture you can include a simple graphic on the infrastructure involved.

The whole point of this section is to imprint it on management that you are trying to do something about their needs, not yours. It's one thing to supply you with resources to tickle one of your fancies, but it's a wholly different thing to pour money into something that they themselves need.

For example:

BoKS provides our whole UNIX environment with mechanisms for user authentication and authorization. NIS+ provides all of the Sun Solaris systems from that same environment with directory services, containing information on user accounts, printers, home directories and automated file transfer interfaces.



Without either of these services it will be impossible for us to maintain proper user management. Also, users will be unable to log in to their servers should either of these services fail. This applies to all departments making use of UNIX servers, from Application and Infrastructure Support, all the way through to the Dealing Room floor.

3. Problems and proposed solutions

"Whoever is first in the field and awaits the coming of the enemy, will be fresh for the fight; whoever is second in the field and has to hasten to battle will arrive exhausted." Sun Tzu ~ "The art of war"

Meaning: when writing your proposal try to keep every possible angle on your ideas into mind. Try to anticipate any questions your reader might have and bring your ideas in such a way that they will appeal to your audience. If you simply describe your goal, instead of providing proper motivation you'll be the one who "is second in the field".

In the previous section of your document you provided your audience with a quick description of the environment involved. Now you'll have to describe what's wrong with the current situation and what kind of effects it may have in the future. If your proposal covers the acquisition or upgrade of multiple objects, cover them separately. For each object define its purpose in the scope you outlined in the previous section. Describe why you will need to change their current state and provide a lengthy description of what will happen if you do not.

However, don't be tempted to exaggerate or to fudge details so things seem worse. First off a proposal which is overly negative may be received badly by your audience. And secondly, you will have to be able to prove all of the points you make. Not only will you look like an ass if you can't, but you may also be putting your job on the line! So try to find the middle road. Zen is all about balance, and so the 'art of getting what you want' should also be.

For example:

Recently the master server has been under increased load, causing both deterioration of performance and stability. This in turn may lead to problems with BoKS and with NIS+, which most probably will lead to symptoms like:

* Users will need more time to log into their UNIX accounts.

* Users may become unable to log into their UNIX accounts.

* User accounts and passwords may lose synchronocity.

Close off each sub-section (one per object) with a clearly marked recommendation and a small table outlining the differences between the current and the desired situation. Keep your recommendation and the table rather generic. Do not specify any specific models or makes of hardware yet.

Of course the example below is focused on the upgrading of a specific server, but you can use such a table to outline your recommendations regarding just about anything. Versions of software for example, or specifics regarding your network architecture. It will work for all kinds of proposals.

For example:

UNIX Support recommends upgrading the master server's hardware to match or exceed current demands on performance



System type: Sun Netra T1 200 (current), - (recommended)

Processor: Ultrasparc IIe, 500 MHz (current), 2x Ultrasparc IIIi, 1 GHz (recomm.)

Memory: 512 MB (current), 1 or 2 GB (recomm.)

Hard drives: 2x 18 GB + 2x 18 GB ext (current), 2x 36 GB, int. mirror (recomm.)

The whole point of this section of your proposal is to convince your readers that they're the captain of Titanic and that you're the guy who can spot the iceberg in time. All is not lost... Yet...

4. What is required to make this work?

"The general who wins a battle makes many calculations in his temple ere the battle is fought. The general who loses a battle makes but few calculations beforehand."

Sun Tzu ~ "The art of war"

Now that you have painted your scenario, and you've provided a vision on how to go about solving things you will need to provide an overview of what you will be needing.

Don't just cover the hardware you'll need to acquire, but also take your time to point out which software you'll need and more importantly: which departments will need to provide resources to implement your proposal. Of course, when it comes to guesstimates regarding time frames, you are allowed to add some slack. But try to keep your balance and provide your audience with an honest estimate.

One thing though: don't mention any figures on costs yet. You'll get to those later on in the proposal.

For example:

A suitable solution for both Replica servers would be the Sun Fire V210. These systems will come with two Ultrasparc II processors and 2 GB of RAM installed. This configuration provides more than enough processing power, but is actually cheaper than a lower spec-ed V210.

5. Other viable options

"Do not interfere with an army that is returning home. When you surround an army, leave an outlet free. Do not press a desperate foe too hard."

Sun Tzu ~ "The art of war"

The above quote seems to be embodied in one of Dilbert's philosophies these days: "Always give management a choice between multiple options, even if there is only one".

Of course, in Dilbert's world management will always choose the least desirable option, for instance choosing to call a new product the 'Chlamydia', because "it sounds Roman". It will be your task to make the option you want to implement to be the most desirable in the eyes of your readers.

In case your proposal involves spending money, this is where you tell management: "Alright, I know times are lean, so here's a number of other options. They're less suitable, but they'll get the job done". Any which way, be sure that even these alternatives will do the job you'd want them to. Never give management the possibility to choose an option that will not be usable in real life.

For example:

Technically speaking it is possible to cut costs back a little by ordering two new servers, instead of four, while re-using two older ones. This alternate scenario would cut the total costs back to about $ 8360,-- (excluding VAT).

If the main subject of your proposal is already the cheapest viable option, say so. Explain at length that you have painstakingly eked out every penny to come up with this proposal. Also mention that there are other options, but that they will cost more money/resources/whatever. Feel free to give some ball park figures.

For example:

Unfortunately there are no cheaper alternatives for the Replica systems. The Sun Fire V120 might have been an option, were it not for the following facts:

It is not in the support matrix as defined by UNIX Support.

It is not natively capable of running IP Multi Pathing.

It will reach its so-called End Of Life state this year.

Basically you need to make management feel good about their decision of giving you what want. You really don't want them to pick any other solution than the one you're proposing, but you are also obliged to tell them about any other viable possibilities.

6. Making it work

In the case of some projects you are going to need the help of other people. It doesn't matter if they are colleagues, people from other departments or external parties. In this section you will make a list of how many resources you are going to need from them.

You don't have to go into heavy details, so give a broad description of the tasks laid out for these other parties. Estimate how much time it will take to perform them in man-hours and also how many people you will need from each source. Not only will this give management a clear picture of all of your necessities, a list like this will also give your readers a sense of the scale of the whole project.

For example:

In order to implement the proposed changes to our overall security we will require the cooperation of a number of our peer departments: Information Risk Management (IRM) will need to provide AS and our customers with clear guidelines, describing the access protocols which will be allowed in the future. It is estimated that one person will require about 36 hours to handle all of the paper work.



Security Operations (GSO) will need to slightly modify their procedures and some of the elements of their administrative tools, to accommodate for the stricter security guidelines. It is estimated that one person will need about 25 hours to make the required alterations.

7. Breaking things down

You'll need to try and keep this section as short as possible, since it covers the costs of all of the viable options that you provided in the past sections. Create a small table, setting off each option against the costs involved. Add a number of columns with simple flags which you can use to steer the reader to the option of your choice.

Reading back I realize that I'll need to clarify that a bit :) Try and recall some of those consumer magazines or sites on the web. Whenever they make a comparison between products they often include a number of columns marked with symbols like + (satisfactory), ++ (exceeds expectations), - (not too good) or -- (horrific). What you'll be doing is thinking of a number of qualities of your options which you can set off against each other.

It goes without saying that you should be honest when assigning these values. If another option starts to look more desirable by now you really have to re-evaluate your proposal.

For example:

A table detailing your various options.

8. Final words

"The clever combatant imposes his will on the enemy, but does not allow the enemy's will to be imposed on him."

Sun Tzu ~ "The art of war"

Use two, maybe three, paragraphs to make one final impressing blow on the reader. Shortly summarize the change(s) that you're proposing and repeat your arguments. Be firm, yet understanding in your writing.

For example:

We have provided you with a number of possible scenarios for replacement, some options more desirable than others. In the end however we are adamant that replacement of these systems is necessary and that postponing these actions may lead to serious problems within our UNIX environment, and thus in our line of business.

Regarding tone and use of language

At all times keep in mind who your target audience is. It is quite easy to fall back into your daily speech patterns when writing an extensive document, while at some point that may actually lead to catastrophe.

Assume that it is alright to use daily speech patterns in a document which will not pass farther than one tier above your level (meaning your supervisor and your colleagues). However, once you start moving beyond that level you will really need to tone down.

Some points of advise:

Regarding versioning and revisioning

At my current employer we have made a habit of including a small table at the beginning of each document which outlines all of the versions this document has gone through. It shows when each version was written and by whom. It also gives a one-liner regarding the modifications and finally each version has a separate line showing who reviewed the document.

Of course it may be wise to you use different tables at times. One table for versions that you pass between yourself and your colleagues and one for the copies that you hand out to management. Be sure to include a line for the review performed by your supervisor in both tables. It's an important step in the life cycle of your proposal.

This may be taking things a bit far for you, but it's something we've grown accustomed to.

Final thoughts

"Begin by seizing something which your opponent holds dear; then he will be amenable to your will." Sun Tzu ~ "The art of war"

Or in other words: management is almost sure to give in, if you simply make sure they know things will go horribly wrong with their environment if you are not allowed to do what you just proposed.

Of course no method is the be-all-and-end-all way of writing proposals, so naturally neither is mine. Some may simply find it too elaborate, while in other cases management may not be very susceptible to this approach. Try and find your own middle road between effort and yield. Just be sure to take your time and to be prepared for any questions you may get about your proposal.


kilala.nl tags: , , ,

View or add comments (curr. 1)

Writing more articles

2004-11-11 21:55:00

The way things are looking right now I'll be writing a whole series of articles for the discerning system admin :) As you know I finished an article on the crafting of proposals a week or so ago. Now I'm also planning to do articles on "keeping personal and project plannings" and on "catastrophe management".

I'll also be using my lovely Powermac G5 for something completely new today! At the office we lost two passwords for NIS+ credentials and luckily we managed to retrieve what we _think_ are the encrypted passwords strings. So now I'll try and use John the Ripper to crack the passwords. I've no clue how long this'll take and I hope I can get things finished before the 20th. 'Cause that's when I need the damn passwords :)


kilala.nl tags: , ,

View or add comments (curr. 0)

Reviewing NIS+ books

2004-11-11 19:51:00

I've added a little review page for books on the topic of NIS+, since that's something I'm currently very into at the office.


kilala.nl tags: , , , ,

View or add comments (curr. 0)

How to write proposals

2004-11-01 09:33:00

Version 2.0 of my tutorial on writing proposals is available from the menu now. Share and enjoy!


kilala.nl tags: , , ,

View or add comments (curr. 0)

Writing technical proposals

2004-10-23 09:05:00

Finally my work on the HOWTO for writing technical proposals is done! I've added the PDF file to the menu bar on the left. Unfortunately, for some reason PDF printing from OpenOffice doesn't always seem to work properly. The file in the menu bar prints perfectly when dragged onto my desktop printer (albeit in black and white, and not in color), but both Preview and Acrobat Reader refuse to open the file.

If any of you guys happen to have any problems opening the file, please let me know. I'll see what I can do to get things fixed.


kilala.nl tags: , , ,

View or add comments (curr. 0)

SCNA erratum

2004-09-22 20:01:00

In the menu of the Sysadmin section you will also find a link to a small erratum which I wrote after reading Rick Bushnell's book. As you can see I found quite a number of errors. I also e-mailed this list to Prentice Hall publishers and hope that they will make proper use of the list.


kilala.nl tags: , , , ,

View or add comments (curr. 0)

SCNA summary done!

2004-09-15 22:13:00

Well, it took me a couple of days, but finally it's done: my summary own the "SCNA study guide" by Rick Bushnell (see the book list). I'll be taking my first shot at the SCNA exam in about a week (the 22nd, keeping my fingers crossed), so I'm happy that I've finished the document. I thought I'd share it with the rest of you; maybe it'll be of some use.

All 29 pages are available for download as a PDF from the Sysadmin section.


kilala.nl tags: , , , , ,

View or add comments (curr. 0)

Burn baby burn! Configuring the OS X firewall.

2004-04-03 00:00:00

It's only been a couple of months since I switched to Apple OS X, but since then I've learnt many a thing about the OS. It was only recently though that I found the need to configure the built in firewall. This little HOWTO'll explain all of the steps I took.

The built in firewall software is one of the many OS X features that Apple likes to tote around, claiming a higher level of security out of the box when compared to other OSes. And yes indeed the firewall software does appear to do its job properly. With one exception...

Conventions used within this document

Before we begin I would like to point out a couple of conventions I will be using in this document. Whenever you encounter any text written in courier new bold, this means that you're either reading commands which need to be entered into the UNIX command line of OS X or a list of packages or menu names. You will also encounter lines starting with the text "kilala:~ thomas$". This is merely the command prompt as displayed on my system and I include it in these texts to indicate the commands to be entered.

Firewalls? What the heck?...

First off I can imagine that some of the people reading this can't even begin to imagine what a firewall is or does. They might've heard the word before on the web or in Apple's (or Microsoft's) PR spiel. I won't go into any technical details, but I'll give a short explanation on the ideas behind a firewall. If you would like more detailed information I recommend a website like http://computer.howstuffworks.com/firewall.htm.

Firewalls are a sort of security measure which work by separating your computer or network from a possibly hostile network, like the Internet. This separation usually takes place by disallowing any and all network traffic to and from your systems, while only allowing a certain number of protocols in and/or out. For instance, a home user may setup his firewall to block everything except outgoing e-mail and browser traffic. On the other hand some companies could be allowing incoming browser requests to their own webserver, next to the already mentioned outgoing e-mail and browser traffic.

One of the most important things to remember though is the fact that a firewall is not the be-all-end-all security measure that fixes all of your problems. It could still be that the software serving the protocols that you do allow through the firewall is buggy with security flaws. Think of Microsoft's ISS webserver software which was famous for security holes in the past.

OS X! What's the problem?

As I said it was only recently that I found the need to manually configure OS X's built in firewall software. I'd always kind of expected the software to work straight out of the box, which it kind of did.

You see, usually with firewall software you'll say "I want to block any and all traffic to and from my box, except this, this and that protocol". Basically you do the same with OS X's firewall, but with a small snake in the grass: the protocols you allow to go in and out of your systems get permission on all of your network interfaces! So if you're hooked up onto the Internet (which I assume since you found my little article) and if you decide to turn on that Windows file sharing, remember that you're sharing your files with the rest of the Internet! You can imagine I was less than pleased with this and I can't even begin to imagine why it took me two months to start thinking about this. Usually I'm more security minded! Anywho, the damage was done and I decided to quickly learn enough about the OS X firewall, so I could configure it properly.

Reconfiguring the OS X firewall

I quickly found out that OS X uses the BSD UNIX default firewall ipfw, which can be configured in many different ways. There's Apple's custom window in the System Preferences panes. Then there are GUI's like Brickhouse and Firewalker which are available through the Internet. And finally you can take the manual approach and enter ipfw firewall rules one by one, by hand.

I chose to use the manual approach, since that is what I'm most familiar with; I've been entering firewall rules since my internship at Spherion when I was still running a firewall on Suse Linux 6.0. An added bonus to entering the rules by hand is that you know 100% sure what the firewall will do, as opposed to rules created or generated by a GUI.

I wouldn't expect Joe and Little Timmy from across the street to use to approach, so I would recommend people who're less technically involved to give software like Brickhouse a try. I hear it's supposed to be pretty good!

For the lazy people...

People who don't like typing big files by hand can download the file Firewall-config.tar from my website. This file contains all files which are to be placed in /Library/StartupItems/Firewall.

First things first

In a minute we'll start looking at how we create rules for our firewall. But as the title says: "First things first"! Because we want our own set of rules to bypass the OS X default rules we'll need to make sure that our configuration gets loaded right after the systems comes up. This is done by adding a new boot configuration for the firewall. I'll just show you all the steps I took, along with some explanations; that should make things clear enough.

First off, make sure that you're in a user account which is allowed to use the sudo command. This could be the Administrator account, but you could also modify your own account for this purpose. Then open up a Terminal.app window.

Last login: Sun Apr 4 09:46:44 on ttyp1

Welcome to Darwin!

kilala:~ thomas$ cd /Library

kilala:~ thomas$ sudo mkdir -p StartupItems/Firewall

Password:

kilala:~ thomas$ sudo chown -R root:wheel StartupItems

kilala:~ thomas$ sudo chmod -R 755 StartupItems

kilala:~ thomas$ cd StartupItems/Firewall

kilala:~ thomas$ sudo cp -rp /Systems/Library/StartupItems/NFS/* .

The previous commands created a new boot configuration directory for the service we will call Firewall. You setup the directories to have the proper ownerships and access permissions. Finally you copied over the startup configuration for the service called NFS as a basis for our own service.

Now I'm hoping that you are already familiar with the vi text editor, because we are going to make heavy use of it. If you have no clue how to use vi, please look up some tips on the Internet first!

kilala:~ thomas$ sudo vi StartupParameters.list

Modify the file to read as follows:

{

Description = "Firewall";

Provides = ("Firewall");

Requires = ("Resolver", "NetworkExtensions");

OrderPreference = "Late";

Messages =

{

start = "Starting custom firewall";

stop = "Stopping custom firewall";

};

}

kilala:~ thomas$ sudo mv NFS Firewall

kilala:~ thomas$ sudo vi Firewall

Modify the file to read as follows:

#!/bin/sh



##

# Setting up the Firewall rules at boot time

##

# Please note: added "FIREWALL=-YES-" entry to /etc/hostconfig



. /etc/rc.common



StartService ()

{

if [ "${FIREWALL:=-NO-}" = "-YES-" ]; then

ConsoleMessage "Adding Firewall Rules"

ipfw -f flush

exec /Library/StartupItems/Firewall/Ruleset

fi

}



StopService ()

{

ConsoleMessage "Removing all Firewall Rules"

ipfw -f flush

}



RestartService ()

{

ConsoleMessage "Removing all Firewall Rules"

ipfw -f flush

if [ "${FIREWALL:=-NO-}" = "-YES-" ]; then

ConsoleMessage "Adding Firewall Rules"

ipfw -f flush

exec /Library/StartupItems/Firewall/Ruleset

fi

}



RunService "$1"

We're almost there :) Only one more file to edit to set up the automatic booting.

kilala:~ thomas$ sudo vi /etc/hostconfig

Modify the file and add the following line at the bottom of the page:

FIREWALL=-YES-

Before setting up the rules

Now we'll get to the brunt of setting up our firewall. Most of the things discussed in this document are things that I had to learn in the course of a day, so please don't expect me to explain everything in detail ^_^; I was lucky enough to have enough past experience with iptables and ipchains, so that helped me in understanding the rules in the following chapter.

Unfortunately the rules below will only apply to people who have one network card in their system and who use a dial-up connection to the Internet. In my system the primary network card, which is used for my home network, is designated as en0. My Internet connection on the other hand is designated as ppp0. You can check your own settings by running the following command while you're connected to the Internet:

kilala:~ thomas$ ifconfig -a | grep UP

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384

en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500

ppp0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1500

The interface lo0 is your loopback interface, which is a virtual network interface not actively used on the network itself. It is mainly used for communications within your system itself. You can recognise your network card by running the command ifconfig for each of the remaining interfaces (for instance ifconfig en0); your network card will have the IP address which is also set in the System Preferences pane.

My whole point is that, if you do not have the exact same situation as I have, you will have to modify the rules below insofar that you exchange each instance of "en0" with your network card name and each instance of "ppp0" with your Internet connection name. Also, if you have more than one network card, be sure to add additional rules for those interfaces as well.

Anyway. On with the show!

Almost there: entering the rules

Now you'll have to edit the final file in this whole setup. Make sure that you're still in the /Library/StartupItems/Firewall directory before going on.

kilala:~ thomas$ sudo vi Ruleset

This will also create a new file, which you will have to fill out completely as below. Once you're more familiar with how these rules work you could start adding rules for additional services. You may notice for example that I don't open up ports for IRC or AIM, since those are both services that I make no use of.

#!/bin/sh

# Firewall ruleset for T. Sluyter (Kilala.valhalla.org)

# Ver 1.00 3rd of April 2004

#

# Allows any and all network traffic on the "inside" network.

# Blocks almost all network traffic to and from the internet.



# Allows only a limited amount of network traffic to and from the internet.

#



# Allow a number in default traffic settings

ipfw add allow ip from any to any via lo0

ipfw add allow tcp from any to any established

ipfw add allow ip from any to any frag

ipfw add allow icmp from any to any icmptype 3,4,11,12

ipfw add deny log ip from 127.0.0.0/8 to any in

ipfw add deny log ip from any to 127.0.0.0/8 in

ipfw add deny log ip from 224.0.0.0/3 to any in

ipfw add deny log tcp from any to 224.0.0.0/3 in



# Allow any and all traffic coming through en0, from local network

ipfw add allow ip from 192.168.0.0/24 to any in recv en0

ipfw add allow ip from any to 192.168.0.0/24 out xmit en0

ipfw add allow tcp from 192.168.0.0/24 to any in recv en0

ipfw add allow tcp from any to 192.168.0.0/24 out xmit en0

ipfw add allow udp from 192.168.0.0/24 to any in recv en0

ipfw add allow udp from any to 192.168.0.0/24 out xmit en0

ipfw add allow icmp from any to any in recv en0

ipfw add allow icmp from any to any out xmit en0



# Allow FTP (File transfer) to the outside

ipfw add allow tcp from any 1024-65535 to any 20-21 out xmit ppp0

ipfw add allow tcp from any 20-21 to any 1024-65535 in recv ppp0



# Allow DNS lookups to outside

ipfw add allow udp from any 1024-65535 to any 53 out xmit ppp0

ipfw add allow udp from any 53 to any 1024-65535 in recv ppp0



# Allow SSH (Secure shell) to outside

ipfw add allow tcp from any 1024-65535 to any 22 out xmit ppp0

ipfw add allow tcp from any 22 to any 1024-65535 in recv ppp0



# Allow HTTP (Web browsing) to outside

ipfw add allow tcp from any 1024-65535 to any 80 out xmit ppp0

ipfw add allow tcp from any 80 to any 1024-65535 in recv ppp0

ipfw add allow tcp from any 1024-65535 to any 8080 out xmit ppp0

ipfw add allow tcp from any 8080 to any 1024-65535 in recv ppp0



# Allow HTTPS (Secure web browsing) to outside

ipfw add allow tcp from any to any 443 out xmit ppp0

ipfw add allow tcp from any to any 1024-65535 in recv ppp0



# Allow POP (Retrieving e-mail) to outside

ipfw add allow tcp from any 1024-65535 to any 110 out xmit ppp0

ipfw add allow tcp from any 110 to any 1024-65535 in recv ppp0



# Allow SMTP (Sending e-mail) to outside

ipfw add allow tcp from any 1024-65535 to any 25 out xmit ppp0

ipfw add allow tcp from any 25 to any 1024-65535 in recv ppp0



# Allow ICMP to and from outside

ipfw add allow icmp from any to any in recv ppp0

ipfw add allow icmp from any to any out xmit ppp0



# Block all of the rest, along with logging

ipfw add deny log tcp from any to any in recv ppp0

ipfw add deny log udp from any to any in recv ppp0

ipfw add deny log ip from any to any in recv ppp0

ipfw add deny log tcp from any to any out xmit ppp0

ipfw add deny log udp from any to any out xmit ppp0

ipfw add deny log ip from any to any out xmit ppp0

Finishing touches

Before we start rebooting our systems it might be wise to first check if our startup scripts are in full working order. You never know what happens if things aren't written a hundred percent correctly ;)

Luckily Apple has provided us with a command which can be used to run a startup script as if the system was rebooting just now. Running the following command should give you a properly configured firewall.

kilala:~ thomas$ sudo SystemStarter start Firewall

Now don't worry if running this command gives you loads of errors about the ppp0 interface not being unavailable. This is of course normal if you're starting the firewall without being logged into the Internet. Like I said: don't worry! The firewall will work properly. You may check if the firewall rules are properly loaded by running:

kilala:~ thomas$ sudo ipfw list

This command should return a list of 41 rules if you followed my example to the letter. You can count them by running sudo ipfw list | wc -l. If all of this seems to work properly, you should reboot your system. Once it's restarted, run the ipfw list command again to see if the firewall came up properly.

And that's about it! ^_^ Congratulate yourself on a job well done and rest assured that you're surfing the web a little bit safer.


kilala.nl tags: , , , ,

View or add comments (curr. 0)

Older blog posts