Kilala.nl - Personal website of Tess Sluijter

Unimportant background
Login
  RSS feed

About me

Blog archives

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

> Weblog

> Sysadmin articles

> Maths teaching

Finally! Red Hat offers at-home exams

2020-09-06 21:18:00

It's been a while in coming and I'm very happy they finally made it! Red Hat have joined the large number of companies who now offer at-home test taking for their professional certifications

I quite enjoyed the way CompTIA handled their at-home examinations, but it looks like Red Hat have taken a very different approach. I still need to take the EX407 exam, so I'd better take a quick look!

Back in 2013 I was one of the first hundred people to use the Red Hat Kiosk exams, still have the souvenir key chain on my laptop bag. Let's see if their at-home tests work better than the Kiosk ones. 


kilala.nl tags: , ,

View or add comments (curr. 0)

Taking the 2020 CompTIA Cloud+ beta

2020-08-13 11:35:00

It's become a bit of a hobby of mine, to take part in CompTIA's "beta" exams: upcoming versions of their certification tests, which are given a trial-run in a limited setting. I've gone through PenTest+, Linux+ and CySA+ so far :)

After failing to get through the payment process at PearsonVue a friendly acquaintaince at CompTIA helped me get access to the Cloud+ beta (whose new version will go live sometime early next year).

I sat the beta test this morning, using the new online, at-home testing provided by PearsonVue. Generally speaking I had the experiences as outlined in the big Reddit thread.

Most importantly, on MacOS the drag-n-drop on PBQs is really slow. You have to click and hold for three seconds before dragging something. Aside from that the experience was pleasurable and it all worked well enough.

I'm not as enthused about the Cloud+ beta as I was about Linux+ and PenTest+ at the time. The questions seemed very repetitive, sometimes very predictable (if "containers" was an option, two out of three times it'd be the correct answer) and some just unimaginative (just throw four abbreviations or acronyms at the test-taker, two or three of which are clearly unrelated). Knowing CompTIA I assume there will be plenty of fine-tuning happening in the next few months.

I'm pretty sure I didn't pass this one, but I'm happy to have had the chance to take a look :)


kilala.nl tags: , ,

View or add comments (curr. 0)

Preparing for PearsonVue at home, online testing

2020-08-12 15:35:00

This Reddit thread offers a plethora of information on the at-home, online test taking offered by PearsonVue.

Big lesson I learned as MacOS user: disable Little Snitch and other filtering / security software while you're taking the test. It feels dirty, but to ensure the software does not encounter any hickups (which may result in you botching the test) you're going to have to. Better yet, don't disable, but quit the software because any popups on your screen will also alert the proctor.

Just to be safe, I made a dummy user account on my Macbook, so I can remove all trace of the software afterwards. Luckily it runs from your downloads folder and doesn't need any admin-level access.


kilala.nl tags: , ,

View or add comments (curr. 0)

Teaching helps you break habits

2020-08-09 19:39:00

It's hilarious how stuck in one's ways one can get. I mean, I've always typed:

netstat -a | grep LISTEN | grep ^tcp

While prepping slides for my students, imagine my mirth when I learned "there's a flag for that". Man, it pays to read man-pages. 

netstat -l4
ss -l4

#EternalNewbie 💖


kilala.nl tags: , ,

View or add comments (curr. 0)

Expanding my homelab: more 11th gen Dell

2020-08-01 20:21:00

R410 and R710

The Dell R410 in my homelab has served me very well so far! With a little upgrade of its memory it's run 20 VMs without any hassle. Finding this particular configuration when I did (at a refurbishing company) was a lucky strike: a decent price for a good pair of Xeons and two large disks. 

I've been wanting to expand my homelab, to mess around with vMotion, Veeam and other cool stuff. Add in the fact that I'd love to offer "my" students a chance to work with "real" virtualization (using my smaller R410) and you've got me scouring various sources for a somewhat bigger piece of kit. After trying a Troostwijk auction and poking multiple refurbishers I struck gold on the Tweakers.net classified ads! 

Pictured above is my new Dell R710, the slightly beefier sister of the R410. It has space for more RAM, for more disk drives and most importantly (for my own sanity): it's a 2U box with larger fans which produces a lot less noise than the R410. The seller even included the original X5550 CPUs seperately.

So! From the get-go I decided to Frankenstein the two boxes, so I could actually put the R410 to use for my students while keeping a bit more performance in my homelab. 

Moving that RAID1 set from the R410 to the R710 was an exciting exercise!

I really did not want to loose all of my VMs and homelab; I've put a year into the environment so far! Officially and ideally, I would setup VMware ESXi on the R710 and then migrate the VMs to the new host. There are many methods:

Couldn't I do it even faster? Well sure, but you can't simply move RAID sets between servers! Most importantly: you'll need similar or the same RAID controllers. In a very lucky break, both the R410 and the R710 have the Dell/LSI Perc 6i. So, on a wish and a prayer, I pludged the RAID set and told the receiving Perc 6i to import foreign configuration. And it worked! 

After booting ESXi from the SD card, it did not show any of the actual data which was a not-so-fun surprise. Turns out that one manual re-mount of the VMFS file system did the trick! All 24 VMs would boot!

So far she's a beaut! Now, onwards, to prep the R410 for my students.


kilala.nl tags: , ,

View or add comments (curr. 0)

CTT+ certification achieved!

2020-07-10 13:45:00

It's official! After passing the theoretical exam in June and completing the practical, virtual classroom assessment this week, I'm now officially CTT+ certified: CompTIA CTT+ Virtual Classroom Trainer Certification.

Many thanks to the people who supported me; you know who you are! 💝


kilala.nl tags: ,

View or add comments (curr. 0)

Dell 11G (11th generation) server firmware updates in 2020

2020-06-13 22:20:00

Update:

One Reddit user suggests that, while my suggested way of working is easier than others, it may also lead to "bricking" of servers: literally rendering them unusable, by applying firmware updates out of order. 

Their suggestion is to instead use the SUU (Server Update Utility) ISO image for the server in question, which may be run either from a booted Windows OS, or through the LCM (Life Cycle Manager). 

More information about the SUU can be found here at Dell.

Also, if you take a look at Dell's instruction video about using the SUU ISO from the LCM, I think we can all agree that this in fact the easiest method bar none. 

EDIT: If it weren't for the fact that the old LCM firmware on the R410 cannot read the SUU files. So you have to use this with Windows or CentOS.

 

TLDR:

If you want to skip all the blah-blah:

 

Introduction

Early in 2019 I purchase a Dell R410, part of Dell's eleventh generation (11G) server line-up from 2010/2011. Since then I've had a lot of fun growing and maintaining my homelab, learning things like Ansible and staying in touch with Linux and Windows administration. 

One task system administrators commonly perform, is the upgrading of firmware: the software that's built into hardware to make it work. If you check out the list of available firmware options for the R410, you'll see that quite a lot of that stuff goes into one simple server. Imagine what it's like to maintain all of that stuff for a whole rack, let alone a data center full of those things!

In the case of the R410, support options from Dell are slipping. While many homelabs (and some enterprises) still rock these now-aging servers, the vendor is slowly decreasing their active support.

In my homelab I have tackled only a small number of firmware updates and I'll quickly discuss the best/easiest way to tackle each. In some cases it took me days of trying to figure them out!

 

A note about Dell's Life Cycle Manager (LCM)

Dell's 11G systems (and later) include the Life Cycle Manager (LMC) which makes firmware updates a lot easier. You reboot your server into the USC (Unified System Configurator), launch the updater and pick the desired firmware updates.

Here's a demo on YouTube.

Unfortunately, the bad news is that somewhere in 2018 Dell dropped the 11G updates from their "catalogs". You can still use the following steps to make your 11G system check for updates, but it won't find any. You can check the catalogs yourself at https://ftp.dell.com/catalog/. Mind you, based on this forum thread, the Dell ftp/downloads site hasn't been without issues over the years.

  1. Boot your server and press F10 to launch System Services.
  2. In the menu, choose USC Settings (or whichever option lets you configure networking). By default USC will not retain its network configuration, or properly start the NIC, so you have to run this configuration each time.
  3. After configuring the network access, go back to the USC menu and choose to Launch the updater
  4. Apply the following settings:
    Server = ftp.dell.com
    Username =
    Password =
    Catalog path = /catalog/
    Proxy =
  5. If you now start the update process, the system will fetch and verify the catalog after which it will throw the following error.
"No update is available. Make sure that the Windows(R) catalog and Dell(TM) Update Packages for Windows(R) are used."

There are no more updates for 11G systems available for LCM.

 

A note about Dell Repository Manager

Technically it's possible to make your own internal clone of Dell's software update site. For a large enterprise, that's a great idea actually! Dell's recommended way of setting up a mirror to host updates for your specific systems, is to use the Repository Manager (DRM).

You could also use DRM to create a bootable USB stick that contains the updates you want, so the system can go and update itself, using LCM. Great stuff!

But you're still going to run into the same issue we discussed in the previous paragraph: 11G updates are no longer available through the catalogued repository. You can only get them from the Dell support site, as per below.

So for 11G, forget about DRM. For anything besides the iDRAC, you will need to boot an OS to update your firmware.

 

iDRAC6 update

Updating the iDRAC integrated management system (if you have it) is the easiest task, assuming that you have the full Enterprise kit with the web GUI. 

  1. Visit Dell's support site for your hardware, like here for the R410
  2. Download what is labeled as the latest "Dell iDRAC monolithic release".
  3. The downloaded file is a .exe self-extracting ZIP file. If you open this ZIP file, you will find a file with extension .d6 in there. 
  4. Visit your iDRAC6 web GUI and choose Update Firmware from the Quick Launch Tasks list. 
  5. Upload the .d6 file we extracted and let the iDRAC do its magic. 

 

Booting an OS to perform updates: BIOS and LCM

My R410 runs VMware ESXi which, while it's a Unix, is not supported to run Dell's firmware updates from. Dell support a plethora of Windows versions, a few other OSes and (for the 11G systems) RHEL 5 or 6 (Red Hat Enterprise Linux). 

I first wanted to try CentOS 6 (a RHEL 6 derivative), because that's an OS I'm quite comfortable with. I grabbed an ISO for CentOS 6 Live, used dd to chuck it onto a USB stick and booted the OS. Running the BIOS and LCM updates worked fine.

  1. On the Dell support site for R410, make sure to choose "Red Hat Enterprise Linux 6" as the target OS.
  2. Then grab the "Dell Server BIOS PowerEdge R410 Version 1.14.0" and "Dell Lifecycle Controller v1.7.5" downloads.
  3. You'll get a .BIN file, which is a shell script including binary content. Basically the Linux equivalent of a self-extracting ZIP. 
  4. Put these .BIN files on another USB stick, or download them using the browser on the CentOS live OS. 
  5. From a terminal, literally run the .BIN file as you would a shell script. It'll do what you need, or maybe throw an error or two that should be easily solved.

However, the BMC update proved to be quite a mess! In the .BIN package you'll find a rat's nest of shell scripts and binaries which have dependencies not available by default on the CentOS 6 live DVD (like procmail and a bunch of older C libraries). I tried fighting my way through all the errors, manually tweaking the code, but finally decided against it. There has to be an easier way!

 

Booting an OS to perform updates: BMC

Thanks to a forum thread at Dell, I learned that there is in fact an easier way. Instead of fighting with these odd Linux packages, let's go back to good ol' trusted DOS! 

FreeDOS that is!

I learned that booting FreeDOS from a USB stick on the R410 is problematic. In my case: it's a no-go. So I took FreeDOS 1.3 and burned their Live CD to a literal CD-ROM. Stuck that in the R410's DVD drive and it boots like a charm!

While FreeDOS does not have USB drivers, there is some magic in the underlying boot loaders that will mount any USB drives attached to the system during boot-time. The USB stick I put in the back USB port was made available to me as C:, while the booted CD-ROM was R:.

What do you put on that USB stick? The contents of the PER410*.exe files available from Dell's support site. Each of these is yet another self-extracting ZIP file, containing all the needed tools for the update. 

After removing the two iDRAC modules (read below) and getting the correct update (see below also), I followed the instructions from Dell's support team in that forum thread,  extracted the ZIP file onto the USB stick, booted FreeDOS and ran "bmcfwud". The system needed a reboot and a second run of bmcfwud. And presto! My BMC was updated!

 

A note about BMC and iDRAC

BMC stands for Baseboard Management Controller. It's Dell's integrated IPMI-based management system, which is literally integrated into the motherboard of the 11G systems. It'll let you do some basic remote management. The most important reason for homelab admins to consider updating BMC is to get version >=1.33 which greatly decreases fan noise

BMC was superceded by iDRAC (integrated Dell Remote Access Controller), which offers cool features like SSH access, a web GUI and much, much more features! Here's a short discussion about it.

For all intents and purposes iDRAC replaces BMC. If you have an iDRAC installed, the BMC will not be active on your 11G system. The fan noise issues on the R410 should be fixed with any recent version of the iDRAC firmware.

So why did I want to update the BMC firmware? 

Because I'm stubborn. =)

Initially, running the updater failed because it said my BMC was at version 2.92. Well, that's impossible!

Turns out, that's because I still had the iDRAC in there! :D I removed both iDRAC daughter cards and tried again. 

A downgrade? While I grabbed the most recent BMC update from Dell's site?! No thank you !

So, funny story: Dell's support site for the R410 states that the most recent available version for BMC's firmware is 1.15. The poweredgec.com site for 11G also confirms this. But if you manually search for them, you'll find newer versions:

Apparently my BMC already had 1.54, so it already had the fan updates from 1.33. Guess all the noise that thing was making was "normal". Anyway, grabbing the 1.70 update and running bmcfwud finally had the desired end result. 


kilala.nl tags: , ,

View or add comments (curr. 0)

Ballet at La Bayadère in Almere

2020-01-09 20:42:00

I am very grateful towards my classmates and our teacher Lyda de Groodt, at ballet school La Bayadère in Almere (Facebook). They've always been awesome to our daughter Dana, to my wife (when she still trained there) and now to me. 

As they say: "Don't judge a book by its cover". Sure, La Bayadère isn't a big brand-name corporation and no, they don't have some fancy modern studio. But they provide quality: personal guidance, a keen eye, discipline and of course a bit of fun! The students with whom I've trained want to learn proper ballet, but it's clear that we also do it for our own enjoyment. Today, I remarked to S. that "what I really love about this group, are all the smiles and laughs". Our group isn't just focused on rigorous dance, we also connect a bit on a personal level. 

I'm really happy to be training with people like T., Q., A. and I.: they never fail to make me feel like I'm twenty years younger again! ^_^

Ballet certainly is different from what you're used to seeing from me, after a sports hiatus of three years and four years of hard-hitting kendo. But I really, really enjoy it. Now that I think of it, it's funny that I haven't started writing about it earlier, seeing how voracious I was about kendo-blogging.


kilala.nl tags: , ,

View or add comments (curr. 0)

Older blog posts