Kilala.nl - Personal website of Thomas Sluyter

Unimportant background
Login
  RSS feed

About me

Blog archives

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

> Weblog

> Sysadmin articles

> Maths teaching

EX413: it's been one heck of a ride!

2017-11-01 20:39:00

2017-11-02: Updates can be found at the bottom.

Five weeks ago, I started a big challenge: pass the RedHat EX413 "certificate of excellence" in Linux server hardening. I've spent roughly sixty hours studying and seven more on the exam, but I've made it! As this post's title suggests it's been one heck of a ride!

Unfortunately, that's not just because of the hard work. 

I prepared for the exam by following Sander van Vugt's Linux Security Hardening video training, at SafariBooks Online. Sander's course focuses on both EX413 and LPI-3 303, so there was quite some material which did not apply to my specific exam. No worries, because it's always useful to repeat known information and to learn new things. Alongside Sander's course I spent a lot of time experimenting in my VM test lab and doing more research with Internet resources. Unfortunately I found Sander's course to be lacking content for one or two key areas of EX413. We have discussed the issues I had with his training and he's assured me that my feedback will find its way into a future update. Good to know. 

Taking the exam was similar to my previous RedHat Kiosk experiences. Back in 2013 I was one of the first hundred people to take a Kiosk exam in the Netherlands (still have the keychain lying around somewhere) and the overall experience is still the same. One change: instead of the workstation with cameras mounted everywhere, I had to work with a Lenovo laptop (good screen, but tiny fonts). The proctor via live chat was polite and responded quickly to my questions.

Now... I said I spent seven hours on the exam: I took it twice. 

Friday 27/10 I needed the full four hours and had not fully finished by the time my clock reached 00:00. This was due to two issues: first, Sander's course had missed one topic completely and second, I had a suspicion that one particular task was literally impossible. Leaving for home, I had a feeling that it could be a narrow "pass". A few hours later I received the verdict: 168/300 points, with 210 being the passing grade. A fail.

I was SO angry! With myself of course, because I felt that I'd messed up something horribly! I knew I hadn't done well, but I didn't expect a 56% score. I put all that anger to good use and booked a retake of the exam immediately. That weekend I spent twelve hours boning up on my problem areas and reviewing the rest.

Come Monday, I arrived at the now familiar laptop first thing in the morning. BAM! BAM! BAM! Most of the tasks I was given were hammered out in quick succession, with a few taking some time because of lengthy command runtimes. In the end I had only one task left: the one which I suspected to be impossible. 

I spoke to the proctor twice about this issue. The first time (1.5 hours into the test) I provided full details of the issue and my explanation for why the task is impossible. The proctor took it up with RedHat support and half an hour later the reply was "this is as intended and is a problem for you to solve". Now I cannot provide you with details about the task, so I'll give you an analogy instead. Task: "Here's a filled-out and signed form. And over here you will find the personnel files for a few employees. Using the signature on the form, ascertain which employee signed the form. Then use his/her personal details to set up a new file.". However, when inspecting the form, you find the signature box to be empty. Blank. There is no signature. 

After finishing all other work I spoke to the proctor again, to reiterate my wish for RedHat to step in. The reply was the same: it works as intended and complaints may be sent to certification-team@. Fine. Since I'd finished all other tasks (and rebooted at least six times along the way to ensure all my work was sound), I finished the exam assuming I'd get a passing score anyway. I felt good! I'd had a good day, banged out the exam in respectable time and I had improved upon my previous results a lot!

I took their suggestion and emailed the Cert Team about the impossible question. Both to help them improve their exams and to get a few extra points on my final score.

A few hours later I was livid.

The results were in: 190/300 points: 63%, where 70% is needed for a pass. All my improved work, with only one unfinished task, had apparently only led to 22pts increase?! And somewhere along the way RedHat says I just left >30% of my points lying around?! No fscking way. 

I sent a follow-up to my first email, politely asking RedHat to consider the impossible assignment, but also to give my exam results a review. I sincerely suspect problems with the automated scoring on my test, because for the life of me I cannot imagine where I went so horribly wrong to miss out on 30% of the full score!

This morning, twentyfour hours after my last email to the Cert Team, I get a new email from the RH Exam Results system. My -first- exam was given a passing score of 210/300. No further feedback at all, just the passing score on the first sitting. 

While I'm very happy to have gotten the EX413, this of course leaves me with some unresolved questions. All three have been fired in RedHat's direction; I hope to have some answers by the end of the week. 

 

In closing I'd like to say that, despite my bad experiences, I still value RedHat for what they do. They provide solid products (RHEL, IDM/IPA and their many other tools) and their practical exams are important to a field of work rife with simple multiple-choice questions. This is exactly why my less-than-optimal experience saddens me: it marrs the great things Redhat do!

 

Update 2017-11-02:

This morning I received an email from the Certification Team at RedHat, informing me that my report of the bugged assignment was warranted. They had made an updates to the exam which apparently had not been fully tested, allowing the problem I ran into to make it into the production exams. RedHat will be A) updating the exam to resolve the issue B) reissuing scores for other affected candidates. 


kilala.nl tags: , ,

View or add comments (curr. 1)

EX413 prep: my cheat sheet

2017-10-29 12:56:00

I used Sander van Vugt's EX413/LPI3 video training to prep for my EX413 exam and expanded upon all that information by performing additional research. All in all, I've spent roughly sixty hours over the past five weeks in order to get up to speed. Over the course, over fifty pages of notes were compiled. :)

I've extract all the really important information from my notes, to make this seven-page EX413 cheat sheet. I hope other students find it useful.

Of course, this is NO SUBSTITUTE for doing your own studying and research. Be sure to put in your time, experimenting with all the software you'll need to know. The summary is based on my own knowledge and experience, so I'm sure I've left out lots of things that other people might need to learn.


kilala.nl tags: , ,

View or add comments (curr. 0)

RHEL / CentOS / Fedora: NetworkManager or dhclient messing with network and DNS settings?

2017-10-28 08:53:00

In my test networks at home I've often run into issues with NetworkManager or dhclient messing with my network settings, most importantly the DNS configuration. Judging by the hundreds of StackExchange and other forum posts to the same effect, I'm certainly not alone. The fact that this seems like such a newbie problem just makes it all the more annoying. 

I've tried many changes, based on those forum discussions, such as:

And funnily enough, things would still be changing my /etc/resolv.conf every time networking was restarted.

Turns out that I am in fact making a RedHat-newbie mistake! I'm stuck in my old ways of manually micro-managing specific settings of a Linux box. I'm so stuck that I've forgotten my lessons from the RHCSA certification: system-config-network-tui

That tool is great at resetting your network config and overwriting it with the exact setup you want. It helps clear out any settings in odd places that might lead to the continuous mucking about with your settings. 


kilala.nl tags: , ,

View or add comments (curr. 0)

PvIB CTF 2017: pen.test event

2017-10-08 10:29:00

the scoreboard

For the third year in a row I competed in the PvIB CTF "Pen.test event", a Jeopardy-style CTF where contestants race to solve puzzles and small hacking challenges. Last year I didn't fare very well at all, but this time aroud things went great! The crowd was nice, my table companions were cool, it was great talking to Anko again and the DJ played awesome beats. I had a blast!

Around 1.5 hours into the competition I went to stretch my legs and get a drink, enjoying the fun we were having. Looking around, sipping on my cola I noticed something odd about the scoreboard! When I'd managed to grab my phonecam I'd already been surpassed by one team, but for at least a short while I'd managed to be in #4 out of the pack of 51 contestants. In the end I finished somewhere halfway , because greater minds than mine managed to keep on scoring points :)

pvib ctf scoreboard

Like before, the challenges were divided into various categories (shown above) and ranked from easy to hard, resulting in different scores per item. I finished the night with 100.000 points (3x10e3, 2x10e4, 1x10e3). I was so, so close on another 10k and 30k points which is why I stuck around until the very last minute!

Web:

  1. I let myself be fooled by the easy Web challenge for way too long. The challenge presented you with a SquirrelMail login page and the task to login and get their email. Assuming it was a veritable SquirrelMail, I assumed no easy software vulnerabilities would be found, so I resorted to password guessing. An hour before the end of the night, Anko asked me "When we start out web pen testing, what are the things you're taught first?". Me: "Well... I reckon... You mean XSS, CSRF and SQL Injection, right?" A: "Absolutely." Me: "Sonuvabitch...". Turns out it was NOT SquirrelMail, just a quick and easy SQLi exercise made to look like it. 
  2. This challenge sent you to an online calculator which would help the voting committee tally their votes, in this case a basic formula line which would return the outcome. Entering gibberish into the line would return a basic Python EVAL failure. Turns out that it was possible to run OS-commands through the EVAL calculation line, which let me list the remote files and to grab the required flag.
  3. Both this exercise and #2 were a bit slow to respond in my browser, so I turned to the Lynx text-based browser. This foregoes all CSS, which was being loaded from the Internet. This time around we were supposed to hack a voting system, to find out the vote-total for each candidate. I noticed that it was based on a JSP that got included by URL, so I downloaded it for further analysis. This code showed me that the voting process makes SOAP calls to retrieve candidates and to place a vote. I also gave me examples of the XML data needed for those soap calls. From here on out, my challenge was to find out how to get voting results instead! I haven't worked with SOAP a lot, but I know there had to be some way of querying the remote end for available procedures and commands. This is where I learned about WSDL, which gave me exactly what I needed: a description of how to request voting results. This needed a little bit more tweaking to the XML, because the candidates were identified by an MD5 hash that needed to be updated as binary data. Darn! Was this close to getting the whole challenge, but was a few minutes too late. 

Learning on the go was hella fun! I got to renew my experience with CURL calls and XML data and learned new things about SOAP. Nice!

Crypto:

  1. I'd figured out the positional encryption scheme for this challenge pretty quickly, as it was clearly based on jumping and looping through the ASCII table, based on a character's position. Despite this, I seem to have had some stupid mistake in my method, because my decrypted text was repeatededly rejected. Again, this close to cracking it, but too little too late. 
  2. We were provided with two enigmatic strings and an encrypted ZIP file. Had no idea how to proceed with this one just yet.
  3. We're provided with Python code for a home-brew crypto, as well as some sample data. Given enough time I'm sure I could have figured out the issue at hand, but in this case ${ENOUGH_TIME} would -GT 2d. So never mind ;)

Cracking crypto never was my strong point ;)

Forensics:

  1. We're given a .CRT certificate for a voting machine, which supposedly is fishing. Making it legible with the OpenSSL command line quickly shows the PvIB CTF flag.
  2. We're given a .DOCX file which was supposed to contain suspicious data. I simply used unzip to extract all the components files of the Word document and searched the various XML contents for the CTF flag. 
  3. We're given a .PNG image that supposedly contains hidden data. One ZSteg install later I have my flag. 

Fun challenges! Not too hard so far.

Misc.:

  1. A PDF file with some hidden data in it. Open the PDF with the viewer on my Kali box made it stand out as a fat blue box. Anko simply grepped for "-i pvib" through the strings-output of the PDF and fared just as well :)
  2. Oooff! I wish I'd had my wife with me! She's great at logical reasoning :) This challenge combined logic (determine whether persons A, B and C are lying or tell the truth), math (Fibonacci and Harshad numbers) and programming (because there's no plausible way of quickly solving the puzzle on paper). Seeing how I can't ever get my ideas straight with the liars/truthers, I skipped this one after about half an hour. 

What a great evening! Better yet, on the way home I managed to get on the Slam! night show and I won a DAB+ radio for our home! :D Awesome-cakes!


kilala.nl tags: , ,

View or add comments (curr. 0)

WTF HP? Your M203dn laser printer defaults to open SNMP write?!

2017-10-04 18:13:00

screenshot from the web interface

We've just bought a new laser printer, mostly for my daughter Dana's schoolwork. Installation was a snap as both Windows and MacOS have made it a fool-proof process. MacOS even gave me a button labeled "Visit printer website"! Of course that's gonna pique my interest!

Yup, the HP Laserjet Pro M203dn (as it's fully named) has a wonderfully helpful web interface! By default, there's no username or password, there's no login prompt whatsoever. Just open for everyone to browse. Which is where I stumble upon the screenshot I'm showing above. Of course the SNMP community strings default to public/public. Why not? But who in the seven hells decided to make that SNMP daemon -writable-?! That's asking for trouble!

... aside from the "no username or password on the admin panel" of course. Ye gods! O_o

Oh and of course the certificate on the https web server was not signed by HP's CA. Because of course I wouldn't want to verify that nobody messed with the firmware or the certs on the printer. 

... *checks around* Yep, HP also don't have a bug bounty program. =_=


kilala.nl tags: ,

View or add comments (curr. 1)

EX413 prep: messing with FreeIPA, Apache Directory Studio and MacOS

2017-10-01 21:44:00

Messing with FreeIPA

In preparation for my upcoming EX413 examination, I'm mucking about with FreeIPA

FreeIPA is a easy-to-setup solution for building the basis of your corporate infrastructure on Linux. It includes an LDAP server, it sets up DNS and a CA (certificate authority) and it serves as Kerberos server. Basically, it's a light version of Active Directory, but targeted at Linux networks. Of course Linux can use AD just fine, but if you don't have AD FreeIPA is the next best thing.

IPA has come a long way over the past ten years. It might still not be fully featured, but it certainly allows you to setup a centralized RBAC platform, not unlike the BoKS product range I've worked with. BoKS offers more functionality (like a password safe and the possibility to easily filter SSH subsystems like allowing SCP or SFTP only), but it's also far from free. 

I'm currently doing exactly what EX413 exams want you to be able to do: install a basic FreeIPA environment, with some users and some centralized SUDO rules. It's the latter that was giving me a little bit of a headache, because I had a hard time figuring out the service account to use for the bind action. Sander van Vugt's training video refers to the service account uid=sudo,cn=sysaccounts,dc=etc,dc=ex413,dc=local, which does not appear to exist out of the box. 

This set me off one a foxhunt that lasted 1.5 hours.

Because this is a sandbox environment, I've set up one account as both the SUDO bind user in /etc/sudo-ldap.conf and in the ADS user interface. Both now work swimmingly! I can "sudo -l" as a normal user and I can mess around the LDAP tree from the warmth and comfort of my MacOS desktop :)

EDIT:

Well I'll be a monkey's uncle! That little rascal of a UID=sudo was hiding inside LDAP all along! I guess I really did make a mistake in my initial ldappasswd command :D Well, at least I learned a thing or two!

EDIT 2:

FOUND IT! The OID I showed up top has an "s" too many! I wrote "sysaccountS", while it's supposed to be "sysaccount". Ace! That's going to make life a lot easier during the exam :)  


kilala.nl tags: , ,

View or add comments (curr. 0)

Speedrunning Redhat's EX413 exam

2017-09-21 15:16:00

booking confirmation

Over the past few weeks, I've been setting up a pen-testing coaching track for ITGilde. I'd planned my agenda for Q3/Q4/Q1 accordingly and had even accepted that my RHCSA and RHCE certifications would lapse in November. Unfortunately I couldn't get enough students together for this winter, so I'm putting the coaching track off until next spring. Huzzah, this frees up plenty of time for studying!

So... Now I'd like to try and retain my Redhat certs, for which I've worked so hard! My deadline's pretty close though, as November's right around the corner. After some investigation I concluded that the most productive way for me to retain these certs, would be through passing one of the RHCA exams. EX413, pertaining to server security, is right up my alley! So, I'll be speedrunning the EX413 studies, trying to finish it all in five weeks time!

I love a good challenge! ^_^


kilala.nl tags: , ,

View or add comments (curr. 2)

Building an on-premise Stratum-1 NTP server

2017-08-11 13:59:00

Recently I've been poking around NTP time servers with a few friends. Our goal was to create an autonomous, reliable and cheap NTP box that could act as an on-premise, in-house Stratum-1 time server. In a world filled with virtual machines that don't have their own hardware clocks, but whose applications demand very strict timekeeping, this can be a godsend.

I could write pages upon pages of what we've done, but the RPi Fatdog blog has a great article on the subject

Using just one Raspberry Pi and a reliable RTC (real-time clock) module you can create an inexpensive time server for your network. The RTC they're referring to supposedly drifts about a minute per year; still not awesome, but alright. *

This setup works well and Windows servers will happily make use of it! Linux NTP clients and other, stricter NTP software will balk at the fact that your Stratum-1 box was never synchronized with another time source. This is proven by the ntpdate command refusing to sync:

$ ntpdate timeserver
4 Mar 12:27:35 ntpdate[1258]: no server suitable for synchronization found

If you turn on the debugging output for ntpdate, you'll see an error that the reference time for the host is in 1900, which is the Epoch time for NTP. The example below shows reftime (though not in 1900):

ntpq>rv
status=0664 leap_none, sync_ntp, 6 events, event_peer/strat_chg
system="UNIX", leap=00, stratum=2, rootdelay=280.62,
rootdispersion=45.26, peer=11673, refid=128.4.1.20,
reftime=af00bb42.56111000  Fri, Jan 15 1993  4:25:38.336, poll=8,
clock=af00bbcd.8a5de000  Fri, Jan 15 1993  4:27:57.540, phase=21.147, freq=13319.46, compliance=2

The quick and easy work-around for this issue is to simply create both Stratum-1 and 2 in-house :) Have one RPi run as S-1, with 2 or 3 RPis working as S-2, that sync their time off the S-1 and who are peered among themselves. Any NTP client will then happily accept your S-2 boxes as NTP source. 

Better than nothing! And cheap to boot. 

 

*: Remi Bergsma wrote an interesting article about Raspberry Pi clock accuracy, with and without RTC.


kilala.nl tags: , ,

View or add comments (curr. 0)

Older blog posts