Note to the reader: This blogpost was written “as it happened”, so it may jump around the place a bit. I’ll try clean it up somewhat before I hit publish, but I probably won’t have time to do much serious editing. Also, there is some value in showing the process, I guess. Or maybe that is just me being too lazy and/or cheap to get the copy edited by someone who can write. Whatever.
A few years ago, a friend of mine presented on the history of LD_PRELOAD based userland rootkits on Linux. I later ended up doing my final year thesis on using memory forensics to analyse this class of rootkits.
One of the rootkits mentioned in Alastairs talk was “lib__mdma.so.1”, a rootkit found in the wild. Recently it came up in discussion over something else, so I went and googled it to see if I could find any reference to it.
Better than a reference, I found a sample.
Along with what appeared to be a sample installed on some probably hacked server, there were also some IRC chatlogs from 2013 of someone who had been infected with this rootkit.
I suspect the hfscc server is hacked, as the “berandal_sym” path is used by certain “symlink shells” to expose the filesystem to a browser.
We hit it up with curl, and notice that the servers telling us the file has been there since 2014 (based on last modified header).
Naturally, our next step is to download the file and begin our analysis.
The MD5 hash of the file is: 16f09c76b455d7ac0540b015c6be4d1d, and that hash didn’t show up on VirusTotal. Interesting. So we have something “new” here.
Given its a shared object, and we pretty much suspect it to be an LD_PRELOAD based rootkit, we can look at its exported functions using
nm, and filtering a little with
From a first look, it appears to be a Jynx variant (Jynx parent?), what with the presence of functions like
I figured the best thing to get a disassembler or decompiler take a look at some of these functions.
I originally was going to use radare2 for this blog post, which I quite like, but when I ran it and told it to super auto analyse, it told me that an r2 developer would come around and do the hard work for me. After putting the kettle on and waiting a while, this did not occur.
Maybe I pressed the wrong “any” key?
So I got Ghidra out instead (because the screenshots were easier to deal with, mostly). A lot of the initial poking about was done with r2.
We will start with
drop_suid_shell_if_env_set, a screenshot of its decompilation is below.
So what this function does, is check for the existence of an environmental variable (named “hax”). If this environmental variable is set, and it is being called from a process where
euid=0, we unset the “hax” environmental variable, set a few new environmental variables, and execute a shell with its process name set to
Similar code can be found in the Jynx2 rootkit, linked here.
I believe can trigger this backdoor on the infected box with the following incantation:
hax=lol sudo, where “sudo” can be any setuid-root binary. We will test this hypothesis later when we infect a box with the rootkit for science purposes.
The next interesting function (to me, anyway) is the remote backdoor. Like Jynx2, it uses an
accept() hook. Unlike Jynx2, it seems to forego bringing a heap of OpenSSL cruft with it, and doesn’t bother encrypting the shell connection. This
drop_dup_shell function gets checked every time
accept() is called, being passed the
sockaddr of the accepted connection. If some checks pass, you get a shell. Otherwise, the socket is passed back unharmed.
We can check the
accept function to verify that
drop_dup_shell is called:
What are these checks? Well, its actually kind of simple. It checks if the connecting clients source port is between a certain range, and then waits for a password to be input. We can see this in the screenshot below. I did some light renaming of a couple of variables, and added comments for readability.
Basically, if our source port is between 2318 and 2351, and we enter the password “charm@nte!”, we get a shell. It would appear that the shell even tries launch a PTY for actually interactive hacking. We can also note that the shell process is launched with the process name
__mdma again. It also prints some welcome messages.
When we search Google for that password, we come across this blog post which mentions it as the password for a botnet’s C&C IRC: https://www.exposedbotnets.com/2011/07/7411756213irc-botnet-hosted-in-united.html
Interestingly, that botnets port number was 2319 – a port within the “magic range” of our backdoor here. I’m not saying its the same actor, but I’m saying its the same actor. If that actor happens to be you, get in touch. I’m curious how this project ended up being developed!
The next function of interest is
connecting_pty_shell_signalhandler, mostly because we were able to attribute that code with supreme levels of confidence as having been copy-pasted from somewhere else.
Look at this decompiler view.
Now compare it to this piece of code: rrs.c
Its the same code. A switch/case block of stuff to handle signals. The backdoor tool “rrs” is worth looking at, and I actually planned to write a bit about it at some point. It is genuinely neat, offering a full TTY, compiling (sometimes) on many platforms, and optionally using OpenSSL for encrypting its connections.
The next interesting bits are the installation and uninstall routines, which are pretty simple – they both open
/etc/ld.so.preload and write to them. Install writes the libraries path, uninstall wipes it out. I think.
Here is the install routine.
And here is the uninstaller.
This category of rootkits all install/uninstall the same way, though many of them rely on an external installer script to do this for them.
So far what we have is a persistent backdoor that can allow remote access and privilege escalation. It also has proper rootkit features, and we are going to get into those next, exploring them with static analysis until we get bored of that.
This rootkit can hide files, directories, network connections, and processes, like any good rootkit should. There is also a weird execve hook that does some stuff that I also need to understand fully, we will explore all these below.
We will start with the execve hook. Its too long to post inline, so you can find the decompiled source code of it in this gist.
My understanding of this clusterfuck of “if” statements is that the rootkit checks (using
strstr) if the executed command is one of a list of ones it doesn’t want to fuck with. If it is, it uninstalls itself, forks, waits for the process to finish, and reinstalls itself.
I’m guessing that this “block list” of processes to avoid infecting was found using trial and error. It includes
ldd, both of which could uncover an active infection, so this probably is a way to evade detection as well as avoid causing system instability. After all, usually people detect these rootkits because some process is breaking horribly. We will be able to try validate this finding later on, by doing dynamic analysis.
Next we will look at each of the hooks, in turn. This is not going to be very exciting, most of the hooks are pretty similar. I’d recommend looking at the Jynx2 source code for very, very similar (if not identical) hooks.
A bunch of the hooks call a function named
is_invisible, so I’ll quickly cover that first. This function acts as a kind of filter, using
strstr to compare the input (which is of type
char) to a couple of strings. If the string is either
__mdma, it returns telling the calling function that this string is, in fact, meant to be invisible.
__xstat64 hooks are virtually identical, so I’ll just show a decompilation of the
__xstat hook below. These hooks define the real
__xstat function as
old_xstat, do a call to the
drop_suid_shell_if_env_set function to trigger that backdoor, and then pass the input filename to
is_invisible returns 0, indicating that there is nothing to hide this time, the real
__xstat function is called (as
old_xstat). Otherwise, they hide the file being checked by returning
exit hook is rather simple, and doesn’t do much at all. It just acts as another way to call
drop_suid_shell_if_env_set and then calls the real exit, by the same method as other hooks – defining the real
fopen64 hooks are pretty much the same, so I will just show
fopen. These ones are a bit more interesting – they set up the “real
old_fopen64), and then do some extra checks.
In the event that
/proc/net/tcp6 are opened, they redirect to a function named
forge_proc_net_tcp in order to hide network connections.
In the event that the path starts with
/proc/ and ends with
maps, the function
forge_proc_pid_maps is called to hide entries from
forge_proc_net_tcp function is, to put it bluntly, a fucking mess. It basically checks if the remote or local port is within the magic range of ports to hide, it filters the connection from the output and returns a painfully crafted fake response.
forge_proc_pid_maps function is quite similar – it checks if the string
__mdma is in the
/proc/PID/maps output, removes it, and passes back a faked result, hiding the existence of the hook.
open hook is kind of neat, it just does some basic filtering on filenames to try avoid
wtmp to avoid getting logged.
readdir64 hooks are basically the same, so as usual, I’ll just cover
This hook checks if the path contains
/proc, passes it on to
is_invisible if so, and also checks for the presence of
ld.so.preload, returning null
dirent values if so, in order to hide these files.
Ok, now to the last two. These two are a bit fucking weird.
I’ll be honest, I have no idea what exactly
strnvis is for. So I’ll just guess its to hide a string? Anyway, this hook compares to see if the string
klikevil is in the
dst parameter. If it is, unless I’m terribly mistaken, it filters it out. I guess this is something for avoiding logging? Maybe its unfinished?
write hook is similar – it specifically filters out the string
klikevil from being written. I think? Pretty tired at this point. So instead, I’ll just be wrong and someone can correct me on twitter.
We have now somewhat exhaustively covered literally every hook in this piece of malware. Not as in depth as some would like, and there is a distinct lack of the mandatory IDA Pro Call Graphs that would show I’m a real reverse engineer (I’m not), but I think this gives us a fairly good picture of the malwares capabilities, and gives us some theories to test out when we actually infect a virtual machine with it.
Given its from 2014, my first thought was “right, lets find an Ubuntu from sometime around then and fuck it up”. So we go and get an ISO, get VirtualBox installed, and get to work.
We went and downloaded the i386 Ubuntu 14.04 ISO, and prayed to the gods that somehow, it would install without any fucking about. This was the hardest bit, as for whatever reason the install media had an exceptionally hard time installing grub.
We dropped our backdoor library in
/lib/lib__mdma.so.1 and entered this path into
/etc.ld.so.preload, and now we can test some theories out.
Firstly, lets try elevate our privileges. We run
hax=lol sudo -l and then CTRL+C, and note it drops us into a root shell.
Next, lets try the
accept() hook. I noticed that this was somewhat broken – it wasn’t playing nice with
apache2 for whatever reason, so I just spun up a
netcat listener on port 81 with
nc -lvp 81 to test it. We then connected to it using
ncat -p 2340 192.168.0.123 81 from our other host.
I notice it tries to spawn a TTY, so I tried get it working with
socat, but had no luck, so I decided to see if I could get the accept backdoor and environment backdoor working in tandem.
I also tested to see if files named
__mdma were visible – they were not! So that filtering clearly works!
Interestingly, every so often the backdoor would uninstall itself. I’m not sure why, I think I was triggering one of its “remove itself and then reinstall” conditions. I didn’t investigate this further due to lack of time.
I tried to see what would happen if I created a file containing the string
klikevil, but nothing interesting seemed to happen, so I’m not really sure whats going on there. That
strnvis hook will remain somewhat of a mystery, I guess, for now. Maybe an Actual Reverse Engineer can work it out – someone who has IDA Pro Call Graphs in their blog posts.
I then checked if the backdoor shell connection showed up in
netstat, and it was successfully filtered out. Very nice.
The final test was rebooting the VM. Does it reboot? Does the preload hook fuck up the entire thing? Does it actually persist?
After all, in the IRC logs it seemed someone was having serious issues with this rootkit fucking up their systems ability to function at all. So I rebooted the VM and waited. It seemed to be taking its sweet time coming back to life. Was it going to make it? Who knows. I went and put the kettle on.
Nope. Dear reader, the poor system was completely fucking hosed.
I managed to get it into recovery boot, mount the filesystem r/w and drop to a root shell, which allowed cleaning it up (by deleting /etc/ld.so.preload), but it appears that
plymouth was having a Big Sad caused by some part of the rootkit crashing.
This is entirely possibly down to just weird fuckery to do with compiling a shared library on one box, and running it on another. Without access to source code or the exact target host? I’ll never know if this thing doesn’t just crash all the time.
So in summary, this malware has the following features:
- Remote access using an
- Local privilege elevation using a magic-environmental variable.
- Post-reboot persistence (in theory!)
- Hides files containing the string
- Hides directories containing the string
- Hides processes containing the string
- Can hide network connections from netstat, etc.
- Userland based rootkit of the LD_PRELOAD class.
I think that just about wraps this one up, to be honest. I was considering memory dumping the VM and doing analysis in Volatility or similar, but I might do that in a future blog post if there is interest.
I also considered doing some IoC’s or something, but that could be a future blog post. Deadline has been hit on this one tbh.
The sample has been uploaded to VirusTotal here: VirusTotal