Massive Cryptomining Campaign Targeting WordPress Sites
On Monday we wrote about the massive spike in brute force attacks on WordPress sites that we observed. As reported, it was the most intense period of attacks we had ever recorded. We believe that a single botnet is behind the attacks.
We were able to isolate the IP addresses from the botnet and then compare them to the IPs from our most recent site cleaning orders. As luck would have it, we got a couple of hits. This afforded us the amazing opportunity to dig in and find out what the attacker is up to, and what we found is really interesting.
Note: The following section is a very technical deep dive into what we have learned about what the attacker is up to. Otherwise, feel free to jump straight to the ‘findings summary’ section below.
What the Attack Is Up To
Most of our investigation came from one site cleaning case. It started when our customer’s hosting company received an abuse complaint, including logs of failed WordPress login attempts from the customer’s server. The server had performed enough attacks to land on our IP blacklist, with over 100,000 attempts on Monday alone.
The server was a managed VPS. The customer was able to give us root access, which allowed us to do some deeper analysis of the processes running on the server. We started our investigation and found quite a mess.
The server’s CPU resources were being consumed by long-running Apache processes, as well as one strange process named “29473” using more resources than everything else.
We also observed thousands of connections from the server going out to port 80 on other servers:
In other words, our customer’s server (the IP address starting with 172) was connecting out to thousands of other web servers.
We also found that “29473” was holding connections open to two IP addresses:
- 220.127.116.11 on port 9090. This Canadian IP address belongs to OVH, a cloud computing based in France. It does not appear to have a domain name associated with it, nor any historical domain name data. We scanned it and found only two ports open: one running SSH and port 9090 apparently running an IRC server.
- 18.104.22.168 on port 8080. This IP address belongs to a network named “Makonix SIA” in Latvia. It didn’t have any domains associated with it either. Our network scan found several ports open. One was an SSH server, and the rest seemed to be web servers which answered all requests with the text “Mining Proxy Online.”
At this point we know the broad strokes of the attacker’s activity on this server. Communications with an IRC server are likely to be command and control (C&C). A process which has consumed enormous amounts of processing power and communicating with a “mining proxy” has to be a cryptocurrency miner, almost certainly for Monero, since it can be mined using regular processors instead of graphics processors. And connections to other web servers are likely to be the WordPress brute force attacks that we know are originating from this server.
The rest of the investigation revealed even more details.
Command & Control
We used tcpdump to record network traffic while we collected the files and dumped memory of running processes. We were able to capture quite a bit of C&C traffic, and since it’s unencrypted IRC traffic, we investigated it to further understand what the attackers are doing. Based on the traffic and analysis of some samples we recovered, the malware appears to be a variant of “Tsunami” or “Kaiten.”
We identified eight C&C servers, all running the IRC daemon on port 8080 or 9090. Each one had a name that followed a pattern, for example the first four servers in the list are all hosted at OVH, and their names are muhstik.ovh1 through muhstik.ovh4.
- 22.214.171.124:9090 muhstik.ovh1
- 126.96.36.199:9090 muhstik.ovh2
- 188.8.131.52:9090 muhstik.ovh3
- 184.108.40.206:9090 muhstik.ovh4
- 220.127.116.11:8080 x.1
- 18.104.22.168:8080 x.2
- 22.214.171.124:8080 x.4
- 126.96.36.199:9090 muhstik.ras1
The command protocol is fairly straightforward. The malware joins the IRC server and sets its username to a string that includes some information about the server on which it’s running. Below is a screenshot from one of our packet captures showing the hacked server’s traffic in red, and the C&C server’s traffic in blue. In that example, the nickname includes “x86” (showing that it’s not a 64-bit server) and the hostname (which we have redacted). The malware receives instructions via private messages from other bots or users.
The server seems to be lax about connections; it seems the only authentication it requires is to follow the right format when joining and setting the nickname, responding to messages, etc.
The majority of the commands coming from the attacker were like the ones in the screenshot – download a script from some server, and then run it silently. The commands are sent at regular intervals, and cycle through a few different methods of downloading the script (wget, curl, etc.). It seems these commands are just sent automatically, probably to make sure the malicious script gets restarted if it happens to crash or be terminated.
We saw a few other commands meant to gather information about the compromised server. Some of them seemed like automated status checks, but we did notice some manual activity. At one point, the attacker sent the command “iptime” and then a moment later sent the correct spelling, “uptime.”
The attacker seems to have compiled their own version of the cryptomining software, but we did see a few instances where they sent commands to manually run cryptominers – more on that below.
Malware Behavior/ Persistence
This malware was not a rootkit – it runs as a regular user account, thankfully. It still tries to be as stealthy as possible. When it starts, it spawns a copy of itself but with a different name, probably chosen at random from files around the server. For example, we mentioned it running under the name “29473,” but we also observed it as “python” and several other common programs.
We found several different variations of the malware. Most of them were designed so that when they’re started up, they delete their own file from the disk. That way, antivirus software won’t identify them (unless it scans programs in memory as well).
For persistence, the malware installed itself as a cron job to run every minute:
* * * * * /var/www/vhosts/[redacted].com/wp-content/plugins/bash > /dev/null 2>&1 &
It also listened for connections on high TCP ports (e.g. 61008 and 63008), but we didn’t observe any traffic to those ports.
Brute Force Attacks
Of course, the malware is also responsible for the brute force attacks. Based on our observations, it uses a combination of common password lists and heuristics based on the domain name and contents of the site that it attacks – including names, usernames, and words. For example, on wordfence.com, it would attempt usernames like “admin” and passwords like “123456.” But it would also attempt “wordfence,” “mark,” and so on.
We also observed it attacking sites running on non-standard ports, using only an IP address rather than a domain name – so don’t think your site is safe from attack just because it’s hidden away somewhere.
In fact, that’s how they compromised our customer’s server.
Their dev site was hosted on the same server with a couple of their production sites, and that compromise started everything. The customer hadn’t installed Wordfence on the dev site; if they had, it could have prevented or at least alerted on the administrator login, and even our free signatures detect the backdoors that the attackers added.
The attackers also infected most of the sites’ PHP files with a single, long line:
Wordfence would have detected this as well.
Some of the malware samples contained the Monero mining software XMRig. In most cases, the attacker configured it to run through one of several proxies, so we don’t know the wallet address associated with the miners. But in a few instances, the attacker manually ran mining commands pointed at pool.supportxmr.com, and included the wallet address. The two addresses we observed were 45Fj1P2s9LiVEVoW4p81cSKP5og6GSF3m9YUQc51o6KzXw1ByufNoTa88NEWBeE7dtjRZRCDj3Ly4a95by6sfzP3UmX3741 and 4ADnikPPkTpD39LunWcMA136o2m2uwnEhheKNmfQPv5kAFsQaxr2VsLeit5GEPdEkd9TxnAkzinWhK8LUFzxmTuc5rT1YDK. You can enter these at supportxmr.com and see their recent performance statistics as well as their payout history:
Suddenly the reason for the frenzied brute-force attacks becomes very clear. At the beginning of this month, the price of Monero had barely broken $200. But its value has since skyrocketed, reaching $378 the day before the attacks started. Monero is designed so that it can be mined by regular CPUs, but that’s still not easy. Even for a hacker using compromised servers, the return on mining wasn’t that great – until recently.
These two addresses, which surely represent only a fraction of this attacker’s mining power, together have received about 217 XMR. At the time of this post, that’s worth almost $100,000. The attacker has decided to go all-in on mining using compromised servers, and he’s trying to compromise as many servers as he can.
To summarize, the attacker is leveraging sophisticated malware to control compromised WordPress servers remotely. The servers are being used to both attack other WordPress sites and to mine for Monero, a cryptocurrency that can be efficiently mined using web server hardware. We discovered evidence showing that the attacker has earned almost $100,000 from mining already, and likely quite a lot more.
An Update on Attack Volumes
Since we initially reported this new brute force attack campaign on Monday, the attack volumes we are seeing have been extremely volatile. We now know that the attacker is using compromised WordPress sites to both launch attacks and mine cryptocurrency, so we theorize that they’re tweaking the resource allocation between the two tasks.
Today in the early morning hours UTC (early evening PST) we saw attack volumes spike again, but we were relieved to see them settle down before they could eclipse our previous peak. What’s scary, though, is that the number of attacking IPs surpassed the previous high, suggesting that the attacker’s botnet has the capacity to dial up volume beyond the peak we saw on Monday, but is choosing to use some of those resources for cryptomining.
There are a number of things you can do to make sure your site hasn’t been compromised by this attacker and to insure that it won’t be later.
- Run a Wordfence scan – the PHP malware that we found on the sites we analyzed are detected by Wordfence, including the free version.
- Check your server resources – the Monero-mining the attacker is doing will use as many CPU resources as it possibly can. If you have the ability to check your site’s resource usage, you can verify if CPU usage is within normal levels. If you have command line access to your server, you can use the utility `htop` to see which processes are using the most CPU.
- Harden your site against brute force attacks – if you haven’t already done so, we provided a list of suggestions in our post on Monday.
- Monitor blacklists – if your site is attacking other sites, it will likely be blacklisted quickly.
- Act quickly if compromised – If you have been infected, you will want to clean your site immediately, as your domain and IP address reputations are going to be damaged very quickly if your site is being used to attack other sites.
We will keep a close eye on this and publish updates as the story develops.
The PHP file you found (php_infection.png) is the infamous 'EITest' script.
I have seen a few of these you may want to block access to *.crypto-pool.fr
I have a feeling this is just the beginning and as crypto continues to explode we are going to see an increase in attacks and their complexity. One thing they should have in common is CPU use, so definitely something to remember.
Thanks for the article and update.
One question WP, how do you make entries on a keyboard and sit on your hands at the same time?
Again, another of my comments will not pass the moderator.
Hi Greg, we are the developers of a WordPress plugin (Wordfence), not WordPress itself. You may want to visit http://wordpress.org and let them know the feedback you've been commenting our blogs with about them, since we cannot pass along your comments about their policies and practices. Best of luck!
Thanks for all the hard work you are all doing, its amazing to use your " Live Traffic" tab, amazing to see whats going on :)
In the protocol screenshot it mentions "27-10-2014" - is the script that old?
Also, (being naive....), can/will Monero trace hashes? Obviously this is ill-gotten, stealing server-resources, illegally accessing digital property etc. And shouldn't they void the coins created this way?
We saw several different timestamps in our captures. The C&C software has been around for several years, so it's possible this particualr C&C server was really running a version that old.
Monero is designed to confidential; once XMR is paid out from the mining pool, there's no way to follow it around like you can with other cryptocurrencies. The coins can't be voided, but the pool (supportxmr) could choose not to make payouts to known malicious miners.
I am a miner in supportxmr, and when i read this article, i was a bit concerned.
So i asked our Administrator about the policy in such a case. And he made very
clear that if he recieves adequate proof that an address has been used for such
illegal activities, it will in fact be banned from future payouts. If you have any further
questions, or want to report any abuse, you can contact email@example.com
or ask for @M5M400 in the chat at supportxmr.com
How do these "people" find out which websites/servers to try hacking? How do they know which are running WordPress?
Try googling "Just another Wordpress site".
"It also listened for connections on high TCP ports (e.g. 61008 and 63008), but we didn’t observe any traffic to those ports."
This makes me wonder if they are gearing up for something and getting those ports ready.
I've noticed some hits from OVH network ranges: blocked them all, most are out of our local range, we only want local traffic. I've noticed in the last few days, IP's that say they are located nearby, but they are sniffing at URL's that don't exist -like images or .png file images.
Blocked them manually. Thanks for the update and keeping us safe Mark.
Great article, and thanks for bringing those addresses to our attention. Dealing with malicious miners is our daily business and it is very exhausting. Not only is it unethical to mine on hacked equipment, the amount of additional serverload those mostly very low-hashrate miners are producing is very inconvenient. Also we get a lot of abuse complaints via our ISPs, since victims of infected boxes see the traffic going to our pool. Needless to say this costs us a lot of time and effort.
The problem is that we are very limited in ways of mitigating malicious miners. Especially if they are using proxy software like xmr-node-proxy or xmrig-proxy (both available freely on github) that basically make them indistinguishable from any other miner. Traffic comes from one IP, in one TCP session and with very high-diff shares, suggesting powerful mining rigs instead of thousands of low-hashrate workers.
We are currently also noticing a very steep increase of JS based webmining traffic on top of the traditional botnet traffic. Pretty sure that will be trending up on wordpress end as well, since it's sadly very easily implemented.
Sadly, the methods of mitigation of such malicious mining traffic are very limited. We usually resort to blocking the payouts of known malicious (or suspicious) addresses. But in most cases the traffic seizes alsmot instantly and gets redirected to a new XMR address and we're back to sqare one. Blocking Proxy IPs is also not a reliable solution, since they just switch over to a backup proxy or spin up a few new ones. Worst case they just switch to one of the many other XMR pools around.
I've talked to a few other pool operators in order to start creating a private IP reputation DB, but the amount of work to maintain it would be exorbitant. Especially considering the amount of support work we would have to do to unban wrongly flagged users again on demand.
Of course I will block payouts to the mentioned addresses (and a few other ones I identified due to being registered with the same throwaway email accounts). Feel free to get in touch at firstname.lastname@example.org if you have any additional intel or new addresses that need to be taken care of.
I've been in the cyber threat intelligence world for some time now. I think that is a very good write-up. My compliments to your CTI team!
Thanks for the insight into the extensive behind the scenes work. To piggyback on an answer to a question posed in your Monday blog post ... this is a great example of why the Premium version of WordFence has a fee associated with it.
Wow, using comprised servers to mine for BC is smart. Actually tracing and working out what they are actually doing ... Hats off to you and your team Brad.
"as a cron job to run every second"
"Every minute", you meant to say? No Cron job can be set to run every second.
Thanks Konstantin, we've updated the post.
Thanks for all the hardwork. You guys are awesome. Long live wordfence.
Amazing hard work, research, and reporting. Hats off to you guys!
I've noticed some hits of OVH Hosting.
Now on, i will check my vps server. Top comand is enough to see the strange activity?
If your web server is being used for mining at the time it will consume all available resources.
I had this happen to a site this week. Not only did it require a full rebuild from backups of site and DB but needed work with google as the infection created thousands of new pages on the site and set up links to those around the world by the looks of it. in addition as the site was running an SEO plugin that linked to google indexing of pages all these new pages were indexed to it created a nice load of negative SEO at the same time as the blackhat SEO for the spammy links.
Thanks for the articles. Always good to know what's going on
Back in November, around the 11th (basing off his last broadcast date), there was a guy on the mobile streaming app live.me, flashing tons of money and bragging about Monero (was linked to [redacted] dot org, not trying to really link the website, so delete if necessary) in his broadcasts and broadcast titles, and another broadcaster I frequent.
He got suspended not long after he gifted tons to the broadcaster. His profile on the app is [redacted] in case anyone is interesting in investigating further. Not hard to find the guy on Facebook, but there is no way this person made so much money off monero legally without pulling something like this.
Thanks for your interest. I've redacted the details - without supporting data we'd prefer not to include that information here. Thanks.
Completely understandable with taking the information out, just hope it helps.
All I'm gonna say is thank god I have my server off for maintenance right now
We are wordpress customers AND use the OVH Canada data center for our server. Did wordfence by chance notify OVH about all of this?
We did not. We currently do not notify hosting providers when one of their IPs appears on our blacklist or is involved in hacking activity. In general this would be very labor intensive for us, and the larger hosts have their own IDS systems that are triggered when a server is compromised. It takes a few days, but if one of their machines starts aggressively hacking, it is eventually shut down by the host.