29 April 2020
In April 2020, the human rights organization Philippine Human Rights Information Center (PhilRights) and the media outlet Northern Dispatch (Nordis) from the Philippines suffered Denial of Service attacks from the very same attacker. Both organizations were reporting on policy conflicts between local government initiatives and the effort of the national government/agencies to undermine decisions of local officials in the Philippines. This report focuses on the DDoS attacks targeting Nordis.
HTTPS floods – identical fingerprint
The human rights organization PhilRights received a HTTP Flood the 24th of April 2020.
Nordis received similar attacks on the 12, 16 and 17 of April 2020. Four waves of flooding with identical signatures of the one received by PhilRights were recorded. Timestamps are expressed in EST (GMT-4), 12h ahead of Manila time (GMT+8).
12/Apr/2020 - 02 AM - 9027 requests 12/Apr/2020 - 03 AM - 25112 requests 16/Apr/2020 - 04 AM - 220003 requests 16/Apr/2020 - 05 AM - 188128 requests 16/Apr/2020 - 06 AM - 24601 requests 16/Apr/2020 - 08 AM - 193913 requests 16/Apr/2020 - 15 AM - 32043 requests 16/Apr/2020 - 16 AM - 254053 requests 16/Apr/2020 - 17 AM - 221634 requests
Despite that Nordis migrated their site to Cloudflare protection, the attacker discovered the hidden backend and flooded the server with HEAD, GET and POST requests.
190.80.188.74 – – [16/Apr/2020:08:14:44 -0400] “POST /QMjFG HTTP/1.1” 301 234 “https://68.233.33.80/%RAND%” “Mozilla
/5.0 (Windows NT 6.2; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0″
The attacks against Nordis increased from the 20th April and larger attacks were recorded on the 20-21 April and 24-28 April.
A botnet of botnets
A total of 6,000 compromised servers were involved in the attacks, most of them compromised routers (RouterOS, Mikrotik). The botnet originated in 106 different countries with large presence in Indonesia, India, Thailand, Brazil, US, China, Bangladesh, Russia and Ukraine
A vast amount of compromised machines were located in Rawafed ISP in Libya and PT Indonesia Comnets Plus in Indonesia.
The attacker was using a mix of techniques to build the botnet. Some of the compromised machines were Mikrotik routers, while other parts were Proxy Servers that can be found in open lists online as proxy DB.
The attacker also had access to a large amount of compromised machines in China and the Philippines.
Only in the Philippines alone, 20+ machines were used to flood the sites. Two of the IPs used belonged to ICT Converge, namely 121.58.246{.}247 and 120.29.124{.}131. The IPs correspond to compromised Mikrotik routers that have been active in botnets since April 2019.
Not all traffic spikes are attacks
In early May 2020, a set of web floods hit the backend of Nordis again. The floods did not get recorded into the webserver logs as they were just connection floods (SYN floods).
In order to investigate the floods, a packet capture recorder in the hosting provider of Nordis was needed. At this point, Nordis was hosted on a shared hosting platform with Host Color (US). Unfortunately, most shared hosting providers do not provide means to record raw traffic so we wrote a piece of code to keep track of all connections to the server in order to track the low level floods and determine if they were targeted or not. During 1-2 May we recorded large floods coming from:
172.65.251{.}170 (Cloudflare)
112.175.124{.}8
112.175.127{.}180
92.61.46{.}/23
193.105.146{.}0/24
We checked these floods against information we record in our traffic sensors and concluded that the floods were not targeting the website but rather using it to launch a SYN-ACK reflection attack against other victims in Korea and Lithuania.
The attacker was spoofing the victim IP addresses and flooded the full Internet (Ports 80, 443 and 53) to obtain a SYN-ACK reflection attack.
Unfortunately, these packet foods aimed to other victims were targeting all the IPs of the shared hosting and congested the website network stack.
CDN services are not enough against DDoS
After receiving the Denial of Service attacks, Nordis moved their site behind Cloudflare and later on to Deflect. Despite being behind DDoS mitigation infrastructure, between the 30th April to the 3th of May 2020, we could see how a few “zombie” bots were still hitting the unprotected backend at Host Color. For example, we could see bots from Iran, Turkey and China hitting the backend 36 hours after the main attack stopped. This is common when the bots loose connection to the Command and Control and there is no way of stopping them.
Moving the website to a CDN service without protecting the backend, is a common mistake among organizations that look for a quick way to mitigate Denial of Service attacks.
Hosting provider leaks customer data
When the attacks took place, Nordis was hosted with Host Color (US) on a shared hosting infrastructure. During the forensic investigation, we wanted to record non-HTTP traffic arriving to the backend server in Host Color. It is not uncommon that attackers flood the servers with UDP or SYN connections that leave no traces in the web logs.
The Nordis website was hosted in a Cpanel platform without “root” access, so we were not able to install any packet capture utility. Hence, we needed to find a creative way to perform low-level monitoring.
After some testing, we discovered that the Kernel /proc filesystem was readable and we could parse the results of /proc/net/tcp and /proc/net/udp and that the output was providing us with real time exiting live connections.
The /proc/net/tcp lists all listening TCP sockets, and then list all established TCP connections. A typical example looks like this:
46: 010310AC:9C4C 030310AC:1770 01 | | | | | |--> connection state | | | | |------> remote TCP port number | | | |-------------> remote IPv4 address | | |--------------------> local TCP port number | |---------------------------> local IPv4 address |----------------------------------> number of entry
What we did not expected was that we could also see the connections of all the others customers hosted in the website.
We reported the problem to Host Color as we found it strange that any customer with SSH or Terminal Cpanel access could have access to other customers’ connection data. We were informed by Host Color that this was the expected behavior (!). This behavior is surely not GDPR friendly.
Media coverage
- [6 May 2020] Nordis Clean compromised routers used to attack critical media – digital media foundation