The State Registration Service caught in denial


November 5, 2017

The State Registration Service caught in denial

When the 15th of October 2017, we received the leaked information from suppermario12 via Kloop.kg, indicating that samara.kg and mls.kg were hosting a fraudulent website designed to “manage bribes and voters” in Kyrgyzstan, we were very surprised.

The initial information released was a home-made video and a brief explanation attached to it in Russian.

Qurium dedicated one full week to verify each of the facts and double checked the evidence with several threat intelligence sources. During five days, we monitored both domains and the IP address  212.112.124.142 with the hope to find any extra evidences to re-confirm the leaked information.

During the first two days of research, we have already collected Google Cache copies of the system and historical DNS records from two independent sources (RiskIQ and Fairsight Security) that were consistent with the information released in the video.

How did we re-confirm that “Samara” was hosted at a Government server?

The State Registration Service (SRS) of Kyrgyzstan has announced the result of their own internal investigation. After less than 48h, SRS has issued a press statement claiming that the websites samara.kg and mls.kg were never hosted in the IP address 212.112.124.142 and the evidence provided it is “photoshopped”. SRS is also demanding an apology for our “biased” forensic evidence.

In order to be as transparent as possible. we have decided to release one more piece of evidence, that helped us to be sure of our findings. Those familiar with reverse proxy technologies might find this evidence very straightforward.

By the time we were informed that mls.kg was hosted in the Government address 212.112.124.142, the site was already back to its original hosting location. We could obtain several DNS SOA records that indicated that indeed someone was making modifications to the mls.kg zone file.  The DNS SOA record is a “counter” that is increased with every DNS change that is publicly available.

One of the tests we performed the late evening of the 15th October (20.30 PM UTC) was to place a “web request” with the domain mls.kg against the address 212.112.124.142. The address was not returning any page but it was responding with a very distinctive “signature” that indicated that the site was “provisioned” in that server in the past and the backend server was no longer reachable. We did several tests with dozen of random domains and only mls.kg was giving back this response.

What does this mean?

It means that although the “samara” system was not longer reachable at that address, the configuration for the domain mls.kg was STILL present in the reverse proxy during the 15, 16 and 17th of October 2017.

When placing connections to the Government server with the domain mls.kg, the server that acts as a reverse proxy was responding with a delay of 60 seconds with the error code “504 Gateway Time-Out” instead of the default error code.

The server kept sending  out the error 504 until the morning of the 18th of October.

15 October 2017, 8.30 PM.  (First sample obtained)

 1 0.000000 10.18.0.10 → 212.112.124.142 TCP 76 60372 → 80 [SYN] Seq=0 Win=29200 Len=0 MSS=1460 SACK_PERM=1 TSval=448325846 TSecr=0 WS=128
 2 0.137433 212.112.124.142 → 10.18.0.10 TCP 76 80 → 60372 [SYN, ACK] Seq=0 Ack=1 Win=28560 Len=0 MSS=1352 SACK_PERM=1 TSval=224375625 TSecr=448325846 WS=128
 3 0.137469 10.18.0.10 → 212.112.124.142 TCP 68 60372 → 80 [ACK] Seq=1 Ack=1 Win=29312 Len=0 TSval=448325880 TSecr=224375625
 4 0.137790 10.18.0.10 → 212.112.124.142 HTTP 163 GET /ivm/view/user/login.xhtml HTTP/1.1  <--- Request of mls.kg domain at 212.112.124.142
 5 0.269557 212.112.124.142 → 10.18.0.10 TCP 68 80 → 60372 [ACK] Seq=1 Ack=96 Win=28672 Len=0 TSval=224375659 TSecr=448325880
 6 60.340974 212.112.124.142 → 10.18.0.10 HTTP 670 HTTP/1.1 504 Gateway Time-out (text/html)  <--- Exactly 60 seconds after
 7 60.340993 10.18.0.10 → 212.112.124.142 TCP 68 60372 → 80 [ACK] Seq=96 Ack=603 Win=30464 Len=0 TSval=448340931 TSecr=224390660
 8 60.341335 10.18.0.10 → 212.112.124.142 TCP 68 60372 → 80 [FIN, ACK] Seq=96 Ack=603 Win=30464 Len=0 TSval=448340931 TSecr=224390660
 9 60.478024 212.112.124.142 → 10.18.0.10 TCP 68 80 → 60372 [FIN, ACK] Seq=603 Ack=97 Win=28672 Len=0 TSval=224390710 TSecr=448340931
 10 60.478074 10.18.0.10 → 212.112.124.142 TCP 68 60372 → 80 [ACK] Seq=97 Ack=604 Win=30464 Len=0 TSval=448340965 TSecr=224390710

17 October 2017, 8 AM. (Site remains provisioned in the reverse-proxy)

  1   0.000000 10.0.148.1 -> 212.112.124.142 TCP 66 50740 > 80 [SYN] Seq=0 Win=29200 Len=0 MSS=1460 SACK_PERM=1 WS=1024
  2   0.099558 212.112.124.142 -> 10.0.148.1 TCP 66 80 > 50740 [SYN, ACK] Seq=0 Ack=1 Win=29200 Len=0 MSS=1380 SACK_PERM=1 WS=128
  3   0.099582 10.0.148.1 -> 212.112.124.142 TCP 54 50740 > 80 [ACK] Seq=1 Ack=1 Win=29696 Len=0
  4   0.099762 10.0.148.1 -> 212.112.124.142 HTTP 128 GET / HTTP/1.1 <--- Request of mls.kg domain at 212.112.124.142
  5   0.397974 10.0.148.1 -> 212.112.124.142 HTTP 128 [TCP Retransmission] GET / HTTP/1.1 
  6   0.502068 212.112.124.142 -> 10.0.148.1 TCP 60 80 > 50740 [ACK] Seq=1 Ack=75 Win=29312 Len=0
  7  60.501914 10.0.148.1 -> 212.112.124.142 TCP 54 [TCP Keep-Alive] 50740 > 80 [ACK] Seq=74 Ack=1 Win=29696 Len=0
  8  60.502838 212.112.124.142 -> 10.0.148.1 HTTP 613 HTTP/1.1 504 Gateway Time-out  (text/html) <--- Exactly 60 seconds after
  9  60.502860 10.0.148.1 -> 212.112.124.142 TCP 54 50740 > 80 [ACK] Seq=75 Ack=560 Win=30720 Len=0
 10  60.503178 10.0.148.1 -> 212.112.124.142 TCP 54 50740 > 80 [FIN, ACK] Seq=75 Ack=560 Win=30720 Len=0
 11  60.600896 212.112.124.142 -> 10.0.148.1 TCP 60 [TCP Dup ACK 8#1] 80 > 50740 [ACK] Seq=560 Ack=75 Win=29312 Len=0
 12  60.602592 212.112.124.142 -> 10.0.148.1 TCP 60 80 > 50740 [FIN, ACK] Seq=560 Ack=76 Win=29312 Len=0
 13  60.602610 10.0.148.1 -> 212.112.124.142 TCP 54 50740 > 80 [ACK] Seq=76 Ack=561 Win=30720 Len=0

18 October 2017, 11 AM. (Site has fully unprovisioned mls.kg)

  1   0.000000 10.0.148.1 -> 212.112.124.142 TCP 66 55544 > 80 [SYN] Seq=0 Win=29200 Len=0 MSS=1460 SACK_PERM=1 WS=1024
  2   0.158485 212.112.124.142 -> 10.0.148.1 TCP 66 80 > 55544 [SYN, ACK] Seq=0 Ack=1 Win=28800 Len=0 MSS=1380 SACK_PERM=1 WS=128
  3   0.158534 10.0.148.1 -> 212.112.124.142 TCP 54 55544 > 80 [ACK] Seq=1 Ack=1 Win=29696 Len=0
  4   0.158660 10.0.148.1 -> 212.112.124.142 HTTP 128 GET / HTTP/1.1 <--- Request of mls.kg domain at 212.112.124.142
  5   0.280302 212.112.124.142 -> 10.0.148.1 TCP 60 80 > 55544 [ACK] Seq=1 Ack=75 Win=28800 Len=0
  6   0.284020 212.112.124.142 -> 10.0.148.1 HTTP 690 HTTP/1.1 301 Moved Permanently  (text/html) <--- Default error when sites are not provisioned
  7   0.284049 10.0.148.1 -> 212.112.124.142 TCP 54 55544 > 80 [ACK] Seq=75 Ack=637 Win=30720 Len=0
  8   0.284334 10.0.148.1 -> 212.112.124.142 TCP 54 55544 > 80 [FIN, ACK] Seq=75 Ack=637 Win=30720 Len=0
  9   0.431625 212.112.124.142 -> 10.0.148.1 TCP 60 80 > 55544 [FIN, ACK] Seq=637 Ack=76 Win=28800 Len=0
 10   0.431647 10.0.148.1 -> 212.112.124.142 TCP 54 55544 > 80 [ACK] Seq=76 Ack=638 Win=30720 Len=0

Conclusion 1

These tests strongly suggest that the website mls.kg was provisioned in the address 212.112.124.142 the 15th of October. That while the backend was no longer reachable in this location, the error code provided by the server (504 Gateway Time-out) can only be explained if the domain is configured in the reverse proxy. This behavior changed the 18th of October, when new configuration changes were implemented in the reverse proxy to fully remove any old configuration.

 

The JSF2 watermark

Thanks to the Google Cache and Cached pages of samara.kg and mls.kg we had the following HTML code:

The first thing that we observed was that the page was using the code

“rel=”stylesheet” href=”/ivm/javax.faces.resource/theme.css.xhtml?ln=primefaces-aristo”

This code corresponds to a technology known as “Java Server Faces” from Oracle. This technology is not very common and not many websites run it. That prove to be a nice “watermark” to find similar websites running the technology in Kyrgyzstan. During our assessment we crawled thousands of websites trying to identify which sites used the same technology and code.

We have been running a network crawler of the top 1800 websites in Kyrgyzstan and by the end of October we found the JSF2 technology in the sites:

shailoo.srs.kg
zakupki.gov.kg
stimgrant.okmot.kg
test.mkk.gov.kg
api.srs.kg
auth.srs.kg
biometrika.srs.kg
data.srs.kg
demoprava.srs.kg
iaau.edu.kg
nomer.srs.kg
opendata.srs.kg
pobeda.srs.kg
prava.srs.kg
services.srs.kg
shailoo.srs.kg
srs.kg
iaau.edu.kg
tizme.srs.kg
www.srs.kg

As we reported in our first forensic report, we were able to rebuild the “Samara” login page using a CSS and JS from the site zakupki.gov.kg.

During our research we found just another website that fully reassambles the technology used in Samara.kg stimgrant.okmot.kg.

The website that depends on the Ministry of Finance has an application with the name “Stimgrant” developed by the company Infosystema

You can find the page here: https://stimgrant.okmot.kg/stimgrand/view/user/login.xhtml?faces-redirect=true or in the archived copy here

The following picture shows the first ten lines of code of the login page of the site samara.kg and the site zakupki.gov.kg. It is interesting to see that the code is identical with the exception that:

  • primefaces.css.xhtml?ln=primefaces Version 4 (zakupki)  Version 5.1 (samara)
  • Samara.kg uses the tag ;stage=Development

Conclusion 2

None of these forensic evidences are conclusive but all together fully support the “Samara” system exists, that was developed by someone with knowledge of JSF2 technology and that the server had access to a backend database from the address   212.112.124.142