Qualys is actively tracking threats which target containers. In our recent analysis, we have identified a few docker instances executing a malware which we term as “LibMiner”. This malware has the capability to deploy and execute Cryptominer. It uses a unique technique for lateral movement across the containers as well as Linux systems, executing on unprotected Redis servers and initiating mining on them. The malware has the ability to protect its termination, thus making it impossible to gain control over it. This blog post uncovers the unique techniques and tactics used by LibMiner.
The diagram in Figure-1 below displays the complete execution flow for LibMiner. The various components of the malware are hosted on publicly accessible servers. The components include bash shell scripts, ELF files, python scripts and a configuration file. The attack works in two stages: In first stage LibMiner is installed via bash shell script “lib_tmp”, that initiates the next stage and launches cryptomining component “glib” which mines Monero cryptocurrency. LibMiner launches several other components like lib_config_t, sider_t, lib_rook_t, etc. each utilized for a specific purpose during the execution.
Figure-1: Shows the execution flow
Though the initial infection mechanism is not known, we identified a few containers having an ENTRYPOINT section set to execute a bash shell script. The ENTRYPOINT section of docker acts as an initial execution point for a docker at runtime. It executes the shell script content shown in the Figure-2 below.
Figure-2: Shell script that is executed at docker Entrypoint
This shell script schedules a CronJob to execute a set of commands after a specific time interval. It resets content in /etc/hosts file to unblock the access to the attacker’s server. Furthermore, it makes use of third-party packages like cURL or Wget to download the “lib_tmp” file from attacker’s server.
lib_tmp file is a shell script. It checks for the presence of security tools like Tencent’s Cloud Aegis and Alibaba’s Server Guard, kills the processes and removes the associated files. (Refer Figure-3)
Figure-3: Code snippet to Uninstall security tools
It updates nameserver in /etc/resolv.conf to Google’s Public DNS. It also kills the processes associated with itself and downloads the latest “lib” binary from the attacker’s server which is the LibMiner malware saved with 4 character long random name in /var/lib directory.
LibMiner is an ELF malware component which is packed using UPX packer. The routines in this binary execute in an infinite loop and fetch several files from the attacker’s server, as shown in Figure-1 above. Among these files, the non-ELF components contain the encoded content that is retrieved in the clear text form by applying Caesar Cipher with “-3” (minus 3) as its key, followed by Base64 decoding.
The section below gives insights of each of these components and details their significance and role in the course of execution of LibMiner:
lib_rook_t / lib_kill_t: Cleanup Module for LibMiner
The execution of LibMiner initiates by fetching, decoding and executing either of these bash shell scripts. The script is responsible to clean all the earlier traces of the execution of this malware. For this purpose, it accomplishes below tasks:
- Enumerate the process names and kill the processes associated with this miner
- Abort the network connections for a predefined list of IP addresses and ports
Figure-4: Decoded code in lib_rook_t and lib_kill_t
lib_boot_t: Upgradation and Persistence of LibMiner
The Boot_Monitor() routine in LibMiner downloads this component. The decoded data in this component is a shell script (Figure-5) that is responsible for downloading the latest copy of the LibMiner file, save it with a random name in /tmp folder and then execute it.
Figure-5: Decoded code snippet in lib_boot_t
This file is also copied as /etc/init.d/symcfget. Figure-6 below shows disassembly for the code that creates the service “symcfget” for persistence. This service executes file /usr/bin/symcfget.
Figure-6: Creates service “symcfget”
lib_config_t and glib: XMRIG Miner and its Configurations are Payload for LibMiner
The Down_Miner() routine in LibMiner downloads both the components. Glib is the payload XMRIG binary used to mine the Monero Cryptocurrency on the infected system. The decoded data in lib_config_t component includes the mining pool domain and the wallet address. This information is passed as parameters while executing the XMRIG binary. The miner uses the wallet address:
“41mu5eUMztFdnKBMauwjpXAY1auz9SgJ4XVR67zRCXzTG8rbo9UsnrWSCRMdynPKm5aewxHzuwxCEQ3DAH62HtJaGAHcogM” and connects to “pool[.]minexmr[.]com:443” mining pool. Total earnings associated with this wallet address for past 6 months is up to 40.41 XMR with a value of approximately US $2,146 as of January 2020.
sider_t: Lateral Movement using Redis
The Redis() routine in LibMiner downloads this component. The decoded data in this component is a python script that is responsible for targeting the container executing Redis in unprotected mode and infecting it with LibMiner.
This script uses ifconfig to get the container IP address and prepares the target IP list using the code snippet in Figure-7. The code below brute forces the last two sections in the IP address, which widens the scope of infection from docker containers exposing port 6379 to the Linux system’s exposing port 6379.
Figure-7: Code snippet to prepare target IP list
For each IP address in the IP list, it attempts to create a remote connection on port 6379, which is the default listening port for the Redis server. It remotely executes code shown in Figure-8 below once the connection is successfully established. It stores the bash shell content as a value in ‘key-value’ entry in Redis, and later configures the ‘dbfilename’ and ‘dir’ fields. The save API for RedisServer dumps the bash shell content in the ‘dbfilename’ root at path /var/spool/cron. In a similar way, two more files are dumped as /var/spool/crontabs/root and /etc/cron.d/root
Figure-8: Code snippet uses Redis server to infect docker containers
The notable point here is that the content that is dumped at the above-mentioned file paths via Redis is the same one from where our analysis has initiated for this docker threat. This is how the LibMiner intends to spread laterally and mine Cryptocurrency to make maximum gain. In our analysis and observation, the attempt to execute this dumped cron content on a remote system fails to execute as intended, due to the presence of Redis DB specific bytes.
Evasive Execution for LibMiner
ChangeName() and ChangePid() routines in “lib” are responsible to sustain the evasive behavior. Thus, it uses no special component to disguise its execution. ChangePid forks a child, exits the parent and then continues the execution with a new PID. The newly forked process executes with a different name which is selected randomly from a predefined list of filepaths shown in Figure-9 below. The filepaths are the paths of whitelisted or genuine applications.
Figure-9: List of file paths used for evasion
The evasion techniques are accompanied by a few anti-analysis techniques which collaboratively make manual identification and termination of LibMiner more difficult. A few techniques are listed below:
- File attribute is set to immutable to prevent modification or rename operations on the file.
- The timeout for root console is very low.
- The commands typed at console are not visible.
We suspect that the malware author doesn’t have significant experience with containers and might be in the process of “containerizing” and “re-weaponizing” their malware as many of the attack components and techniques are not well-designed for a containerized environment.
Delivering Cryptominers likely remains to be the favorite revenue stream for attackers. Attackers have shifted their focus to target misconfigured and unprotected docker containers and install miners to achieve gain maximum. We expect to witness the widespread use of such attacks.
Detection and Remediation
Qualys recommends the following security best practices for customers with concerns around these latest types of container attacks.
- Use QID 19994 : Redis Server Accessible Without Authentication with Qualys Vulnerability Management to detect vulnerable systems.
- Use updated packages and applications on your hosts.
- Use Redis DB in protected mode to avoid the access from external clients and apply policies to restrict root access. Qualys Policy Compliance helps you to check for such misconfigurations around open Docker hosts.
- Qualys addresses use cases for container security across the container pipeline. In particular, Qualys Container Security provides comprehensive inventory of containers via various sensors and performs security assessment of containers and run-time protection.