Proxmox & Cuckoo – a powerful combo for your home malware lab

I had some free time recently and decided to make use of one old desktop computer that was lying around. With a fairly strong HW configuration, it was an ideal target for my home malware sandbox environment that I long wanted, but never had the chance to set up. If you want to analyze malware, a sandbox for dynamic analysis is always useful. It allows you to focus on relevant parts of the behavior so you don’t waste time while debugging. Luckily, there are plenty of online sandboxes nowadays that one can use for free, but having your own can be of advantage sometimes. In this post, I will go through the process of setting up Cuckoo Sandbox running in Proxmox Virtual Environment, all on the same machine, running in your own network.

Acknowledgements

I would like to thank Scott Nusbaum from TrustedSec for publishing a very nice blog series about Cuckoo Sandbox, how to install it and how to deploy it with Proxmox. I use Scott’s code for the Proxmox machinery. Should you bump into any problems following my post, make sure to check out his for further reference. I would also like to thank Petr Beneš for his help with issues I ran into with Proxmox and Cuckoo. Make sure to visit his GitHub, there are some really amazing things there. Many thanks to Cuckoo Sandbox authors and to Proxmox Server Solutions for providing excellent open source software!

Word(s) of warning

I assume that you are reading this post because you are interested in malware analysis. Before continuing however, I have to warn you, you should always be very cautious when running malware. Although KVM/QEMU is generally considered to be safe, and there is little chance that you will encounter a sample that is able to escape from the virtual machine, it is always better to be on the safe side. This is especially true when running malware with a working internet connection. The analyzed sample in your sandbox can connect to other machines and potentially infect them (including machines in your own network). I would like to note that the network configuration presented in this post is far from ideal. I have all machines on the same network (PVE, Cuckoo host and guest VM). Generally, you do not want to do it this way. The reason, why I opted for the easiest network configuration, is that I’m a n00b in everything network related, I’m lazy … and that I happen to have a dedicated internet connection with a dedicated router that ensures my Proxmox machine has internet connection but is completely separated from my home LAN. If you still want to go on (I hope you do), please continue reading.

Installing Proxmox VE

Proxmox VE (PVE) is an open-source virtualization platform. It integrates LXC containers, KVM hypervisor and a lot more. Basically, it enables you to run different virtualized operating systems on one machine in an easy and convenient way.

Installing PVE is easy. Just follow the instructions on their web. First, download the installation ISO image. You can install from DVD or from USB stick. I chose to use the USB, because it is more convenient from my point of view and also a lot faster. If you are on Windows, I recommend to use Etcher to flash the USB from the ISO. I tried Rufus before and the installer was complaining that it could not mount the CD-ROM drive. I recommend leaving all settings on default during installation. My setup was using 2 SSD disks with 250GB capacity connected to a HW RAID controller (coupled in RAID 0). PVE also supports software raid, so if you have some SSDs to use, you can couple them. The installation process is fast, at the end, you will have to specify the network configuration, just pick an IP that will be excluded from your local DHCP pool, this will be the static IP for your PVE box. Set the root password for the box, reboot, and with a little luck, you are ready to go. Just point your browser to https://pve-box-static-ip:8006 to access the web UI. You will need the root password to log in to the web management console that enables you to create VMs and containers and pretty much manage anything you need.

Proxmox VE web management console

Setting up Ubuntu in PVE

After installing PVE, the next step is to set up Cuckoo. Proxmox is based on Debian, so there is a possibility  to install Cuckoo Sandbox directly into PVE (as described by Scott). I’ve tried this approach and I constantly bumped into problems. Some of the required packages on Debian have different names, some will cause conflicts and some do not exist or are obsolete. I found that a much better approach is to deploy an LXC container with Ubuntu (the native Linux distro for Cuckoo). This is surprisingly easy and fast to do in PVE. All you really have to do is download the LXC template for Ubuntu, create a new LXC container and configure it.

All these steps are accessible from the PVE web management console. You will get a working Ubuntu instance with almost the same performance as it would run on bare-metal machine plus you get snapshot support so that you can save your Ubuntu machine anytime and restore if something is messed up.

Downloading the Ubuntu template is simple. Just go to the local storage in your proxmox node, select Content and click on Templates.

Download Ubuntu LXC template in PVE

Deploying Ubuntu into an LXC container is just a few clicks. Just click the Create CT button in the top right corner of the web management console. It will open a wizard with several steps. You have to specify a name for the container. The ID should be unique, PVE will offer you the next free ID. You fill in the root password. Then you select a previously downloaded template (in our case, it is Ubuntu Bionic 18.04) to create the container from. You specify the disk size, memory size and number of processors cores. The network configuration might be different depending on what you want. In my case, I just added a static IPv4 address and set the same gateway as for the PVE box. Click on Finish button and you are ready to go. After you start the container, you can access the console through the browser. Log in with the specified root password and voila, you have a fully functional Ubuntu machine in less than a minute.

Deploy Ubuntu into an LXC container

This Ubuntu machine will be your Cuckoo host. Note the static IP that you’ve set during the configuration, you will need it later.

Installing Cuckoo

With Ubuntu deployed and running, you can continue with installing Cuckoo. At the time of writing this post the version of Cuckoo Sandbox is 2.0.6. The documentation for Cuckoo has a nice and detailed section on installation. I recommend to read through it if you run into any problems. The installation guide below follows the one from the documentation, I just omitted some parts that are not absolutely necessary. Start with the classic update and upgrade commands to make sure your Ubuntu instance is up-to-date.

apt-get update
apt-get upgrade

Next, install the requirements.

apt-get install python python-pip python-dev libffi-dev libssl-dev
apt-get install python-virtualenv python-setuptools
apt-get install libjpeg-dev zlib1g-dev swig
apt-get install mongodb

I (and Cuckoo authors) recommend to install PostgreSQL as it performs better than MySQL.

apt-get install postgresql libpq-dev

You can skip the installation of virtualization software as that it entirely handled by PVE. Continue with installing tcpdump that will allow Cuckoo to dump network activity during malware analysis. These packages should already be installed on your Ubuntu system.

apt-get install tcpdump
apt-get install libcap2-bin

To be able to configure tcpdump, you need to create a new user under which Cuckoo will run. It is highly recommended to run Cuckoo under a non-privileged and dedicated user account. In this post, I use the user name cuckoo.

adduser cuckoo

Now, you can configure tcpdump to run correctly under the new user.

groupadd pcap
usermod -a -G pcap cuckoo
chgrp pcap /usr/sbin/tcpdump
setcap cap_net_raw,cap_net_admin=eip /usr/sbin/tcpdump

All the above commands were executed under root so far. Now is the time to switch to the newly created cuckoo user. You will install Cuckoo under this non-privileged user as you definitely don’t want to run Cuckoo under root. It is also recommended to install Cuckoo into a virtual environment. It has a lot of advantages and is generally considered best practice. First, switch the user to cuckoo and switch to its home directory.

su cuckoo
cd $HOME

Creating a virtual environment is very easy.

virtualenv vcuckoo

Activate the virtual environment (mind the space after the period).

. vcuckoo/bin/activate

Now, you are inside the virtual environment named vcuckoo (you should see a (vcuckoo) prefix on your command line). There are some Python libraries required by Cuckoo and the machinery code for Proxmox that have to be installed separately. Start by installing M2Crypto. It requires swig that you already installed before.

pip install m2crypto

The documentation for Cuckoo asks for pip install m2crypto==0.24.0 but it failed to install for me. You will also need to install promoxer that is used by the machinery code.

pip install proxmoxer

Now, you can install Cuckoo similarly as all other Python modules.

pip install -U cuckoo

If everything went fine, you should have a correctly installed Cuckoo Sandbox instance. You should test it by running cuckoo -d. This is a very important step as it will create the Cuckoo Working Directory (CWD) for you.

Cuckoo first run creates CWD

You can exit from the virtual environment for now.

deactivate

Preparing the guest VM for Cuckoo

The guest VM for Cuckoo is where the malware will be running during the analysis. In my case, this is a Windows 7 64-bit machine. I chose Windows 7 because it is less noisy in terms of running processes and network communication than Windows 10 and the absolute majority of malware will run on this OS just fine. Most of today’s computers are already 64-bit machines so it is wise to go with x64 version of Windows to support 64-bit malware too. Preparing the VM is a delicate thing. You want to ensure that malware will run on the machine. Therefore, it is good to install all kinds of support applications and software libraries that malware can use during execution. The list can be pretty long, but for a bare minimum, make sure you install Office (an older version that supports both the old and new document formats), Adobe Reader, Chrome, Firefox, Java and all versions of Visual C++ redistributables. Cuckoo has a nice documentation about guest VM preparation. Make sure you read it.

If you want to create a new guest VM, you will probably need an installation ISO file with the given OS you would like to use. You first need to upload this ISO to PVE. This can be done by clicking on the local storage item under the proxmox node and selecting the Upload button in the Content sub-menu.

Upload ISO in PVE

Creating a new guest VM for Cuckoo is very similar to the process of deploying an LXC container. Just click on the Create VM button and follow the wizard. Make sure you add a disk big enough to install the OS and all the tools needed but it does not have to be too big (Cuckoo will fake the disk size and try to fool the malware sample). You should also add at least 2 CPU cores as some malware uses core count to check if they are running in a VM (single core machines are not very common nowadays).

Create guest VM in PVE

Start the VM and complete the OS installation. Through the rest of this post, I will assume that the guest VM uses Windows 7.

After finishing the installation, create an initial snapshot of the VM so you can return if something goes wrong. Disable automatic updates and firewall. The most important thing is to deploy the Cuckoo Agent. This agent is used by the sandbox to communicate with the VM, copy malware samples and get back the results. It is implemented in Python, so you have to install it into the VM. I recommend to obfuscate the Python installation, maybe install it into a non-standard directory path and rename the executable. Next, you have to copy the agent script to the VM. You can use Python for this. On the Ubuntu machine, navigate to the Cuckoo Working Directory (CWD, by default it is .cuckoo) and switch to agent sub-directory. Once there, run SimpleHTTPServer that comes with Python. It will share the contents of the directory.

su cuckoo
cd $HOME
cd .cuckoo/agent
python -m SimpleHTTPServer 8080

Navigate to http://cuckoo-host-ip:8080 in the guest VM and you can download the agent.py file that you need.

Download agent.py shared from CWD

Save the agent script to some random directory and rename it to agent.pyw. This will prevent a console window spawning when running it. You can register the agent to run on startup. Set a fixed IP for your VM inside Windows and restart the machine. If you haven’t registered the agent for startup, you have to run it now. Finally, create a snapshot of the machine. Note the name of the snapshot and the IP address, you will need it later.

Configuring Cuckoo

You have an Ubuntu machine with Cuckoo installed and a guest VM to run malware. The last step is to add Proxmox support to your existing Cuckoo installation and configure it according to your setup. First, create a new Proxmox user. You can do this under Datacenter -> Permissions -> Users. This user will be used from Cuckoo to authenticate with the Proxmox API that allows for controlling the guest VM. I suggest that you put this user to @pve realm instead of the default @pva. Don’t forget to specify a password for this user. You can change it anytime by clicking on the Password button.

Create new user in PVE for Cuckoo

Now, you have to assign permissions to this user. As permissions are applied individually to each VM, just select you Cuckoo guest VM and click on Permissions menu. Add the newly created user PVEVMAdmin role.

Add permissions for the new user to control Cuckoo guest VM

Next, download the proxmox archive that contains the proxmox machinery file and the proxmox configuration support for Cuckoo. Extract the archive and copy the files to your Cuckoo host machine. I recommend to use WinSCP. Just log in as the user that you installed Cuckoo under and copy the files to the home directory.

The only file that you will change (patch) in Cuckoo’s installation is config.py. This file should not change very often, but just to make sure you are not removing anything important, do a diff between the one you copied over and the one that you installed with Cuckoo. I’m assuming that the virtual environment is named vcuckoo.

diff config.py vcuckoo/lib/python2.7/site-packages/cuckoo/common/config.py

The only difference you should see is the added configuration handler for the new proxmox.conf file.

Config.py patch to handle proxmox configuration

If everything looks fine, you are ready to copy the files to the right directories. As mentioned before, config.py and proxmox.py are parts of Cuckoo’s installation. The file proxmox.conf will be copied to CWD.

cp config.py vcuckoo/lib/python2.7/site-packages/cuckoo/common
cp proxmox.py vcuckoo/lib/python2.7/site-packages/cuckoo/machinery
cp proxmox.conf .cuckoo/conf

Configure Cuckoo to use the newly added proxmox machinery. You have to edit cuckoo.conf in CWD. Fire up nano and edit the machinery field. You also have to specify the correct IP address for the result server, which is your Cuckoo host machine.

nano .cuckoo/conf/cuckoo.conf
Machinery configuration
Result server IP configuration

Enable MongoDB for Cuckoo. This is not absolutely necessary, but it will allow you to use Cuckoo’s web interface.

nano .cuckoo/conf/reporting.conf
Enable MongoDB

Add PVE and guest VM configuration to proxmox.conf. You have to set the fields marked with “TODO”. Set the Proxmox user name and password that you have created earlier. Add the IP of your Proxmox machine. For the machine configuration, specify the name of the guest VM, the snapshot name that you have created with running Cuckoo agent and the static IP of the guest VM.

nano .cuckoo/conf/proxmox.conf
Proxmox and guest VM configuration

As the last step, download the Cuckoo Community Repository that contains signatures and modules provided by Cuckoo users.

. vcuckoo/bin/activate
cuckoo community

Running Cuckoo

You can run Cuckoo in a lot of ways depending on your needs. I will only go through the basics in this post. For further details, please refer to the excellent usage documentation.

You should already know how to run Cuckoo by now. This should be the first thing you do if you want to test your configuration.

. vcuckoo/bin/activate
cuckoo

You should see some diagnostic messages from the proxmoxer module that is used by the proxmox machinery to communicate with the Proxmox API.

Running Cuckoo Sandbox

As you can see, after successful start, Cuckoo is waiting for analysis tasks. The easiest way to submit an analysis is to use the cuckoo script again. You have to open another terminal for that, as Cuckoo must actively run in your current one. You can use PuTTY or any other client you prefer. Connect to your Cuckoo host machine and log in as the user that has Cuckoo installed. Enter the virtual environment and run cuckoo again with submit argument. In the example below, I created a test executable named cuckoo-test.exe and copied it to Cuckoo host machine with WinSCP. Then, I submitted it for analysis.

. vcuckoo/bin/activate
cuckoo submit ~/cuckoo-test.exe
Submit file for analysis

Cuckoo assigns an ID for every submitted task. As this was the first task submitted, it got ID #1. You can see in the other terminal (where Cuckoo is running) that the executable was received by Cuckoo and that the analysis completed successfully.

Cuckoo analysis log

The results of the analyses are stored in CWD/storage/analyses/ID.

If you need a more convenient way to submit samples to Cuckoo and also to review the results, I definitely recommend to use Cuckoo’s web interface. The web interface needs MongoDB to be allowed, but you have already configured that. The only remaining part is to start the web interface. It will be running side-by-side with Cuckoo.

. vcuckoo/bin/activate
cuckoo web runserver 0.0.0.0:8000

If everything went fine, after navigating your browser to http://cuckoo-host-ip:8000, you should be greeted by Cuckoo’s web interface.

Closing word(s)

I hope you found this post useful. I did try to follow an optimal order of actions needed to set up Cuckoo Sandbox with Proxmox. I also tried to keep the configuration changes (compared to the default) at a minimum level. You really can tweak Cuckoo a lot. Make sure to read the documentation to learn about the available options and be very careful when working with malware. If you have any questions, feel free to drop them in the comments section below. Stay safe and enjoy your personal Cuckoo Sandbox!

3 thoughts on “Proxmox & Cuckoo – a powerful combo for your home malware lab”

  1. Hi!
    Thank you for your instructions. Mostly it worked well, but we had a couple of errors in the pytnon script proxmox.py. Could you please look into lines 165 and 197? You getting id of vm via %d, so integer is expected, but vmid actually stores a string. So we substituted %d to %s and everything worked fine.
    165:
    #log.debug(“Starting vm %s id %d”, label, vmid)
    log.debug(“Starting vm %s id %s”, label, vmid)

    197:
    #log.debug(“Stopping vm %s id %d” % ( label, vmid ) )
    log.debug(“Stopping vm %s id %s” % ( label, vmid ) )

  2. Hi,
    how did you snort the windows VM sandbox traffic, if it doesn’t pass through the LXC cointainer?
    Many thanks

    Francesco

Leave a Reply

Your email address will not be published. Required fields are marked *