Comments on: Reverse engineering with a VM http://rdist.root.org/2007/04/21/reverse-engineering-with-a-vm/ Embedded security, crypto, software protection Mon, 08 Mar 2010 21:19:29 +0000 http://wordpress.com/ hourly 1 By: newsham http://rdist.root.org/2007/04/21/reverse-engineering-with-a-vm/#comment-1035 newsham Sun, 29 Apr 2007 07:19:31 +0000 http://rdist.root.org/2007/04/21/reverse-engineering-with-a-vm/#comment-1035 re: "backstep," I just came across simics' "hindsight": http://www.virtutech.com/products/simics-hindsight.html "Simics Hindsight is the first complete, general-purpose tool for reverse execution and debugging of arbitrary electronic systems." re: “backstep,” I just came across simics’ “hindsight”:
http://www.virtutech.com/products/simics-hindsight.html
“Simics Hindsight is the first complete, general-purpose tool for reverse execution and debugging of arbitrary electronic systems.”

]]> By: Nate Lawson http://rdist.root.org/2007/04/21/reverse-engineering-with-a-vm/#comment-852 Nate Lawson Tue, 24 Apr 2007 01:37:15 +0000 http://rdist.root.org/2007/04/21/reverse-engineering-with-a-vm/#comment-852 Jordan, that's great that they added that feature. I'm a little disappointed it doesn't include "backstep" capability though since they obviously have the full log of the instruction trace. It's really helpful to be able to back up the state one op at a time to do a binary search for a bug. You could do that with the VMware approach by dividing the recorded trace into smaller and smaller chunks, but it's not as easy to use for this. It's still good that they are going that direction. Jordan, that’s great that they added that feature. I’m a little disappointed it doesn’t include “backstep” capability though since they obviously have the full log of the instruction trace. It’s really helpful to be able to back up the state one op at a time to do a binary search for a bug. You could do that with the VMware approach by dividing the recorded trace into smaller and smaller chunks, but it’s not as easy to use for this. It’s still good that they are going that direction.

]]> By: Jordan Wiens http://rdist.root.org/2007/04/21/reverse-engineering-with-a-vm/#comment-842 Jordan Wiens Mon, 23 Apr 2007 18:04:26 +0000 http://rdist.root.org/2007/04/21/reverse-engineering-with-a-vm/#comment-842 Also in VMWare 6.0, Record and Replay: http://blogs.vmware.com/sherrod/2007/04/the_amazing_vm_.html Sounds pretty similar to what you're suggesting. When I first heard about it, I was pretty excited. VMWare 6.0 is impressive. Also in VMWare 6.0, Record and Replay:

http://blogs.vmware.com/sherrod/2007/04/the_amazing_vm_.html

Sounds pretty similar to what you’re suggesting. When I first heard about it, I was pretty excited. VMWare 6.0 is impressive.

]]> By: nolan http://rdist.root.org/2007/04/21/reverse-engineering-with-a-vm/#comment-841 nolan Mon, 23 Apr 2007 17:42:45 +0000 http://rdist.root.org/2007/04/21/reverse-engineering-with-a-vm/#comment-841 Nate, What you describe can work. One (and only one) VM can be linearly mapped with PA 0 being MA 0. VMware doesn't do this, nor does Xen or KVM out of the box. It would be very difficult to make this happen on a hosted VMM, as most OSes do not provide an easy way to allocate large contiguous chunks of memory, much less ones that start at PA 0. There is a company called "Neocleus" that has patches for Xen for this purpose, but I don't think they've released them publicly yet. I believe it is their intention to do so eventually, so you might try contacting Guy Zana there if you're interested. Nate,

What you describe can work. One (and only one) VM can be linearly mapped with PA 0 being MA 0.

VMware doesn’t do this, nor does Xen or KVM out of the box. It would be very difficult to make this happen on a hosted VMM, as most OSes do not provide an easy way to allocate large contiguous chunks of memory, much less ones that start at PA 0.

There is a company called “Neocleus” that has patches for Xen for this purpose, but I don’t think they’ve released them publicly yet. I believe it is their intention to do so eventually, so you might try contacting Guy Zana there if you’re interested.

]]> By: Nate Lawson http://rdist.root.org/2007/04/21/reverse-engineering-with-a-vm/#comment-825 Nate Lawson Mon, 23 Apr 2007 03:56:19 +0000 http://rdist.root.org/2007/04/21/reverse-engineering-with-a-vm/#comment-825 nolan, thanks for the explanation. It's been a while since I looked at VMware-type approaches. Is there any reason why the host OS can't reserve MAs that equal PAs requested by the guest OS? That way the DMA will proceed normally without overwriting anything. The host could even spoof an overly restrictive E820h BIOS map to convince the guest OS not to use anything other than a strict range of PAs. What do you think? Remember, the goal is to provide full device passthrough without modifying the guest OS. It doesn't matter if the host OS has to reserve 90% of its RAM or open itself up security-wise during the reverse engineering. nolan, thanks for the explanation. It’s been a while since I looked at VMware-type approaches. Is there any reason why the host OS can’t reserve MAs that equal PAs requested by the guest OS? That way the DMA will proceed normally without overwriting anything. The host could even spoof an overly restrictive E820h BIOS map to convince the guest OS not to use anything other than a strict range of PAs.

What do you think? Remember, the goal is to provide full device passthrough without modifying the guest OS. It doesn’t matter if the host OS has to reserve 90% of its RAM or open itself up security-wise during the reverse engineering.

]]> By: nolan http://rdist.root.org/2007/04/21/reverse-engineering-with-a-vm/#comment-819 nolan Mon, 23 Apr 2007 01:58:59 +0000 http://rdist.root.org/2007/04/21/reverse-engineering-with-a-vm/#comment-819 Nate, When you're running an OS in a VMM, the virtual memory mapping has 3 layers instead of the usual 2. To use VMware's terminology, you have your guest kernel mapping from Virtual Addresses (VAs) to Physical Addresses (PAs). The monitor then maps the guest's PAs to Machine Addresses (MAs) which are what in the non virtualized case you would call Physical Addresses. This mapping is usually done using shadow page tables, but Intel and AMD chips coming real soon now will support the extra mapping in hardware (called Extended Page Tables and Nested Page Tables respectively). If you're more used to Xen's terminology, simply s/PA/Guest PA/g and s/MA/Host PA/g. With that background out of the way, you have a driver in your guest poking PAs into the hardware, telling it to do a DMA to or from them. Without an IOMMU to do the PA->MA mapping, the hardware will happily use the PA as an MA, and clobber whatever memory had the bad luck to be at that MA. You certainly won't get correct results; more likely you'll get total flaming death. Xen can do PCI device passthrough with paravirtualized guests without an IOMMU because it changes the guest kernel's DMA mapping functions to translate PAs to MAs. This doesn't help you with Windows, and while it works correctly, it is obviously completely insecure; any guest with direct access to a DMA capable device can simply program it directly to DMA anywhere in memory, including memory owned by other guests or the hypervisor/VMM. Nate,

When you’re running an OS in a VMM, the virtual memory mapping has 3 layers instead of the usual 2. To use VMware’s terminology, you have your guest kernel mapping from Virtual Addresses (VAs) to Physical Addresses (PAs). The monitor then maps the guest’s PAs to Machine Addresses (MAs) which are what in the non virtualized case you would call Physical Addresses. This mapping is usually done using shadow page tables, but Intel and AMD chips coming real soon now will support the extra mapping in hardware (called Extended Page Tables and Nested Page Tables respectively). If you’re more used to Xen’s terminology, simply s/PA/Guest PA/g and s/MA/Host PA/g.

With that background out of the way, you have a driver in your guest poking PAs into the hardware, telling it to do a DMA to or from them. Without an IOMMU to do the PA->MA mapping, the hardware will happily use the PA as an MA, and clobber whatever memory had the bad luck to be at that MA. You certainly won’t get correct results; more likely you’ll get total flaming death.

Xen can do PCI device passthrough with paravirtualized guests without an IOMMU because it changes the guest kernel’s DMA mapping functions to translate PAs to MAs. This doesn’t help you with Windows, and while it works correctly, it is obviously completely insecure; any guest with direct access to a DMA capable device can simply program it directly to DMA anywhere in memory, including memory owned by other guests or the hypervisor/VMM.

]]>