Jonathan Brossard CTO - P1 Code Security

[email protected]

[email protected]


Virtualization : big picture Attack surface analysis Introducing the Virtual 8086 mode Practical use : Fuzzing using vm86()

Virtualization : big picture

Market shares


Virtualization : market shares

Source : Forrester Research 2009

78% of companies have production servers virtualized.

20% only have virtualized servers.

Virtualization : market shares

Source : Forrester Research 2009

VMWare is present in 98% of the companies.

Microsoft virtualization products are used by 17%.

Citrix/Xen is used by 10%.

Virtualization : Definitions


Virtualization is the name given to the simulation with higher level components, of lower level components.

NOTE: Virtualization of applications (as opposed to full Oses) is out of topic.

Virtualization : Definitions

Virtual Machine

A virtual machine (VM) is : "an efficient, isolated duplicate of a real machine".

--Gerald J. Popek and Robert P. Goldberg (1974). "Formal Requirements for Virtualizable Third Generation Architectures", Communications of the ACM.

Virtualization : Definitions


Requires the modification of the guest Oses (eg: Xen, UML, Qemu with kquemu, VMWare Workstation with VMWare Tools).

Opposed to « full virtualization ».


Virtualization : Definitions

There are two types of virtualizations : Virtual Machine Monitors (or Hypervisors) of type I and type II.

Virtualization : Definitions

Hypervisors of type I

Run on bare metal (eg: Xen, Hyper-V, VMWare ESX).

Type I Hypervisor

Virtualization : Definitions

Hypervizors of type II

Run as a process inside a host OS to virtualize guests Oses (eg: Qemu, Virtualbox, VMWare Workstation, Parallels).

Type II hypervisor

Virtualization : Definitions


Isolation of the userland part of the OS to simulate independant machines (eg: Linux-Vservers, Solaris « Zones », BSD « jails », OpenVZ under GNU/Linux).


Attack surface analysis

Privilege escalation on the


VMware Tools HGFS Local Privilege Escalation Vulnerability



Privilege escalation on the


CVE-2009-2267 « Mishandled exception on page fault in VMware » Tavis Ormandy and Julien Tinnes

Attacking other guests

Vmare workstation guest isolation weaknesses (clipboard transfer)


DoS (Host + Guests)

CVE-2007-4591 CVE-2007-4593 (bad ioctls crashing the Host+Guests)

Escape to host

Rafal Wojtczuk (Invisible things, BHUS 2008)

IDEFENSE VMware Workstation Shared Folders Directory Traversal Vulnerability (CVE-2007-1744)

(hardware level) attack



outb, outw, outl, outsb, outsw, outsl, inb, inw, inl, insb, insw, insl, outb_p, outw_p, outl_p, inb_p, inw_p, inl_p Problems: sequence, multiple ports


int ioctl(int d, int request, ...) Problems : arbitrary input size !

Introducing the Virtual 8086 mode

Introduced with Intel 386 (1985)

Introducing the Virtual 8086 mode

Intel x86 cpus support 3 modes

-Protected mode

-Real mode

-System Management Mode (SMM)

Introducing the

Virtual 8086 mode

Protected mode

This mode is the native state of the processor. Among the capabilities of protected mode is the ability to directly execute “real-address mode” 8086 software in a protected, multi-tasking environment. This feature is called virtual-8086 mode, although it is not actually a processor mode. Virtual-8086 mode is actually a protected mode attribute that can be enabled for any task.

Introducing the

Virtual 8086 mode

Real-address mode

This mode implements the programming environment of the Intel 8086 processor with extensions (such as the ability to switch to protected or system management mode). The processor is placed in real-address mode following power-up or a reset.

Introducing the

Virtual 8086 mode

System management mode (SMM)

This mode provides an operating system or executive with a transparent mechanism for implementing platform specific functions such as power management and system security. The processor enters SMM when the external SMM interrupt pin (SMI#) is activated or an SMI is received from the advanced programmable interrupt controller (APIC).

Nice things about Real

mode / Virtual 8086 mode

Direct access to hardware via interruptions !


Mov ah, 0x42 ; read sector from drive Mov ch, 0x01 ; Track

Mov cl, 0x02 ; Sector Mov dh, 0x03 ; Head

Mov dl, 0x80 ; Drive (here first HD) Mov bx, offset buff ; es:bx is destination

Int 0x13 ; hard disk operation


ax*bx*cx*dx (per interruption)

Id est: [0;65535]^4 ~ 1.8 * 10^19

=> still huge

=> much better than ioctl()'s arbitrary input length !

Introducing the Virtual 8086 mode

Putting it all together...

Introducing the

Virtual 8086 mode


The hypervisor runs under protected

mode (ring0, ring1 (!!) or ring3).

All of the guests run in protected mode.

Introducing the

Virtual 8086 mode

The kernel boots in (16b) real mode, and then switches to protected mode (32b).

The cpu normally doesn't get back to

real mode untill next reboot.