This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


[Xen-devel] FreeBSD boot loader on VT-x based Xen system

To: xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: [Xen-devel] FreeBSD boot loader on VT-x based Xen system
From: Randy Thelen <rthelen@xxxxxxxxxx>
Date: Thu, 25 May 2006 17:39:25 -0700
Delivery-date: Thu, 25 May 2006 17:40:03 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Folks --

I'm trying to run FreeBSD 6.0 on a VT-x based Xen system. I'm experiencing a problem where the boot loader is crashing before the FreeBSD kernel (proper) is loaded.

The sequence of operations is that the VMX domain is constructed, the BIOS is executed, GRUB loads, I type the instructions to load FreeBSD (a boot loader) and that code begins to execute and then crashes.

Here's a console log that we captured:

GNU GRUB  version 0.95  (638K lower / 261112K upper memory)
[ Minimal BASH-like line editing is supported. For the first word, TAB lists possible command completions. Anywhere else TAB lists the possible
completions of a device/filename.]

grub> root (hd0,1,a)
    Filesystem type is ufs2, partition type 0xa5

grub> kernel /boot/loader
[FreeBSD-a.out, loadaddr=0x200000, text=0x1000, data=0x32000, bss=0x0, entry =0x200000]

grub> boot
BTX loader 1.00  Starting in protected mode (base mem=9fc00)
Arguments passed (esp=67e88):
<howto=80000000 bootdev=a0300000 junk=0 0 0 bootinfo=67ec4>
Relocated bootinfo (size=48) to 9fbb8
Relocated arguments (size=18) to 9fba0
BTX version is 1.01
Relocated kernel (size=780) to 9000
Client base address is a000
Client format is ELF
text segment: offset=1000 vaddr=0 filesz=2a3b0 memsz=2a3b0
data segment: offset=2c000 vaddr=2c000 filesz=53cc memsz=b4c0
Loading complete

int=0000000d  err=00000000  efl=00010006  eip=000090ac
eax=00094c00  ebx=00202820  ecx=00000033  edx=0000a000
esi=00209701  edi=00001f98  ebp=0000a000  esp=000017f0
cs=0008  ds=0000  es=0000    fs=0020  gs=0020  ss=0010
cs:eip=ff 35 0c 90 00 00 51 51-51 51 52 b1 07 6a 00 e2
       fc 61 07 1f 0f a1 0f a9-cf fa bc 00 18 00 00 0f
ss:esp=2b 00 00 00 02 02 00 00-00 4c 09 00 33 00 00 00
       0a 69 6e 74 3d 30 30 30-30 30 30 30 64 20 20 65
BTX halted

The instructions that were being executed can be found, in a FreeBSD source tree, at:


Here they are for your perusal:

init.8:         xorl %ecx,%ecx                  # Zero
                movb $SEL_SDATA,%cl             # To 32-bit
                movw %cx,%ss                    #  stack
 * Launch user task.
                movb $SEL_TSS,%cl               # Set task
                ltr %cx                         #  register
                movl $MEM_USR,%edx              # User base address
                movzwl %ss:BDA_MEM,%eax         # Get free memory
                shll $0xa,%eax                  # To bytes
                subl $0x1000,%eax               # Less arg space
                subl %edx,%eax                  # Less base
                movb $SEL_UDATA,%cl             # User data selector
                pushl %ecx                      # Set SS
                pushl %eax                      # Set ESP
                push $0x202                     # Set flags (IF set)
                push $SEL_UCODE                 # Set CS
                pushl btx_hdr+0xc               # Set EIP

The error occurs with the last instruction shown above: pushl btx_hdr +0xc.

Here's a disassembly of those instructions:

0000907f (02) 31c9                     XOR ECX, ECX
00009081 (02) b1 10                    MOV CL, 0x10
00009083 (02) 8ed1                     MOV SS, CX
00009085 (02) b1 38                    MOV CL, 0x38
00009087 (03) 0f00d9                   LTR CX
0000908a (05) ba 00a00000              MOV EDX, 0xa000
0000908f (08) 36 0fb705 13040000       MOVZX EAX, SS:[0x413]
00009097 (03) c1e0 0a                  SHL EAX, 0xa
0000909a (05) 2d 00100000              SUB EAX, 0x1000
0000909f (02) 29d0                     SUB EAX, EDX
000090a1 (02) b1 33                    MOV CL, 0x33
000090a3 (01) 51                       PUSH ECX
000090a4 (01) 50                       PUSH EAX
000090a5 (05) 68 02020000              PUSH 0x202
000090aa (02) 6a 2b                    PUSH 0x2b
000090ac (06) ff35 0c000000            PUSH DWORD [0xc]

And, we can see that the instruction stream at 90ac matches the instruction stream in the eip dump in the exception display. (Note that I'm disassembling the btx.o file, and so the linker has not fixed up the reference to btx_hdr+0xc to be 900c, instead we're seeing just 0xc. Note too, that this explains why I believe I can replace the reference to btx_hdr+0xc with an explicit reference to the code segment register; see below.)

My guess is that there's a bug in VMXAssist. I'd like some corroboration of that point, if possible. In fact, because the code had long jumped to the above code sequence to enter 32bit mode, are we still in VMXAssist? Or have we entered into pure execution mode?

It appears that the problem is based on an incorrect data segment register value. The code segment register contains 8, which points to a valid code segment descriptor. But, the ds register is 0, and that's never good. How or why this code works on real hardware is undoubtedly a side effect of ... I dunno a bug, legacy, or a design that's not clear to me. At any rate, If I replace the push instruction with:

                pushl %cs:btx_hdr+0xc

referencing the code segment selector, then we get past the exception noted above and into our next problem. I'm just curious about this particular bug. Can somebody lend some insight or direction in tracking this down? I'm more than happy to do lots of leg work on tracking this down, but a little direction would go a long way to help.

-- Randy

Xen-devel mailing list

<Prev in Thread] Current Thread [Next in Thread>
  • [Xen-devel] FreeBSD boot loader on VT-x based Xen system, Randy Thelen <=