it's possible.
any ideas to trace the function stack of xen guest? like "bt" command in gdb.
I did some analysis:
1. the call flow is opcode()->fetch8()->address()
2. only the printf in address() will change the behaver of crash.
3. and the crash EIP (0xD0800) is in the address() from the objdump.
4. the address() will be invoked more then 40, 000 times in one
simulation, before the crash.
5. seems there are no recursive invoking in opcode(), fetch8(), address()
6. from the output of "xen dmesg", before the crash, a instructions
sequence is simulated several times (you could check the previous
mails i send for "xen dmesg" output)
7. before the trap, the simulated instruction is "movw %ax, *0xD07FE",
and the "*0xD07FE" is just the address of address(), (you could get
the objdump output from previous mails too), so i think it's the
simulation which crash the memory of address().
On 8/8/07, Keir Fraser <keir@xxxxxxxxxxxxx> wrote:
> Stack corruption/overflow, possibly?
>
> K.
>
> On 7/8/07 17:06, "Brady Chen" <chenchp@xxxxxxxxx> wrote:
>
> > Yes, the printfs are the only changes. once I remove these prints, the
> > trap comes back, with the same EIP (D0800)
> >
> > I tried to keep the first two printfs, the trap comes with different
> > EIP(D19FD)
> > static unsigned
> > address(struct regs *regs, unsigned seg, unsigned off)
> > {
> > uint64_t gdt_phys_base;
> > unsigned long long entry;
> > unsigned seg_base, seg_limit;
> > unsigned entry_low, entry_high;
> >
> > printf("f 1\n");
> > if (seg == 0) {
> > if (mode == VM86_REAL || mode == VM86_REAL_TO_PROTECTED)
> > return off;
> > else
> > panic("segment is zero, but not in real mode!\n");
> > }
> >
> > printf("f 2\n");
> >
> > xen dmesg output:
> > (XEN) HVM3: 0x0000D71F: 0xD00:0x071F (0) opc 0x83
> > (XEN) HVM3: f 1
> > (XEN) HVM3: f 2
> > (XEN) HVM3: 0x0000D71F: 0xD00:0x071F (0) external interrupt 8
> > (XEN) HVM3: f 1
> > (XEN) HVM3: f 1
> > (XEN) HVM3: f 1
> > (XEN) HVM3: Trap (0x6) while in real mode
> > (XEN) HVM3: eax CFAE ecx 0 edx 0 ebx D75B4
> > (XEN) HVM3: esp D7564 ebp D75A0 esi 71F edi 8
> > (XEN) HVM3: trapno 6 errno 0
> > (XEN) HVM3: eip D19FD cs 10 eflags 13046
> > (XEN) HVM3: uesp CFAE uss 0
> > (XEN) HVM3: ves D4C44 vds 8 vfs 83 vgs 71F
> > (XEN) HVM3: cr0 50032 cr2 0 cr3 0 cr4 651
> > (XEN) HVM3:
> > (XEN) HVM3: Halt called from %eip 0xD037C
> >
> >
> > and the objdump shows that:
> > 000d1970 <interrupt>:
> > d1970: 55 push %ebp
> > d1971: 89 e5 mov %esp,%ebp
> > d1973: 57 push %edi
> > d1974: 89 d7 mov %edx,%edi
> > d1976: 56 push %esi
> > ....
> > d19f8: 66 89 30 mov %si,(%eax)
> > d19fb: 31 d2 xor %edx,%edx
> > d19fd: 8d 34 bd 00 00 00 00 lea 0x0(,%edi,4),%esi
> > d1a04: 81 63 30 ff fd ff ff andl $0xfffffdff,0x30(%ebx)
> > d1a0b: 89 d8 mov %ebx,%eax
> > d1a0d: 89 34 24 mov %esi,(%esp)
> >
> >
> > On 8/7/07, Keir Fraser <keir@xxxxxxxxxxxxx> wrote:
> >> Very weird. The emulations now aren't at the same address as before either
> >> (0xd4c3 rather than 0xd71b). Is the *only* difference that you added these
> >> printf()s -- is it at all possible that the guest is executing down a
> >> different path here for other reasons? If it's really down to the printf()s
> >> then I guess you'll have to shuffle/remove printf()s to get the old
> >> behaviour back.
> >>
> >> -- Keir
> >>
> >> On 7/8/07 12:35, "Brady Chen" <chenchp@xxxxxxxxx> wrote:
> >>
> >>> it's strange:
> >>> if i add these prints, i get " Unknown opcode", not "trap".
> >>> ===added printf
> >>> [root@localhost firmware]# hg diff -p vmxassist/vm86.c
> >>> diff -r 6f18f5bdeea3 tools/firmware/vmxassist/vm86.c
> >>> --- a/tools/firmware/vmxassist/vm86.c Mon Aug 06 15:33:42 2007 +0100
> >>> +++ b/tools/firmware/vmxassist/vm86.c Tue Aug 07 19:33:55 2007 +0800
> >>> @@ -40,7 +40,7 @@ static struct regs saved_rm_regs;
> >>> static struct regs saved_rm_regs;
> >>>
> >>> #ifdef DEBUG
> >>> -int traceset = 0;
> >>> +int traceset = ~0;
> >>>
> >>> char *states[] = {
> >>> "<VM86_REAL>",
> >>> @@ -128,6 +128,7 @@ address(struct regs *regs, unsigned seg,
> >>> unsigned seg_base, seg_limit;
> >>> unsigned entry_low, entry_high;
> >>>
> >>> + printf("f 1\n");
> >>> if (seg == 0) {
> >>> if (mode == VM86_REAL || mode == VM86_REAL_TO_PROTECTED)
> >>> return off;
> >>> @@ -135,12 +136,16 @@ address(struct regs *regs, unsigned seg,
> >>> panic("segment is zero, but not in real mode!\n");
> >>> }
> >>>
> >>> + printf("f 2\n");
> >>> if (mode == VM86_REAL || seg > oldctx.gdtr_limit ||
> >>> (mode == VM86_REAL_TO_PROTECTED && regs->cs == seg))
> >>> return ((seg & 0xFFFF) << 4) + off;
> >>>
> >>> + printf("f 3\n");
> >>> gdt_phys_base = guest_linear_to_phys(oldctx.gdtr_base);
> >>> + printf("f 4\n");
> >>> if (gdt_phys_base != (uint32_t)gdt_phys_base) {
> >>> + printf("f 5\n");
> >>> printf("gdt base address above 4G\n");
> >>> cpuid_addr_value(gdt_phys_base + 8 * (seg >> 3), &entry);
> >>> } else
> >>> @@ -152,14 +157,17 @@ address(struct regs *regs, unsigned seg,
> >>> seg_base = (entry_high & 0xFF000000) | ((entry >> 16) &
> >>> 0xFFFFFF);
> >>> seg_limit = (entry_high & 0xF0000) | (entry_low & 0xFFFF);
> >>>
> >>> + printf("f 6\n");
> >>> if (entry_high & 0x8000 &&
> >>> ((entry_high & 0x800000 && off >> 12 <= seg_limit) ||
> >>> (!(entry_high & 0x800000) && off <= seg_limit)))
> >>> return seg_base + off;
> >>> + printf("f 7\n");
> >>>
> >>> panic("should never reach here in function address():\n\t"
> >>> "entry=0x%08x%08x, mode=%d, seg=0x%08x,
> >>> offset=0x%08x\n",
> >>> entry_high, entry_low, mode, seg, off);
> >>> + printf("f 8\n");
> >>>
> >>> return 0;
> >>> }
> >>> @@ -286,6 +294,7 @@ fetch8(struct regs *regs)
> >>> unsigned addr = address(regs, regs->cs, MASK16(regs->eip));
> >>>
> >>> regs->eip++;
> >>> + printf("f 9\n");
> >>> return read8(addr);
> >>> }
> >>>
> >>> ===output when add many printf
> >>> (XEN) HVM12: 0x0000D4C3: 0xD00:0x04C3 (0) addr32addr32f 1
> >>> (XEN) HVM12: f 2
> >>> (XEN) HVM12: f 9
> >>> (XEN) HVM12: f 1
> >>> (XEN) HVM12: f 2
> >>> (XEN) HVM12: 0x0000D4C3: 0xD00:0x04C3 (0) data32data32f 1
> >>> (XEN) HVM12: f 2
> >>> (XEN) HVM12: f 9
> >>> (XEN) HVM12: f 1
> >>> (XEN) HVM12: f 2
> >>> (XEN) HVM12: 0x0000D4C3: 0xD00:0x04C3 (0) opc 0x83opc 0xD7704f 1
> >>> (XEN) HVM12: f 2
> >>> (XEN) HVM12: Unknown opcode at 0D00:04C3=0xD4C3
> >>> (XEN) HVM12: Halt called from %eip 0xD3B4A
> >>>
> >>> On 8/7/07, Brady Chen <chenchp@xxxxxxxxx> wrote:
> >>>> Hi, yes, it's crashed in fetch8. it's very slow after I add this print
> >>>> info.
> >>>> the main function of fetch8 seems to be address(). seems crashed in
> >>>> address().
> >>>>
> >>>> (XEN) HVM7: after write16 of movw
> >>>> (XEN) HVM7: top of opcode
> >>>> (XEN) HVM7: Before fetch8
> >>>> (XEN) HVM7: eax 7E80 ecx 2D1B edx 0 ebx
> >>>> 404E
> >>>> (XEN) HVM7: esp D76F4 ebp 1FF0 esi 7BE edi
> >>>> C37FE
> >>>> (XEN) HVM7: trapno D errno 0
> >>>> (XEN) HVM7: eip 71F cs D00 eflags 33206
> >>>> (XEN) HVM7: uesp CFB4 uss 0
> >>>> (XEN) HVM7: ves D00 vds D00 vfs 0 vgs
> >>>> 0
> >>>> (XEN) HVM7: cr0 50032 cr2 0 cr3 0 cr4
> >>>> 651
> >>>> (XEN) HVM7:
> >>>> (XEN) HVM7: Trap (0x6) while in real mode
> >>>> (XEN) HVM7: eax D00 ecx 0 edx 71F ebx
> >>>> 89
> >>>> (XEN) HVM7: esp D75E4 ebp D7630 esi D7620 edi
> >>>> D00
> >>>> (XEN) HVM7: trapno 6 errno 0
> >>>> (XEN) HVM7: eip D0800 cs 10 eflags 13046
> >>>> (XEN) HVM7: uesp 71F uss D76D4
> >>>> (XEN) HVM7: ves D7610 vds D3AB9 vfs D762C vgs
> >>>> D7644
> >>>> (XEN) HVM7: cr0 50032 cr2 0 cr3 0 cr4
> >>>> 651
> >>>> (XEN) HVM7:
> >>>> (XEN) HVM7: 0xd0800 is 0xFFFF
> >>>> (XEN) HVM7: 0xd0804 is 0x7D8B
> >>>> (XEN) HVM7: Halt called from %eip 0xD037C
> >>>>
> >>>>
> >>>> On 8/7/07, Keir Fraser <keir@xxxxxxxxxxxxx> wrote:
> >>>>> How about trying:
> >>>>> printf("Before fetch8\n");
> >>>>> dump_regs(regs);
> >>>>> opc = fetch8(regs);
> >>>>> printf("After fetch8\n");
> >>>>> switch (opc) { ...
> >>>>>
> >>>>> This will let you see what eip is being fetched from, and also confirm
> >>>>> that
> >>>>> the crash happens within fetch8().
> >>>>>
> >>>>> You could also try adding more printf()s inside fetch8() and address()
> >>>>> to
> >>>>> find out which specific bit of fetch8() is crashing (if that indeed the
> >>>>> function that is crashing).
> >>>>>
> >>>>> -- Keir
> >>>>>
> >>>>> On 7/8/07 11:30, "Brady Chen" <chenchp@xxxxxxxxx> wrote:
> >>>>>
> >>>>>> Hi, Keir,
> >>>>>> I made the change as you said:
> >>>>>> change diff is:
> >>>>>> [root@localhost firmware]# hg diff vmxassist/vm86.c
> >>>>>> diff -r 6f18f5bdeea3 tools/firmware/vmxassist/vm86.c
> >>>>>> --- a/tools/firmware/vmxassist/vm86.c Mon Aug 06 15:33:42 2007 +0100
> >>>>>> +++ b/tools/firmware/vmxassist/vm86.c Tue Aug 07 18:26:12 2007 +0800
> >>>>>> @@ -40,7 +40,7 @@ static struct regs saved_rm_regs;
> >>>>>> static struct regs saved_rm_regs;
> >>>>>>
> >>>>>> #ifdef DEBUG
> >>>>>> -int traceset = 0;
> >>>>>> +int traceset = ~0;
> >>>>>>
> >>>>>> char *states[] = {
> >>>>>> "<VM86_REAL>",
> >>>>>> @@ -620,6 +620,7 @@ movr(struct regs *regs, unsigned prefix,
> >>>>>> TRACE((regs, regs->eip - eip,
> >>>>>> "movw %%%s, *0x%x", rnames[r], addr));
> >>>>>> write16(addr, MASK16(val));
> >>>>>> + printf("after write16 of movw\n");
> >>>>>> }
> >>>>>> return 1;
> >>>>>>
> >>>>>> @@ -1305,6 +1306,7 @@ opcode(struct regs *regs)
> >>>>>> unsigned eip = regs->eip;
> >>>>>> unsigned opc, modrm, disp;
> >>>>>> unsigned prefix = 0;
> >>>>>> + printf("top of opcode\n");
> >>>>>>
> >>>>>> if (mode == VM86_PROTECTED_TO_REAL &&
> >>>>>> oldctx.cs_arbytes.fields.default_ops_size) {
> >>>>>> @@ -1712,6 +1714,8 @@ trap(int trapno, int errno, struct regs
> >>>>>> if (trapno == 14)
> >>>>>> printf("Page fault address 0x%x\n", get_cr2());
> >>>>>> dump_regs(regs);
> >>>>>> + printf("0xd0800 is 0x%0x\n", *((unsigned
> >>>>>> short*)0xd0800));
> >>>>>> + printf("0xd0804 is 0x%0x\n", *((unsigned
> >>>>>> short*)0xd0804));
> >>>>>> halt();
> >>>>>> }
> >>>>>> }
> >>>>>>
> >>>>>>
> >>>>>> here is the output:
> >>>>>> (XEN) HVM6: top of opcode
> >>>>>> (XEN) HVM6: 0x0000D71F: 0xD00:0x071F (0) data32
> >>>>>> (XEN) HVM6: 0x0000D71F: 0xD00:0x071F (0) opc 0x83
> >>>>>> (XEN) HVM6: top of opcode
> >>>>>> (XEN) HVM6: 0x0000D71B: 0xD00:0x071B (0) %es:
> >>>>>> (XEN) HVM6: 0x0000D71B: 0xD00:0x071B (0) addr32
> >>>>>> (XEN) HVM6: 0x0000D71D: 0xD00:0x071D (0) movw %ax, *0xD07FE
> >>>>>> (XEN) HVM6: after write16 of movw
> >>>>>> (XEN) HVM6: top of opcode
> >>>>>> (XEN) HVM6: Trap (0x6) while in real mode
> >>>>>> (XEN) HVM6: eax D00 ecx 0 edx 71F ebx
> >>>>>> 71E
> >>>>>> (XEN) HVM6: esp D7554 ebp D75A0 esi D7590 edi
> >>>>>> D00
> >>>>>> (XEN) HVM6: trapno 6 errno 0
> >>>>>> (XEN) HVM6: eip D0800 cs 10 eflags 13046
> >>>>>> (XEN) HVM6: uesp D4C29 uss 2
> >>>>>> (XEN) HVM6: ves D4C18 vds D4D9C vfs D07FE vgs
> >>>>>> D75B4
> >>>>>> (XEN) HVM6: cr0 50032 cr2 0 cr3 0 cr4
> >>>>>> 651
> >>>>>> (XEN) HVM6:
> >>>>>> (XEN) HVM6: 0xd0800 is 0xFFFF
> >>>>>> (XEN) HVM6: 0xd0804 is 0x7D8B
> >>>>>> (XEN) HVM6: Halt called from %eip 0xD037C
> >>>>>>
> >>>>>> objdump:
> >>>>>> d07ef: e9 2f ff ff ff jmp d0723 <address+0x23>
> >>>>>> d07f4: 8b 55 08 mov 0x8(%ebp),%edx
> >>>>>> d07f7: 89 f8 mov %edi,%eax
> >>>>>> d07f9: 8b 5d f4 mov 0xfffffff4(%ebp),%ebx
> >>>>>> d07fc: 8b 75 f8 mov 0xfffffff8(%ebp),%esi
> >>>>>> d07ff: 25 ff ff 00 00 and $0xffff,%eax
> >>>>>> d0804: 8b 7d fc mov 0xfffffffc(%ebp),%edi
> >>>>>> d0807: 89 ec mov %ebp,%esp
> >>>>>> d0809: c1 e0 04 shl $0x4,%eax
> >>>>>> d080c: 01 d0 add %edx,%eax
> >>>>>> d080e: 5d pop %ebp
> >>>>>>
> >>>>>> seems the memory is correct, it's crashed in opcode()
> >>>>>> and i think it's fetch8(regs) which crash the system. I tried
> >>>>>> fetch8(regs) in trap(), but it cause more traps, and let the hvm guest
> >>>>>> be reset.
> >>>>>>
> >>>>>> On 8/7/07, Keir Fraser <keir@xxxxxxxxxxxxx> wrote:
> >>>>>>> On 7/8/07 10:29, "Keir Fraser" <keir@xxxxxxxxxxxxx> wrote:
> >>>>>>>
> >>>>>>>> What would be useful is to try to add tracing to see how far
> >>>>>>>> vmxassist
> >>>>>>>> gets
> >>>>>>>> after its last line of tracing before the trap occurs. That last line
> >>>>>>>> is
> >>>>>>>> currently from vm86.c, line 620. You might try adding extra printf()
> >>>>>>>> statements imemdiately after the write16() on line 622, and also at
> >>>>>>>> the
> >>>>>>>> top
> >>>>>>>> of the opcode() function. We need to find out at what point vmxassist
> >>>>>>>> is
> >>>>>>>> jumping to this bogus address d0800.
> >>>>>>>
> >>>>>>> Oh, another possibility is that vmxassist has been corrupted in
> >>>>>>> memory.
> >>>>>>> This
> >>>>>>> is particularly likely because, according to the objdump, the
> >>>>>>> 'instruction'
> >>>>>>> that starts at d0800 is actually valid (it'd be an ADD of some sort).
> >>>>>>>
> >>>>>>> So, within trap() you might want to read say 16 bytes starting at
> >>>>>>> 0xd0800
> >>>>>>> and printf() them. So we can see if they match what objdump says
> >>>>>>> should
> >>>>>>> be
> >>>>>>> there.
> >>>>>>>
> >>>>>>> -- Keir
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> Xen-devel mailing list
> >>>>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
> >>>>>> http://lists.xensource.com/xen-devel
> >>>>>
> >>>>>
> >>>>
> >>>
> >>> _______________________________________________
> >>> Xen-devel mailing list
> >>> Xen-devel@xxxxxxxxxxxxxxxxxxx
> >>> http://lists.xensource.com/xen-devel
> >>
> >>
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > http://lists.xensource.com/xen-devel
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
Home