You could give that a try, but really it shouldn't be going at
0xc0000-0x100000 at all. There are usually ROM images residing there.
This is more likely to be a mis-emulation. Can you get a dump of the bytes
around 0xd680-0xd780? Then we could try and work out what the guest is
trying to execute, and see whether emulation is going wrong. A register dump
from the guest (dump_regs()) at the start of every call to opcode() might
also be useful.
-- Keir
On 8/8/07 09:25, "Brady Chen" <chenchp@xxxxxxxxx> wrote:
> Hi Keir,
> I think the 7th issue I mentioned is the root cause,
> so I have a question.
> For real mode simulation, the simulator is running in the same space
> with the codes to-be-simulated? then how to protect simulator from
> being modified by to-be-simulated code?
>
> can I change the address of vmxassist to a higher address? just try to
> give more space to the to-be-simulated windows.
>
> On 8/8/07, Brady Chen <chenchp@xxxxxxxxx> wrote:
>> it's possible.
>> any ideas to trace the function stack of xen guest? like "bt" command in gdb.
>>
>> I did some analysis:
>> 1. the call flow is opcode()->fetch8()->address()
>> 2. only the printf in address() will change the behaver of crash.
>> 3. and the crash EIP (0xD0800) is in the address() from the objdump.
>> 4. the address() will be invoked more then 40, 000 times in one
>> simulation, before the crash.
>> 5. seems there are no recursive invoking in opcode(), fetch8(), address()
>> 6. from the output of "xen dmesg", before the crash, a instructions
>> sequence is simulated several times (you could check the previous
>> mails i send for "xen dmesg" output)
>> 7. before the trap, the simulated instruction is "movw %ax, *0xD07FE",
>> and the "*0xD07FE" is just the address of address(), (you could get
>> the objdump output from previous mails too), so i think it's the
>> simulation which crash the memory of address().
>>
>> On 8/8/07, Keir Fraser <keir@xxxxxxxxxxxxx> wrote:
>>> Stack corruption/overflow, possibly?
>>>
>>> K.
>>>
>>> On 7/8/07 17:06, "Brady Chen" <chenchp@xxxxxxxxx> wrote:
>>>
>>>> Yes, the printfs are the only changes. once I remove these prints, the
>>>> trap comes back, with the same EIP (D0800)
>>>>
>>>> I tried to keep the first two printfs, the trap comes with different
>>>> EIP(D19FD)
>>>> static unsigned
>>>> address(struct regs *regs, unsigned seg, unsigned off)
>>>> {
>>>> uint64_t gdt_phys_base;
>>>> unsigned long long entry;
>>>> unsigned seg_base, seg_limit;
>>>> unsigned entry_low, entry_high;
>>>>
>>>> printf("f 1\n");
>>>> if (seg == 0) {
>>>> if (mode == VM86_REAL || mode == VM86_REAL_TO_PROTECTED)
>>>> return off;
>>>> else
>>>> panic("segment is zero, but not in real mode!\n");
>>>> }
>>>>
>>>> printf("f 2\n");
>>>>
>>>> xen dmesg output:
>>>> (XEN) HVM3: 0x0000D71F: 0xD00:0x071F (0) opc 0x83
>>>> (XEN) HVM3: f 1
>>>> (XEN) HVM3: f 2
>>>> (XEN) HVM3: 0x0000D71F: 0xD00:0x071F (0) external interrupt 8
>>>> (XEN) HVM3: f 1
>>>> (XEN) HVM3: f 1
>>>> (XEN) HVM3: f 1
>>>> (XEN) HVM3: Trap (0x6) while in real mode
>>>> (XEN) HVM3: eax CFAE ecx 0 edx 0 ebx D75B4
>>>> (XEN) HVM3: esp D7564 ebp D75A0 esi 71F edi 8
>>>> (XEN) HVM3: trapno 6 errno 0
>>>> (XEN) HVM3: eip D19FD cs 10 eflags 13046
>>>> (XEN) HVM3: uesp CFAE uss 0
>>>> (XEN) HVM3: ves D4C44 vds 8 vfs 83 vgs 71F
>>>> (XEN) HVM3: cr0 50032 cr2 0 cr3 0 cr4 651
>>>> (XEN) HVM3:
>>>> (XEN) HVM3: Halt called from %eip 0xD037C
>>>>
>>>>
>>>> and the objdump shows that:
>>>> 000d1970 <interrupt>:
>>>> d1970: 55 push %ebp
>>>> d1971: 89 e5 mov %esp,%ebp
>>>> d1973: 57 push %edi
>>>> d1974: 89 d7 mov %edx,%edi
>>>> d1976: 56 push %esi
>>>> ....
>>>> d19f8: 66 89 30 mov %si,(%eax)
>>>> d19fb: 31 d2 xor %edx,%edx
>>>> d19fd: 8d 34 bd 00 00 00 00 lea 0x0(,%edi,4),%esi
>>>> d1a04: 81 63 30 ff fd ff ff andl $0xfffffdff,0x30(%ebx)
>>>> d1a0b: 89 d8 mov %ebx,%eax
>>>> d1a0d: 89 34 24 mov %esi,(%esp)
>>>>
>>>>
>>>> On 8/7/07, Keir Fraser <keir@xxxxxxxxxxxxx> wrote:
>>>>> Very weird. The emulations now aren't at the same address as before either
>>>>> (0xd4c3 rather than 0xd71b). Is the *only* difference that you added these
>>>>> printf()s -- is it at all possible that the guest is executing down a
>>>>> different path here for other reasons? If it's really down to the
>>>>> printf()s
>>>>> then I guess you'll have to shuffle/remove printf()s to get the old
>>>>> behaviour back.
>>>>>
>>>>> -- Keir
>>>>>
>>>>> On 7/8/07 12:35, "Brady Chen" <chenchp@xxxxxxxxx> wrote:
>>>>>
>>>>>> it's strange:
>>>>>> if i add these prints, i get " Unknown opcode", not "trap".
>>>>>> ===added printf
>>>>>> [root@localhost firmware]# hg diff -p vmxassist/vm86.c
>>>>>> diff -r 6f18f5bdeea3 tools/firmware/vmxassist/vm86.c
>>>>>> --- a/tools/firmware/vmxassist/vm86.c Mon Aug 06 15:33:42 2007 +0100
>>>>>> +++ b/tools/firmware/vmxassist/vm86.c Tue Aug 07 19:33:55 2007 +0800
>>>>>> @@ -40,7 +40,7 @@ static struct regs saved_rm_regs;
>>>>>> static struct regs saved_rm_regs;
>>>>>>
>>>>>> #ifdef DEBUG
>>>>>> -int traceset = 0;
>>>>>> +int traceset = ~0;
>>>>>>
>>>>>> char *states[] = {
>>>>>> "<VM86_REAL>",
>>>>>> @@ -128,6 +128,7 @@ address(struct regs *regs, unsigned seg,
>>>>>> unsigned seg_base, seg_limit;
>>>>>> unsigned entry_low, entry_high;
>>>>>>
>>>>>> + printf("f 1\n");
>>>>>> if (seg == 0) {
>>>>>> if (mode == VM86_REAL || mode == VM86_REAL_TO_PROTECTED)
>>>>>> return off;
>>>>>> @@ -135,12 +136,16 @@ address(struct regs *regs, unsigned seg,
>>>>>> panic("segment is zero, but not in real
>>>>>> mode!\n");
>>>>>> }
>>>>>>
>>>>>> + printf("f 2\n");
>>>>>> if (mode == VM86_REAL || seg > oldctx.gdtr_limit ||
>>>>>> (mode == VM86_REAL_TO_PROTECTED && regs->cs == seg))
>>>>>> return ((seg & 0xFFFF) << 4) + off;
>>>>>>
>>>>>> + printf("f 3\n");
>>>>>> gdt_phys_base = guest_linear_to_phys(oldctx.gdtr_base);
>>>>>> + printf("f 4\n");
>>>>>> if (gdt_phys_base != (uint32_t)gdt_phys_base) {
>>>>>> + printf("f 5\n");
>>>>>> printf("gdt base address above 4G\n");
>>>>>> cpuid_addr_value(gdt_phys_base + 8 * (seg >> 3), &entry);
>>>>>> } else
>>>>>> @@ -152,14 +157,17 @@ address(struct regs *regs, unsigned seg,
>>>>>> seg_base = (entry_high & 0xFF000000) | ((entry >> 16) &
>>>>>> 0xFFFFFF);
>>>>>> seg_limit = (entry_high & 0xF0000) | (entry_low & 0xFFFF);
>>>>>>
>>>>>> + printf("f 6\n");
>>>>>> if (entry_high & 0x8000 &&
>>>>>> ((entry_high & 0x800000 && off >> 12 <= seg_limit) ||
>>>>>> (!(entry_high & 0x800000) && off <= seg_limit)))
>>>>>> return seg_base + off;
>>>>>> + printf("f 7\n");
>>>>>>
>>>>>> panic("should never reach here in function address():\n\t"
>>>>>> "entry=0x%08x%08x, mode=%d, seg=0x%08x,
>>>>>> offset=0x%08x\n",
>>>>>> entry_high, entry_low, mode, seg, off);
>>>>>> + printf("f 8\n");
>>>>>>
>>>>>> return 0;
>>>>>> }
>>>>>> @@ -286,6 +294,7 @@ fetch8(struct regs *regs)
>>>>>> unsigned addr = address(regs, regs->cs, MASK16(regs->eip));
>>>>>>
>>>>>> regs->eip++;
>>>>>> + printf("f 9\n");
>>>>>> return read8(addr);
>>>>>> }
>>>>>>
>>>>>> ===output when add many printf
>>>>>> (XEN) HVM12: 0x0000D4C3: 0xD00:0x04C3 (0) addr32addr32f 1
>>>>>> (XEN) HVM12: f 2
>>>>>> (XEN) HVM12: f 9
>>>>>> (XEN) HVM12: f 1
>>>>>> (XEN) HVM12: f 2
>>>>>> (XEN) HVM12: 0x0000D4C3: 0xD00:0x04C3 (0) data32data32f 1
>>>>>> (XEN) HVM12: f 2
>>>>>> (XEN) HVM12: f 9
>>>>>> (XEN) HVM12: f 1
>>>>>> (XEN) HVM12: f 2
>>>>>> (XEN) HVM12: 0x0000D4C3: 0xD00:0x04C3 (0) opc 0x83opc 0xD7704f 1
>>>>>> (XEN) HVM12: f 2
>>>>>> (XEN) HVM12: Unknown opcode at 0D00:04C3=0xD4C3
>>>>>> (XEN) HVM12: Halt called from %eip 0xD3B4A
>>>>>>
>>>>>> On 8/7/07, Brady Chen <chenchp@xxxxxxxxx> wrote:
>>>>>>> Hi, yes, it's crashed in fetch8. it's very slow after I add this print
>>>>>>> info.
>>>>>>> the main function of fetch8 seems to be address(). seems crashed in
>>>>>>> address().
>>>>>>>
>>>>>>> (XEN) HVM7: after write16 of movw
>>>>>>> (XEN) HVM7: top of opcode
>>>>>>> (XEN) HVM7: Before fetch8
>>>>>>> (XEN) HVM7: eax 7E80 ecx 2D1B edx 0 ebx
>>>>>>> 404E
>>>>>>> (XEN) HVM7: esp D76F4 ebp 1FF0 esi 7BE edi
>>>>>>> C37FE
>>>>>>> (XEN) HVM7: trapno D errno 0
>>>>>>> (XEN) HVM7: eip 71F cs D00 eflags 33206
>>>>>>> (XEN) HVM7: uesp CFB4 uss 0
>>>>>>> (XEN) HVM7: ves D00 vds D00 vfs 0 vgs
>>>>>>> 0
>>>>>>> (XEN) HVM7: cr0 50032 cr2 0 cr3 0 cr4
>>>>>>> 651
>>>>>>> (XEN) HVM7:
>>>>>>> (XEN) HVM7: Trap (0x6) while in real mode
>>>>>>> (XEN) HVM7: eax D00 ecx 0 edx 71F ebx
>>>>>>> 89
>>>>>>> (XEN) HVM7: esp D75E4 ebp D7630 esi D7620 edi
>>>>>>> D00
>>>>>>> (XEN) HVM7: trapno 6 errno 0
>>>>>>> (XEN) HVM7: eip D0800 cs 10 eflags 13046
>>>>>>> (XEN) HVM7: uesp 71F uss D76D4
>>>>>>> (XEN) HVM7: ves D7610 vds D3AB9 vfs D762C vgs
>>>>>>> D7644
>>>>>>> (XEN) HVM7: cr0 50032 cr2 0 cr3 0 cr4
>>>>>>> 651
>>>>>>> (XEN) HVM7:
>>>>>>> (XEN) HVM7: 0xd0800 is 0xFFFF
>>>>>>> (XEN) HVM7: 0xd0804 is 0x7D8B
>>>>>>> (XEN) HVM7: Halt called from %eip 0xD037C
>>>>>>>
>>>>>>>
>>>>>>> On 8/7/07, Keir Fraser <keir@xxxxxxxxxxxxx> wrote:
>>>>>>>> How about trying:
>>>>>>>> printf("Before fetch8\n");
>>>>>>>> dump_regs(regs);
>>>>>>>> opc = fetch8(regs);
>>>>>>>> printf("After fetch8\n");
>>>>>>>> switch (opc) { ...
>>>>>>>>
>>>>>>>> This will let you see what eip is being fetched from, and also confirm
>>>>>>>> that
>>>>>>>> the crash happens within fetch8().
>>>>>>>>
>>>>>>>> You could also try adding more printf()s inside fetch8() and address()
>>>>>>>> to
>>>>>>>> find out which specific bit of fetch8() is crashing (if that indeed the
>>>>>>>> function that is crashing).
>>>>>>>>
>>>>>>>> -- Keir
>>>>>>>>
>>>>>>>> On 7/8/07 11:30, "Brady Chen" <chenchp@xxxxxxxxx> wrote:
>>>>>>>>
>>>>>>>>> Hi, Keir,
>>>>>>>>> I made the change as you said:
>>>>>>>>> change diff is:
>>>>>>>>> [root@localhost firmware]# hg diff vmxassist/vm86.c
>>>>>>>>> diff -r 6f18f5bdeea3 tools/firmware/vmxassist/vm86.c
>>>>>>>>> --- a/tools/firmware/vmxassist/vm86.c Mon Aug 06 15:33:42 2007 +0100
>>>>>>>>> +++ b/tools/firmware/vmxassist/vm86.c Tue Aug 07 18:26:12 2007 +0800
>>>>>>>>> @@ -40,7 +40,7 @@ static struct regs saved_rm_regs;
>>>>>>>>> static struct regs saved_rm_regs;
>>>>>>>>>
>>>>>>>>> #ifdef DEBUG
>>>>>>>>> -int traceset = 0;
>>>>>>>>> +int traceset = ~0;
>>>>>>>>>
>>>>>>>>> char *states[] = {
>>>>>>>>> "<VM86_REAL>",
>>>>>>>>> @@ -620,6 +620,7 @@ movr(struct regs *regs, unsigned prefix,
>>>>>>>>> TRACE((regs, regs->eip - eip,
>>>>>>>>> "movw %%%s, *0x%x", rnames[r], addr));
>>>>>>>>> write16(addr, MASK16(val));
>>>>>>>>> + printf("after write16 of movw\n");
>>>>>>>>> }
>>>>>>>>> return 1;
>>>>>>>>>
>>>>>>>>> @@ -1305,6 +1306,7 @@ opcode(struct regs *regs)
>>>>>>>>> unsigned eip = regs->eip;
>>>>>>>>> unsigned opc, modrm, disp;
>>>>>>>>> unsigned prefix = 0;
>>>>>>>>> + printf("top of opcode\n");
>>>>>>>>>
>>>>>>>>> if (mode == VM86_PROTECTED_TO_REAL &&
>>>>>>>>> oldctx.cs_arbytes.fields.default_ops_size) {
>>>>>>>>> @@ -1712,6 +1714,8 @@ trap(int trapno, int errno, struct regs
>>>>>>>>> if (trapno == 14)
>>>>>>>>> printf("Page fault address 0x%x\n",
>>>>>>>>> get_cr2());
>>>>>>>>> dump_regs(regs);
>>>>>>>>> + printf("0xd0800 is 0x%0x\n", *((unsigned
>>>>>>>>> short*)0xd0800));
>>>>>>>>> + printf("0xd0804 is 0x%0x\n", *((unsigned
>>>>>>>>> short*)0xd0804));
>>>>>>>>> halt();
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> here is the output:
>>>>>>>>> (XEN) HVM6: top of opcode
>>>>>>>>> (XEN) HVM6: 0x0000D71F: 0xD00:0x071F (0) data32
>>>>>>>>> (XEN) HVM6: 0x0000D71F: 0xD00:0x071F (0) opc 0x83
>>>>>>>>> (XEN) HVM6: top of opcode
>>>>>>>>> (XEN) HVM6: 0x0000D71B: 0xD00:0x071B (0) %es:
>>>>>>>>> (XEN) HVM6: 0x0000D71B: 0xD00:0x071B (0) addr32
>>>>>>>>> (XEN) HVM6: 0x0000D71D: 0xD00:0x071D (0) movw %ax, *0xD07FE
>>>>>>>>> (XEN) HVM6: after write16 of movw
>>>>>>>>> (XEN) HVM6: top of opcode
>>>>>>>>> (XEN) HVM6: Trap (0x6) while in real mode
>>>>>>>>> (XEN) HVM6: eax D00 ecx 0 edx 71F ebx
>>>>>>>>> 71E
>>>>>>>>> (XEN) HVM6: esp D7554 ebp D75A0 esi D7590 edi
>>>>>>>>> D00
>>>>>>>>> (XEN) HVM6: trapno 6 errno 0
>>>>>>>>> (XEN) HVM6: eip D0800 cs 10 eflags 13046
>>>>>>>>> (XEN) HVM6: uesp D4C29 uss 2
>>>>>>>>> (XEN) HVM6: ves D4C18 vds D4D9C vfs D07FE vgs
>>>>>>>>> D75B4
>>>>>>>>> (XEN) HVM6: cr0 50032 cr2 0 cr3 0 cr4
>>>>>>>>> 651
>>>>>>>>> (XEN) HVM6:
>>>>>>>>> (XEN) HVM6: 0xd0800 is 0xFFFF
>>>>>>>>> (XEN) HVM6: 0xd0804 is 0x7D8B
>>>>>>>>> (XEN) HVM6: Halt called from %eip 0xD037C
>>>>>>>>>
>>>>>>>>> objdump:
>>>>>>>>> d07ef: e9 2f ff ff ff jmp d0723 <address+0x23>
>>>>>>>>> d07f4: 8b 55 08 mov 0x8(%ebp),%edx
>>>>>>>>> d07f7: 89 f8 mov %edi,%eax
>>>>>>>>> d07f9: 8b 5d f4 mov 0xfffffff4(%ebp),%ebx
>>>>>>>>> d07fc: 8b 75 f8 mov 0xfffffff8(%ebp),%esi
>>>>>>>>> d07ff: 25 ff ff 00 00 and $0xffff,%eax
>>>>>>>>> d0804: 8b 7d fc mov 0xfffffffc(%ebp),%edi
>>>>>>>>> d0807: 89 ec mov %ebp,%esp
>>>>>>>>> d0809: c1 e0 04 shl $0x4,%eax
>>>>>>>>> d080c: 01 d0 add %edx,%eax
>>>>>>>>> d080e: 5d pop %ebp
>>>>>>>>>
>>>>>>>>> seems the memory is correct, it's crashed in opcode()
>>>>>>>>> and i think it's fetch8(regs) which crash the system. I tried
>>>>>>>>> fetch8(regs) in trap(), but it cause more traps, and let the hvm guest
>>>>>>>>> be reset.
>>>>>>>>>
>>>>>>>>> On 8/7/07, Keir Fraser <keir@xxxxxxxxxxxxx> wrote:
>>>>>>>>>> On 7/8/07 10:29, "Keir Fraser" <keir@xxxxxxxxxxxxx> wrote:
>>>>>>>>>>
>>>>>>>>>>> What would be useful is to try to add tracing to see how far
>>>>>>>>>>> vmxassist
>>>>>>>>>>> gets
>>>>>>>>>>> after its last line of tracing before the trap occurs. That last
>>>>>>>>>>> line
>>>>>>>>>>> is
>>>>>>>>>>> currently from vm86.c, line 620. You might try adding extra printf()
>>>>>>>>>>> statements imemdiately after the write16() on line 622, and also at
>>>>>>>>>>> the
>>>>>>>>>>> top
>>>>>>>>>>> of the opcode() function. We need to find out at what point
>>>>>>>>>>> vmxassist
>>>>>>>>>>> is
>>>>>>>>>>> jumping to this bogus address d0800.
>>>>>>>>>>
>>>>>>>>>> Oh, another possibility is that vmxassist has been corrupted in
>>>>>>>>>> memory.
>>>>>>>>>> This
>>>>>>>>>> is particularly likely because, according to the objdump, the
>>>>>>>>>> 'instruction'
>>>>>>>>>> that starts at d0800 is actually valid (it'd be an ADD of some sort).
>>>>>>>>>>
>>>>>>>>>> So, within trap() you might want to read say 16 bytes starting at
>>>>>>>>>> 0xd0800
>>>>>>>>>> and printf() them. So we can see if they match what objdump says
>>>>>>>>>> should
>>>>>>>>>> be
>>>>>>>>>> there.
>>>>>>>>>>
>>>>>>>>>> -- Keir
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Xen-devel mailing list
>>>>>>>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>>>>>>>>> http://lists.xensource.com/xen-devel
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Xen-devel mailing list
>>>>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>>>>>> http://lists.xensource.com/xen-devel
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Xen-devel mailing list
>>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>>>> http://lists.xensource.com/xen-devel
>>>
>>>
>>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|