WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] Re: Linux Stubdom Problem

To: Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>
Subject: [Xen-devel] Re: Linux Stubdom Problem
From: Jiageng Yu <yujiageng734@xxxxxxxxx>
Date: Wed, 27 Jul 2011 20:56:05 +0800
Cc: Ian Campbell <Ian.Campbell@xxxxxxxxxxxxx>, Anthony PERARD <anthony.perard@xxxxxxxxx>, "Xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Wed, 27 Jul 2011 05:57:14 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=OUjjBjov6qEEjDMsaL5tVGIADAP+9WbQ6eMVg5IC1LI=; b=Ch/AsKzYI4CB7vdWtocNmNnwXe4kJ958/TU8l5hAKCTtZ3LGrTQv0R8jQfszt5DJJY lwZlHbPA+pS2ZcWV001Xmb9YGhLIvVz9KJfyJGdchDFvlNaxkQki/xXfoK33FSj32jdY KyGPdoFEeJ4zso1ps7JWFQMH5Pb8A3LvXyJFQ=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <alpine.DEB.2.00.1107271224350.12963@kaball-desktop>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <CAJ0pt14RBmT+bCGhU6szMW4Aje-mBQ-WVR8Vb7wOLefgauatbA@xxxxxxxxxxxxxx> <alpine.DEB.2.00.1107211815370.12963@kaball-desktop> <CAJ0pt15gNOhDb5K+oaqN2w8NxQ5HmtVp6YBYY+yZY6gpOST5+Q@xxxxxxxxxxxxxx> <CAJ0pt16xxQTqE==JGSZ-W=e4zwHV41Cs-wXE5V=x7SG5Aak+3g@xxxxxxxxxxxxxx> <alpine.DEB.2.00.1107271224350.12963@kaball-desktop>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
2011/7/27 Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>:
> On Tue, 26 Jul 2011, Jiageng Yu wrote:
>> 2011/7/26 Jiageng Yu <yujiageng734@xxxxxxxxx>:
>> > 2011/7/22 Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>:
>> >> On Thu, 21 Jul 2011, Jiageng Yu wrote:
>> >>> 2011/7/19 Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>:
>> >>> > CC'ing Tim and xen-devel
>> >>> >
>> >>> > On Mon, 18 Jul 2011, Jiageng Yu wrote:
>> >>> >> 2011/7/16 Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>:
>> >>> >> > On Fri, 15 Jul 2011, Jiageng Yu wrote:
>> >>> >> >> 2011/7/15 Jiageng Yu <yujiageng734@xxxxxxxxx>:
>> >>> >> >> > 2011/7/15 Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>:
>> >>> >> >> >> On Fri, 15 Jul 2011, Jiageng Yu wrote:
>> >>> >> >> >>> > Does it mean you are actually able to boot an HVM guest 
>> >>> >> >> >>> > using Linux
>> >>> >> >> >>> > based stubdoms?? Did you manage to solve the framebuffer 
>> >>> >> >> >>> > problem too?
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >> >>> The HVM guest is booted. But the boot process is terminated 
>> >>> >> >> >>> because
>> >>> >> >> >>> vga bios is not invoked by seabios. I have got stuck here for 
>> >>> >> >> >>> a week.
>> >>> >> >> >>>
>> >>> >> >> >>
>> >>> >> >> >> There was a bug in xen-unstable.hg or seabios that would 
>> >>> >> >> >> prevent vga bios from
>> >>> >> >> >> being loaded, it should be fixed now.
>> >>> >> >> >>
>> >>> >> >> >> Alternatively you can temporarely work around the issue with 
>> >>> >> >> >> this hacky patch:
>> >>> >> >> >>
>> >>> >> >> >> ---
>> >>> >> >> >>
>> >>> >> >> >>
>> >>> >> >> >> diff -r 00d2c5ca26fd tools/firmware/hvmloader/hvmloader.c
>> >>> >> >> >> --- a/tools/firmware/hvmloader/hvmloader.c      Fri Jul 08 
>> >>> >> >> >> 18:35:24 2011 +0100
>> >>> >> >> >> +++ b/tools/firmware/hvmloader/hvmloader.c      Fri Jul 15 
>> >>> >> >> >> 11:37:12 2011 +0000
>> >>> >> >> >> @@ -430,7 +430,7 @@ int main(void)
>> >>> >> >> >>             bios->create_pir_tables();
>> >>> >> >> >>     }
>> >>> >> >> >>
>> >>> >> >> >> -    if ( bios->load_roms )
>> >>> >> >> >> +    if ( 1 )
>> >>> >> >> >>     {
>> >>> >> >> >>         switch ( virtual_vga )
>> >>> >> >> >>         {
>> >>> >> >> >>
>> >>> >> >> >>
>> >>> >> >> >
>> >>> >> >> > Yes. Vga bios is booted. However, the upstram qemu receives a 
>> >>> >> >> > SIGSEGV
>> >>> >> >> > signal subsequently. I am trying to print the call stack when
>> >>> >> >> > receiving the signal.
>> >>> >> >> >
>> >>> >> >>
>> >>> >> >> Hi,
>> >>> >> >>
>> >>> >> >>    I find the cause of SIGSEGV signal:
>> >>> >> >>
>> >>> >> >>    cpu_physical_memory_rw(target_phys_addr_t addr, uint8_t *buf, 
>> >>> >> >> int
>> >>> >> >> len, int is_write)
>> >>> >> >>                   ->memcpy(buf, ptr + (addr & ~TARGET_PAGE_MASK), 
>> >>> >> >> l);
>> >>> >> >>
>> >>> >> >>     In my case, ptr=0 and addr=0xc253e, when qemu attempts to vist
>> >>> >> >> 0x53e address, the SIGSEGV signal is generated.
>> >>> >> >>
>> >>> >> >>     I believe the qemu is trying to vist vram in this moment. This
>> >>> >> >> code seems no problem, and I will continue to find the root cause.
>> >>> >> >>
>> >>> >> >
>> >>> >> > The vram is allocated by qemu, see hw/vga.c:vga_common_init.
>> >>> >> > qemu_ram_alloc under xen ends up calling xen_ram_alloc that calls
>> >>> >> > xc_domain_populate_physmap_exact.
>> >>> >> > xc_domain_populate_physmap_exact is the hypercall that should ask 
>> >>> >> > Xen to
>> >>> >> > add the missing vram pages in the guest. Maybe this hypercall is 
>> >>> >> > failing
>> >>> >> > in your case?
>> >>> >>
>> >>> >>
>> >>> >> Hi,
>> >>> >>
>> >>> >>    I continue to invesgate this bug and find hypercall_mmu_update in
>> >>> >> qemu_remap_bucket(xc_map_foreign_bulk) is failing:
>> >>> >>
>> >>> >> do_mmu_update
>> >>> >>       ->mod_l1_entry
>> >>> >>              ->  if ( !p2m_is_ram(p2mt) || unlikely(mfn == 
>> >>> >> INVALID_MFN) )
>> >>> >>                          return -EINVAL;
>> >>> >>
>> >>> >>    mfn==INVALID_MFN, because :
>> >>> >>
>> >>> >> mod_l1_entry
>> >>> >>       ->gfn_to_mfn(p2m_get_hostp2m(pg_dom), l1e_get_pfn(nl1e), 
>> >>> >> &p2mt));
>> >>> >>               ->p2m->get_entry
>> >>> >>                         ->p2m_gfn_to_mfn
>> >>> >>                                -> if ( gfn > p2m->max_mapped_pfn )
>> >>> >>                                    /* This pfn is higher than the
>> >>> >> highest the p2m map currently holds */
>> >>> >>                                    return _mfn(INVALID_MFN);
>> >>> >>
>> >>> >>    The p2m->max_mapped_pfn is usually 0xfffff. In our case,
>> >>> >> mmu_update.val exceeds 0x8000000100000000.  Additionally, l1e =
>> >>> >> l1e_from_intpte(mmu_update.val); gfn=l1e_get_pfn(l1e ). Therefore, gfn
>> >>> >> will exceed 0xfffff.
>> >>> >>
>> >>> >>    In the case of minios based stubdom, the mmu_update.vals do not
>> >>> >> exceed 0x8000000100000000. Next, I will invesgate why mmu_update.val
>> >>> >> exceeds 0x8000000100000000.
>> >>> >
>> >>> > It looks like the address of the guest that qemu is trying to map is 
>> >>> > not
>> >>> > valid.
>> >>> > Make sure you are running a guest with less than 2GB of ram, otherwise
>> >>> > you need the patch series that Anthony sent on Friday:
>> >>> >
>> >>> > http://marc.info/?l=qemu-devel&m=131074042905711&w=2
>> >>>
>> >>> Not this problem. I never alloc more than 2GB for the hvm guest. The
>> >>> call stack in qemu is:
>> >>>
>> >>> qemu_get_ram_ptr
>> >>>       ->qemu_map_cache(addr, 0, 1)
>> >>>                  -> if (!entry->vaddr_base || entry->paddr_index !=
>> >>> address_index ||
>> >>>                                           !test_bit(address_offset >>
>> >>> XC_PAGE_SHIFT, entry->valid_mapping)) {
>> >>>                            qemu_remap_bucket(entry, size ? :
>> >>> MCACHE_BUCKET_SIZE, address_index);
>> >>>                                  ->xc_map_foreign_bulk(xen_xc,
>> >>> xen_domid, PROT_READ|PROT_WRITE,
>> >>>
>> >>>                 pfns, err, nb_pfn);
>> >>>
>> >>> The qemu tries to map pages from hvm guest(xen_domid) to linux
>> >>> stubdom. But some hvm pages' pfns are larger than 0xfffff. So, in the
>> >>> p2m_gfn_to_mfn, the judgement condition is valid:(p2m->max_mapped_pfn
>> >>> = 0xfffff)
>> >>>
>> >>>     if ( gfn > p2m->max_mapped_pfn )
>> >>>         /* This pfn is higher than the highest the p2m map currently 
>> >>> holds */
>> >>>         return _mfn(INVALID_MFN);
>> >>>
>> >>>  In minios stubdom case, the hvm pages' pfns do not exceed 0xfffff.
>> >>> Maybe the address translation in linux stubdom cause this probem?
>> >>
>> >> Trying to map a pfn > 0xfffff is clearly a mistake if the guest's memory
>> >> does not exceed 2G:
>> >>
>> >> 0xfffff * 4096 > 2G
>> >>
>> >>
>> >>>  BTW, in minios stubdom case, there seems no hvmloader process. Is it
>> >>> needed in linux stubdom?
>> >>
>> >> hvmloader is the first thing that runs within the guest, it is not a
>> >> process in the stubdom or in dom0.
>> >> It is required in both minios and linux stubdoms.
>> >
>> > Hi Stefano,
>> >
>> >      I patched these patches, but we still have the same problem.
>> > However, I noticed the qemu_get_ram_ptr(s->vram_offset) in
>> > vga_common_init function was also failed. Maybe this can explain the
>> > previous problem, which happened in the phase of trying to remap
>> > 0xc0000-0xc8fff of hvm guest into stubdom.
>> >
>> >     I have traced the process of qemu_get_ram_ptr(s->vram_offset) and
>> > located the failure in p2m_gfn_to_mfn function:
>> >
>> >     pod_retry_l3:
>> >        if ( (l3e_get_flags(*l3e) & _PAGE_PRESENT) == 0 )
>> >        {
>> >                 .....
>> >                 return _mfn(INVALID_MFN);
>> >        }
>> >
>> >     I will continue to analyze this failure.
>> >
>> >     Thanks!
>> >
>> > Jiageng Yu.
>> >
>>
>>
>> Hi,
>>
>>     I compared the two executions of vga_common_init function in dom0
>> and linux based stubdom. The former succeeded and the later was
>> failed. First, they have the same call stack:
>>
>> Dom0 & Stubdom
>> _________________________________________________________
>> vga_common_init
>>      ->qemu_get_ram_ptr(s->vram_offset)
>>            ->block->host = xen_map_block(block->offset, block->length);
>>                  ->xc_map_foreign_bulk()
>>                         ->linux_privcmd_map_foreign_bulk()
>>                                ->xen_remap_domain_mfn_range()
>>                                      ->HYPERVISOR_mmu_update()
>> __________________________________________________________
>>
>> Xen
>> __________________________________________________________
>> do_mmu_update()
>>    ->case MMU_PT_UPDATE_PRESERVE_AD:
>>    ->case PGT_l1_page_table:
>>    ->mod_l1_entry(va, l1e, mfn,cmd == MMU_PT_UPDATE_PRESERVE_AD, v, 
>> pg_owner);
>>           ->mfn_x(gfn_to_mfn(p2m_get_hostp2m(pg_dom),
>> l1e_get_pfn(nl1e), &p2mt));
>>                  ->gfn_to_mfn_type_p2m()
>>                         ->p2m->get_entry(p2m, gfn, t, &a, q);
>>                                ->p2m_gfn_to_mfn(p2m,gfn,t,&a,q)
>>                                       ->if ( (l3e_get_flags(*l3e) &
>> _PAGE_PRESENT) == 0 )
>>                                       ->    Error happens!
>>
>> The qemu in dom0 can find the l3e of hvm guest, but the qemu in linux
>> stubdom cannot find the l3e. In my case, s->vram_offset=0x40000000,
>> vga_ram_size=0x800000. Therefore, we are going to map hvm guest's
>> address area(pfn:0x40000, size:8M) into linux stubdom's address space.
>>
>> In p2m_gfn_to_mfn function, p2m->domain->domain_id=hvm guest,
>> gfn=0x40000, t=p2m_mmio_dm.
>> mfn = pagetable_get_mfn(p2m_get_pagetable(p2m) = 0x10746e;
>> map_domain_page(mfn_x(mfn)) is also success. However, after executing:
>> l3e += ( (0x40000 << PAGE_SHIFT) >> L3_PAGETABLE_SHIFT)
>> the l3e->l3 =0 , and the error happens.
>>
>> So, in linux stubdom, when we are going to map the specified hvm
>> guest's address(pfn:0x40000, size:8M), we find these pages of hvm
>> guest are not present. This is never happened in qemu of dom0. Could
>> you give me some prompts to this problem?
>
>
> It seems that you are trying to map pages that don't exist.
> The pages in question should be allocated by:
>
> qemu_ram_alloc(NULL, "vga.vram", vga_ram_size)
>    qemu_ram_alloc_from_ptr
>        xen_ram_alloc
>            xc_domain_populate_physmap_exact
>
> so I would add some printf and printk on this code path to find out if
> xc_domain_populate_physmap_exact fails for some reasons.

Hmm.. the linux stubdom kernel had a wrong p2m pair
<gfn(0x40000),mfn(0x127bd2)> for some reason. But next, the
xc_domain_populate_physmap_exact will setup the correct p2m pair
<gfn(0x40000),mfn(0x896b7)>. However, the p2m pair in stubdom kernel
has not been updated, because the fllowing access to 0x40000 still
uses 0x127bd2.

I notice you have a patch: xen: modify kernel mappings corresponding
to granted pages. I think maybe it could slove my problem.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel