# HG changeset patch # User yamahata@xxxxxxxxxxxxx # Node ID 652ed19e089391ae145cd3660dd554128203852c # Parent 60d6edcdfde399f4a1ef2cd4c0b7ddb06c191208 fixed some races in ia64_do_page_fault(), vcpu_itc_i(), vcpu_itc_d() and vcpu_fc(). introduce struct p2m_entry and check it later and try it again. PATCHNAME: introduce_p2m_entry Signed-off-by: Isaku Yamahata diff -r 60d6edcdfde3 -r 652ed19e0893 xen/arch/ia64/vmx/vmx_process.c --- a/xen/arch/ia64/vmx/vmx_process.c Mon Jun 12 16:54:12 2006 +0900 +++ b/xen/arch/ia64/vmx/vmx_process.c Mon Jun 12 18:09:07 2006 +0900 @@ -58,7 +58,6 @@ extern void die_if_kernel(char *str, struct pt_regs *regs, long err); extern void rnat_consumption (VCPU *vcpu); -extern unsigned long translate_domain_mpaddr(unsigned long mpaddr); extern void alt_itlb (VCPU *vcpu, u64 vadr); extern void itlb_fault (VCPU *vcpu, u64 vadr); extern void ivhpt_fault (VCPU *vcpu, u64 vadr); diff -r 60d6edcdfde3 -r 652ed19e0893 xen/arch/ia64/vmx/vtlb.c --- a/xen/arch/ia64/vmx/vtlb.c Mon Jun 12 16:54:12 2006 +0900 +++ b/xen/arch/ia64/vmx/vtlb.c Mon Jun 12 18:09:07 2006 +0900 @@ -425,7 +425,7 @@ u64 translate_phy_pte(VCPU *v, u64 *pte, phy_pte.val = *pte; addr = *pte; addr = ((addr & _PAGE_PPN_MASK)>>ps<domain, addr); + addr = lookup_domain_mpa(v->domain, addr, NULL); if(addr & GPFN_IO_MASK){ *pte |= VTLB_PTE_IO; return -1; diff -r 60d6edcdfde3 -r 652ed19e0893 xen/arch/ia64/xen/faults.c --- a/xen/arch/ia64/xen/faults.c Mon Jun 12 16:54:12 2006 +0900 +++ b/xen/arch/ia64/xen/faults.c Mon Jun 12 18:09:07 2006 +0900 @@ -236,9 +236,10 @@ void ia64_do_page_fault (unsigned long a fault = vcpu_translate(current,address,is_data,&pteval,&itir,&iha); if (fault == IA64_NO_FAULT || fault == IA64_USE_TLB) { u64 logps; - pteval = translate_domain_pte(pteval, address, itir, &logps); + struct p2m_entry entry; + pteval = translate_domain_pte(pteval, address, itir, &logps, &entry); vcpu_itc_no_srlz(current,is_data?2:1,address,pteval,-1UL,logps); - if (read_seqretry(vtlb_lock, seq)) { + if (read_seqretry(vtlb_lock, seq) || p2m_entry_retry(&entry)) { vcpu_flush_tlb_vhpt_range(address & ((1 << logps) - 1), logps); goto again; diff -r 60d6edcdfde3 -r 652ed19e0893 xen/arch/ia64/xen/fw_emul.c --- a/xen/arch/ia64/xen/fw_emul.c Mon Jun 12 16:54:12 2006 +0900 +++ b/xen/arch/ia64/xen/fw_emul.c Mon Jun 12 18:09:07 2006 +0900 @@ -370,7 +370,7 @@ efi_translate_domain_addr(unsigned long if (*fault != IA64_NO_FAULT) return 0; } - return ((unsigned long) __va(translate_domain_mpaddr(mpaddr))); + return ((unsigned long) __va(translate_domain_mpaddr(mpaddr, NULL))); } static efi_status_t @@ -541,7 +541,7 @@ do_ssc(unsigned long ssc, struct pt_regs case SSC_WAIT_COMPLETION: if (arg0) { // metaphysical address - arg0 = translate_domain_mpaddr(arg0); + arg0 = translate_domain_mpaddr(arg0, NULL); /**/ stat = (struct ssc_disk_stat *)__va(arg0); ///**/ if (stat->fd == last_fd) stat->count = last_count; /**/ stat->count = last_count; @@ -556,7 +556,7 @@ do_ssc(unsigned long ssc, struct pt_regs arg1 = vcpu_get_gr(current,33); // access rights if (!running_on_sim) { printf("SSC_OPEN, not implemented on hardware. (ignoring...)\n"); arg0 = 0; } if (arg0) { // metaphysical address - arg0 = translate_domain_mpaddr(arg0); + arg0 = translate_domain_mpaddr(arg0, NULL); retval = ia64_ssc(arg0,arg1,0,0,ssc); } else retval = -1L; @@ -573,7 +573,7 @@ if (!running_on_sim) { printf("SSC_OPEN, unsigned long mpaddr; long len; - arg2 = translate_domain_mpaddr(arg2); + arg2 = translate_domain_mpaddr(arg2, NULL); req = (struct ssc_disk_req *) __va(arg2); req->len &= 0xffffffffL; // avoid strange bug len = req->len; @@ -584,7 +584,7 @@ if (!running_on_sim) { printf("SSC_OPEN, retval = 0; if ((mpaddr & PAGE_MASK) != ((mpaddr+len-1) & PAGE_MASK)) { // do partial page first - req->addr = translate_domain_mpaddr(mpaddr); + req->addr = translate_domain_mpaddr(mpaddr, NULL); req->len = PAGE_SIZE - (req->addr & ~PAGE_MASK); len -= req->len; mpaddr += req->len; retval = ia64_ssc(arg0,arg1,arg2,arg3,ssc); @@ -594,7 +594,7 @@ if (!running_on_sim) { printf("SSC_OPEN, //if (last_count >= PAGE_SIZE) printf("ssc(%p,%lx)[part]=%x ",req->addr,req->len,retval); } if (retval >= 0) while (len > 0) { - req->addr = translate_domain_mpaddr(mpaddr); + req->addr = translate_domain_mpaddr(mpaddr, NULL); req->len = (len > PAGE_SIZE) ? PAGE_SIZE : len; len -= PAGE_SIZE; mpaddr += PAGE_SIZE; retval = ia64_ssc(arg0,arg1,arg2,arg3,ssc); diff -r 60d6edcdfde3 -r 652ed19e0893 xen/arch/ia64/xen/mm.c --- a/xen/arch/ia64/xen/mm.c Mon Jun 12 16:54:12 2006 +0900 +++ b/xen/arch/ia64/xen/mm.c Mon Jun 12 18:09:07 2006 +0900 @@ -253,7 +253,7 @@ gmfn_to_mfn_foreign(struct domain *d, un if (d == dom0) return(gpfn); #endif - pte = lookup_domain_mpa(d,gpfn << PAGE_SHIFT); + pte = lookup_domain_mpa(d,gpfn << PAGE_SHIFT, NULL); if (!pte) { panic("gmfn_to_mfn_foreign: bad gpfn. spinning...\n"); } @@ -264,7 +264,8 @@ gmfn_to_mfn_foreign(struct domain *d, un // address, convert the pte for a physical address for (possibly different) // Xen PAGE_SIZE and return modified pte. (NOTE: TLB insert should use // PAGE_SIZE!) -u64 translate_domain_pte(u64 pteval, u64 address, u64 itir__, u64* logps) +u64 translate_domain_pte(u64 pteval, u64 address, u64 itir__, u64* logps, + struct p2m_entry* entry) { struct domain *d = current->domain; ia64_itir_t itir = {.itir = itir__}; @@ -306,7 +307,7 @@ u64 translate_domain_pte(u64 pteval, u64 address, pteval, itir.itir); } #endif - pteval2 = lookup_domain_mpa(d,mpaddr); + pteval2 = lookup_domain_mpa(d, mpaddr, entry); arflags = pteval & _PAGE_AR_MASK; arflags2 = pteval2 & _PAGE_AR_MASK; if (arflags != _PAGE_AR_R && arflags2 == _PAGE_AR_R) { @@ -319,7 +320,7 @@ u64 translate_domain_pte(u64 pteval, u64 pteval2, arflags2, mpaddr); #endif pteval = (pteval & ~_PAGE_AR_MASK) | _PAGE_AR_R; -} + } pteval2 &= _PAGE_PPN_MASK; // ignore non-addr bits pteval2 |= (pteval & _PAGE_ED); @@ -329,7 +330,8 @@ u64 translate_domain_pte(u64 pteval, u64 } // given a current domain metaphysical address, return the physical address -unsigned long translate_domain_mpaddr(unsigned long mpaddr) +unsigned long translate_domain_mpaddr(unsigned long mpaddr, + struct p2m_entry* entry) { unsigned long pteval; @@ -341,7 +343,7 @@ unsigned long translate_domain_mpaddr(un } } #endif - pteval = lookup_domain_mpa(current->domain,mpaddr); + pteval = lookup_domain_mpa(current->domain, mpaddr, entry); return ((pteval & _PAGE_PPN_MASK) | (mpaddr & ~PAGE_MASK)); } @@ -492,20 +494,6 @@ ____lookup_domain_mpa(struct domain *d, return GPFN_INV_MASK; return INVALID_MFN; } - -unsigned long -__lookup_domain_mpa(struct domain *d, unsigned long mpaddr) -{ - unsigned long machine = ____lookup_domain_mpa(d, mpaddr); - if (machine != INVALID_MFN) - return machine; - - printk("%s: d 0x%p id %d current 0x%p id %d\n", - __func__, d, d->domain_id, current, current->vcpu_id); - printk("%s: bad mpa 0x%lx (max_pages 0x%lx)\n", - __func__, mpaddr, (unsigned long)d->max_pages << PAGE_SHIFT); - return INVALID_MFN; -} #endif // for debug @@ -545,7 +533,8 @@ __tr_entry_print_all(const char* func, i __tr_entry_print_all(__func__, __LINE__, entry, nent) #endif -unsigned long lookup_domain_mpa(struct domain *d, unsigned long mpaddr) +unsigned long lookup_domain_mpa(struct domain *d, unsigned long mpaddr, + struct p2m_entry* entry) { volatile pte_t *pte; @@ -566,6 +555,8 @@ unsigned long lookup_domain_mpa(struct d pte_t tmp_pte = *pte;// pte is volatile. copy the value. if (pte_present(tmp_pte)) { //printk("lookup_domain_page: found mapping for %lx, pte=%lx\n",mpaddr,pte_val(*pte)); + if (entry != NULL) + p2m_entry_set(entry, pte, tmp_pte); return pte_val(tmp_pte); } else if (VMX_DOMAIN(d->vcpu[0])) return GPFN_INV_MASK; @@ -586,6 +577,8 @@ unsigned long lookup_domain_mpa(struct d tr_entry_print_all(¤t->arch.dtlb, 1); #endif + if (entry != NULL) + p2m_entry_set(entry, NULL, __pte(0)); //XXX This is a work around until the emulation memory access to a region // where memory or device are attached is implemented. return pte_val(pfn_pte(0, __pgprot(__DIRTY_BITS | _PAGE_PL_2 | _PAGE_AR_RWX))); @@ -595,7 +588,7 @@ unsigned long lookup_domain_mpa(struct d #if 1 void *domain_mpa_to_imva(struct domain *d, unsigned long mpaddr) { - unsigned long pte = lookup_domain_mpa(d,mpaddr); + unsigned long pte = lookup_domain_mpa(d, mpaddr, NULL); unsigned long imva; pte &= _PAGE_PPN_MASK; diff -r 60d6edcdfde3 -r 652ed19e0893 xen/arch/ia64/xen/vcpu.c --- a/xen/arch/ia64/xen/vcpu.c Mon Jun 12 16:54:12 2006 +0900 +++ b/xen/arch/ia64/xen/vcpu.c Mon Jun 12 18:09:07 2006 +0900 @@ -29,7 +29,6 @@ extern void setfpreg (unsigned long regn extern void setfpreg (unsigned long regnum, struct ia64_fpreg *fpval, struct pt_regs *regs); extern void panic_domain(struct pt_regs *, const char *, ...); -extern unsigned long translate_domain_mpaddr(unsigned long); extern IA64_BUNDLE __get_domain_bundle(UINT64); typedef union { @@ -1978,18 +1977,24 @@ IA64FAULT vcpu_itc_d(VCPU *vcpu, UINT64 { unsigned long pteval, logps = itir_ps(itir); BOOLEAN swap_rr0 = (!(ifa>>61) && PSCB(vcpu,metaphysical_mode)); + struct p2m_entry entry; if (logps < PAGE_SHIFT) { printf("vcpu_itc_d: domain trying to use smaller page size!\n"); //FIXME: kill domain here while(1); } +again: //itir = (itir & ~0xfc) | (PAGE_SHIFT<<2); // ignore domain's pagesize - pteval = translate_domain_pte(pte, ifa, itir, &logps); + pteval = translate_domain_pte(pte, ifa, itir, &logps, &entry); if (!pteval) return IA64_ILLOP_FAULT; if (swap_rr0) set_one_rr(0x0,PSCB(vcpu,rrs[0])); vcpu_itc_no_srlz(vcpu,2,ifa,pteval,pte,logps); if (swap_rr0) set_metaphysical_rr0(); + if (p2m_entry_retry(&entry)) { + vcpu_flush_tlb_vhpt_range(ifa & ((1 << logps) - 1), logps); + goto again; + } return IA64_NO_FAULT; } @@ -1997,6 +2002,7 @@ IA64FAULT vcpu_itc_i(VCPU *vcpu, UINT64 { unsigned long pteval, logps = itir_ps(itir); BOOLEAN swap_rr0 = (!(ifa>>61) && PSCB(vcpu,metaphysical_mode)); + struct p2m_entry entry; // FIXME: validate ifa here (not in Xen space), COULD MACHINE CHECK! if (logps < PAGE_SHIFT) { @@ -2004,13 +2010,18 @@ IA64FAULT vcpu_itc_i(VCPU *vcpu, UINT64 //FIXME: kill domain here while(1); } +again: //itir = (itir & ~0xfc) | (PAGE_SHIFT<<2); // ignore domain's pagesize - pteval = translate_domain_pte(pte, ifa, itir, &logps); + pteval = translate_domain_pte(pte, ifa, itir, &logps, &entry); // FIXME: what to do if bad physical address? (machine check?) if (!pteval) return IA64_ILLOP_FAULT; if (swap_rr0) set_one_rr(0x0,PSCB(vcpu,rrs[0])); vcpu_itc_no_srlz(vcpu, 1,ifa,pteval,pte,logps); if (swap_rr0) set_metaphysical_rr0(); + if (p2m_entry_retry(&entry)) { + vcpu_flush_tlb_vhpt_range(ifa & ((1 << logps) - 1), logps); + goto again; + } return IA64_NO_FAULT; } @@ -2040,10 +2051,14 @@ IA64FAULT vcpu_fc(VCPU *vcpu, UINT64 vad UINT64 mpaddr, paddr; IA64FAULT fault; +again: fault = vcpu_tpa(vcpu, vadr, &mpaddr); if (fault == IA64_NO_FAULT) { - paddr = translate_domain_mpaddr(mpaddr); + struct p2m_entry entry; + paddr = translate_domain_mpaddr(mpaddr, &entry); ia64_fc(__va(paddr)); + if (p2m_entry_retry(&entry)) + goto again; } return fault; } diff -r 60d6edcdfde3 -r 652ed19e0893 xen/include/asm-ia64/domain.h --- a/xen/include/asm-ia64/domain.h Mon Jun 12 16:54:12 2006 +0900 +++ b/xen/include/asm-ia64/domain.h Mon Jun 12 18:09:07 2006 +0900 @@ -12,10 +12,34 @@ #include #include +struct p2m_entry { + volatile pte_t* pte; + pte_t used; +}; + +static inline void +p2m_entry_set(struct p2m_entry* entry, volatile pte_t* pte, pte_t used) +{ + entry->pte = pte; + entry->used = used; +} + +static inline int +p2m_entry_retry(struct p2m_entry* entry) +{ + //XXX see lookup_domian_pte(). + // NULL is set for invalid gpaddr for the time being. + if (entry->pte == NULL) + return 0; + + return (pte_val(*entry->pte) != pte_val(entry->used)); +} + extern void domain_relinquish_resources(struct domain *); /* given a current domain metaphysical address, return the physical address */ -extern unsigned long translate_domain_mpaddr(unsigned long mpaddr); +extern unsigned long translate_domain_mpaddr(unsigned long mpaddr, + struct p2m_entry* entry); /* Flush cache of domain d. If sync_only is true, only synchronize I&D caches, diff -r 60d6edcdfde3 -r 652ed19e0893 xen/include/asm-ia64/mm.h --- a/xen/include/asm-ia64/mm.h Mon Jun 12 16:54:12 2006 +0900 +++ b/xen/include/asm-ia64/mm.h Mon Jun 12 18:09:07 2006 +0900 @@ -149,8 +149,6 @@ extern unsigned long max_page; extern void __init init_frametable(void); void add_to_domain_alloc_list(unsigned long ps, unsigned long pe); - -extern unsigned long gmfn_to_mfn_foreign(struct domain *d, unsigned long gpfn); static inline void put_page(struct page_info *page) { @@ -428,7 +426,8 @@ extern void __assign_domain_page(struct extern void __assign_domain_page(struct domain *d, unsigned long mpaddr, unsigned long physaddr, unsigned long flags); extern void assign_domain_page(struct domain *d, unsigned long mpaddr, unsigned long physaddr); extern void assign_domain_io_page(struct domain *d, unsigned long mpaddr, unsigned long flags); -extern unsigned long lookup_domain_mpa(struct domain *d, unsigned long mpaddr); +struct p2m_entry; +extern unsigned long lookup_domain_mpa(struct domain *d, unsigned long mpaddr, struct p2m_entry* entry); extern void *domain_mpa_to_imva(struct domain *d, unsigned long mpaddr); #ifdef CONFIG_XEN_IA64_DOM0_VP @@ -436,7 +435,6 @@ extern unsigned long assign_domain_mach_ extern unsigned long assign_domain_mach_page(struct domain *d, unsigned long mpaddr, unsigned long size, unsigned long flags); int domain_page_mapped(struct domain *d, unsigned long mpaddr); int efi_mmio(unsigned long physaddr, unsigned long size); -extern unsigned long __lookup_domain_mpa(struct domain *d, unsigned long mpaddr); extern unsigned long ____lookup_domain_mpa(struct domain *d, unsigned long mpaddr); extern unsigned long do_dom0vp_op(unsigned long cmd, unsigned long arg0, unsigned long arg1, unsigned long arg2, unsigned long arg3); extern unsigned long dom0vp_zap_physmap(struct domain *d, unsigned long gpfn, unsigned int extent_order); @@ -445,7 +443,7 @@ extern unsigned long dom0vp_add_physmap( extern volatile unsigned long *mpt_table; extern unsigned long gmfn_to_mfn_foreign(struct domain *d, unsigned long gpfn); -extern u64 translate_domain_pte(u64 pteval, u64 address, u64 itir__, u64* logps); +extern u64 translate_domain_pte(u64 pteval, u64 address, u64 itir__, u64* logps, struct p2m_entry* entry); #define machine_to_phys_mapping mpt_table #define INVALID_M2P_ENTRY (~0UL) @@ -491,7 +489,7 @@ set_gpfn_from_mfn(unsigned long mfn, uns gmfn_to_mfn_foreign((_d), (gpfn)) #define __gpfn_invalid(_d, gpfn) \ - (lookup_domain_mpa((_d), ((gpfn)<