When flush tlb mask, we need consider the cpu_online_map.

The same is true for EPT flushes.

We noticed sometime system hang on cpu online/offline stress test. The
reason is because flush_tlb_mask from __get_page_type is deadloop.

This should be caused by a small windows in cpu offline.  The
cpu_online_map is changed and the interrupt is disabled at
take_cpu_down() for the to-be-offline CPU.

However, the __sync_lazy_execstate() called from idle_task_exit() in
the idle_loop() for the to-be-offline CPU. At that time, the
stop_machine_run is finished already, and __get_page_type may be
called in other CPU before the __sync_lazy_execstate().

Thanks Jan pointing out issue in my original patch.

Signed-off-by: Jiang, Yunhong <yunhong.jiang@intel.com>

2 files changed, 5 insertions, 2 deletions

diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 62c923a0a7..b54dc8e82c 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -1234,7 +1234,9 @@ void ept_sync_domain(struct domain *d)
      * the ept_synced mask before on_selected_cpus() reads it, resulting in
      * unnecessary extra flushes, to avoid allocating a cpumask_t on the stack.
      */
-    d->arch.hvm_domain.vmx.ept_synced = d->domain_dirty_cpumask;
+    cpus_and(d->arch.hvm_domain.vmx.ept_synced,
+             d->domain_dirty_cpumask, cpu_online_map);
+
     on_selected_cpus(&d->arch.hvm_domain.vmx.ept_synced,
                      __ept_sync_domain, d, 1);
 }
diff --git a/xen/arch/x86/smp.c b/xen/arch/x86/smp.c
index b2f1eadfad..2a01923900 100644
--- a/xen/arch/x86/smp.c
+++ b/xen/arch/x86/smp.c
@@ -228,7 +228,8 @@ void flush_area_mask(const cpumask_t *mask, const void *va, unsigned int flags)
     if ( !cpus_subset(*mask, *cpumask_of(smp_processor_id())) )
     {
         spin_lock(&flush_lock);
-        cpus_andnot(flush_cpumask, *mask, *cpumask_of(smp_processor_id()));
+        cpus_and(flush_cpumask, *mask, cpu_online_map);
+        cpu_clear(smp_processor_id(), flush_cpumask);
         flush_va      = va;
         flush_flags   = flags;
         send_IPI_mask(&flush_cpumask, INVALIDATE_TLB_VECTOR);