sched: Fix hotplug vs ttwu race
Valntin noted that ttwu uses is_cpus_allowed() with ->pi_lock held,
which isn't serialized against setting balance_push_set(), which uses
rq->lock.
Therefore it is possible for a remote ttwu to fail to observe the
balance_push setting and still wake a task to the CPU that's going
down.
Ensure all TTWU (which is ran with preemption disabled) observe the
balance_push_set() before we wait by employing
get_state_synchronize_rcu() + cond_synchronize_rcu() to avoid adding a
second full synchornize_rcu() to sched_cpu_deactivate.
Furthermore, we must also avoid the wakeup being split in two parts
due to TTWU_QUEUE, we could fail to observe the balance_push_set() for
ttwu, and get the IPI after balance_hotplug_wait(). To avoid this,
never queue for !active CPUs.
Signed-off-by:
Peter Zijlstra (Intel) <peterz@infradead.org>
Loading
Please register or sign in to comment