Re: [PATCH] Relay CPU Hotplug support

[Gautham R Shenoy <ego at in dot ibm dot com>]

Hi Andrew,

While we are at this per-subsystem cpuhotplug "locking", here's a
proposal that might put an end to the workqueue deadlock woes.

I'm yet to cook up a patch for this, but here's the idea in brief.

On Wed, Dec 20, 2006 at 11:23:50PM -0800, Andrew Morton wrote:
> to the relay driver.  Why do that - you don't own cpu_online_map (but you
> do get some notifications when it wants to change, that's all).

How about: Let each hot-cpu-aware subsystem maintain it's own 
online_cpus mask. Thus we can eliminate the global online_cpus mask 
and also have a clear picture of what data the per-subsystem mutexes
are protecting :)

In kenel/cpu.c

_cpu_down()
{
	send_all_pre_cpu_down_notifications();
	.
	.
	.
	send_notifications_to_lock_per_subsystem_mutexes();
	__stop_machine_run();
	send_notifications_to_update_per_subsystem_online_cpus_mask();
	send_notifications_to_release_per_subsystem_mutexes();
	.
	.
	.
	send_all_post_cpu_down_notifications();
	
}

Ditto for _cpu_up().

This will not only reduce the lock-contention , but will also 
allow the pre/post hotplug notifications handlers to make calls to 
function which are cpu-hotplug-aware (like create_workqueue,
destroy_workqueues etc) without ending up in a recursive deadlock
as the persubsystem mutexes would have been released by then. 

And since we are sending notifications to update 
per_subsystem_cpus_mask before sending the
post_cpu_hotplug_notifications, the post_notification handlers
will be executing with the consistent value of the online_cpus mask.

Does anybody see a problem with this "update_now-cleanup_later" 
approach ?

Thanks and Regards
gautham.
-- 
Gautham R Shenoy
Linux Technology Center
IBM India.
"Freedom comes with a price tag of responsibility, which is still a bargain,
because Freedom is priceless!"