Re: [ECOS] some issues relating to current_thread

[Nick Garnett <nickg at ecoscentric dot com>]

Brij Bihari Pandey <fuzzhead012@yahoo.com> writes:

> In Linux, we are moving from coarse grain locking to
> fine grained locking, but looking at schedlock
> controlled approach for protection of state variables
> in semaphore/mutex/... and the schedlock
> implementation in eCos the coarseness is surprising
> wrt "SMP implementation", more so because eCos is
> supposed to be a RTOS. Please correct me if I am wrong
> in this understanding of mine about eCos.
>

Comparing eCos to Linux is not really valid. The original coarse
grained locking in Linux locked everything in the kernel: scheduler,
file system, network stack, VM, device drivers etc. In eCos the
scheduler lock only locks the scheduler data structures, so it
controls a much smaller domain than the coarse Linux lock. File
systems, network stacks and device drivers already use mutexes to
control concurrency.

More recent versions of Linux implement a per-thread nested preemption
lock and spinlocks always acquire it.  The equivalent in eCos would be
to implement a per-CPU scheduler lock and add spinlocks to all data
structures. This was not done because one of the goals of adding SMP
to eCos was to avoid making major changes to the code base. It would
have needed the expansion of each data structure with a spinlock,
extra code and extra complexity to avoid deadlocks and races. Since
most eCos targets are uniprocessors, and will remain so for the
forseeable future, it was not considered acceptable to make these pay
for what is still essentially an experimental feature.

eCos SMP is aimed at very modest SMP systems, 2 or 4 CPUs at most. The
level of contention and the delay to each CPU will be small and it
pays to keep things as simple as possible. Linux SMP needs to cope
with big servers with dozens of CPUs, and needs to maximize kernel
throughput, so the approach needs to be slightly different.

> what I gather from the source code of ecos (current
> cvs) that the schedlock approach not only protects the
> state variable of (for example) a particular mutex say
> mutex1 from simultaneous access by two threads trying
> to lock the same mutex1 but also --
> because of this schedlock approach in locking, one
> thread trying to lock mutex1 on one CPU, will
> unnecessarily get delayed in locking process because
> another thread is trying to lock mutex2 on another
> CPU.
>

I did look closely at implementing mutexes with a spinlock or a
compare-and-swap mechanism. While this could be made to work for
uncontended mutexes that do not implement any priority inversion
protocol, it does not work for protected mutexes. It is much more
important for real time performace to handle priority inversion
correctly than to be able to acquire and release uncontended locks
quickly.

> Won't it be better to use spinlock variables to
> protect state variables of mutex/semaphore/.. in SMP
> scenario, rather than schedlock?

In theory, yes. In practice it is such a fundamental rewrite of lots
of code to achieve very little for most targets that it will probably
never happen. If, in 5 years time, it turns out that every cellphone
and set-top-box is running an 8 CPU SMP system then we might
reconsider :-)


-- 
Nick Garnett - eCos Kernel Architect
http://www.eCosCentric.com/


-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss