Re: Scheme is too complicated

[Maciej Stachowiak <mstachow@mit.edu>]

ian@bickiia.earlhall.earlham.edu writes:
> Jay Glascoe writes:
> ...
> > > Hmm.  I think your definition of slow and mine are different.  Could
> > > you clarify what you mean when you say 'slow'?  How do you account for
> > > the performance of Stalin?
> ...
> > the functional guy is sleek and pretty, but those "vector->list" and
> > "append" bits are performance killers. 
> 
> I think the issue is that a functional programming style makes for a
> lot of copying (with some associated garbage collection).  Computers
> work much more quickly on in-place algorithms, where no copying is
> necessary.  And, deep down, (all?) computers are imperative.
> 

Actually, writing and reading to the same memory a lot can be more
costly on some architectures than the write-once-read-many (modulo GC)
behavior of some functional programs. In particular, in a shared
memory multiprocessor architecture it is nice to avoid writing a lot
to memory that someone else might be reading a lot.

> However, programming in a functional style can lead to great
> optimizations, because the behavior can be predicted before runtime
> much better.  I imagine Stalin uses this to its advantage(?)
> Distributed programming also benefits from this -- I'm sure there are
> other benefits as well.
> 
> But Guile isn't compiled or optimized, so the advantages of functional 
> programming are mostly stylistic/aesthetic.
> 

Well, I think set! is still moderately expensive in Guile - if you can
get away without unnecessary copying _or_ set!, without using non-tail
recursion (see my posted versions of Jay's code) you still win.

> 
> As to the size of C/Perl/Guile programs, I think a lot of it has to do
> with the libraries.  I find the various C libraries to not be able to
> integrate themselves into the language very nicely at all.  Perl does
> this much better, not to mention having so much functionality built
> right into the language.  I also find that shell scripting (who's
> library is the programs on my machine) works very nicely for me.  And
> what could be shorter than an Awk script for a job which Awk is good
> at?
> 

I agree with you that good libraries are essential to compact code
size - but a key aspect of this is how well the language supports
powerful abstractions. Perl, Python, Guile, etc all support
abstraction much better than C, but I think a Scheme-based language
can potentially do even better.


> 
> But now I've gotten to thinking about libraries.
> 

I'm glad people are thinking about them. I have thinking about them a
lot too.

> I'm not sure what the Scheme environment really is -- it's stuck in
> the elegance of its mathematical roots.  But *practical* elegance is
> to make a system whereby a programmer can work in a rich environment
> while maintaining certain things:
> 
>   * Readable programs: both a matter of making all function calls as
>     meaningfully named as possible -- where namespace-declarations
>     *aren't* meaningful, but short descriptive names are; making the
>     library functions fit well into the structure of the code the
>     programmer is writing -- avoiding wrapping everything in lambdas
>     and allowing the code to be structured as best fits the problem.
> 

Can you give me an example of what you mean by "wrapping everything in
lambdas" and how it is bad? Schemers tend to be big on passing code to
control-flow constructs by passing it as procedures, which I think is
quite logical in many cases.

>   * Transparent code: all aspects of the code can be inspected and
>     understood, including the library functions.
> 

Mikael suggested a while ago that we should set things up so that
calling procedure-source on a primitive that only is a primitive for
performance reasons, i.e. it can be written in Scheme in terms of
other things, should produce equivalent Scheme code. We could do this
by storing the Scheme implementations on disk somewhere and reading
them on demand.

>   * Mnemonic: a rich environment means having a lot of functions.
>     Remembering each one and the forms it takes will get difficult
>     without some means of managing it.  Keyword arguments are (IMHO) a 
>     great way of doing this.
> 

In scwm, nearly every library module I have uses keyword arguments. I
agree with you that it is a truly excellent interface for many things.

[...snip documentation section...]


> There's two issues: what could a good library system in Guile fix, and
> what could language extensions in Guile fix.  I wouldn't stand behind
> language extensions -- when does it stop being Scheme and become some
> little brother of Common Lisp?  But maybe there's a way to do these
> things in Scheme and I just don't see it.
>

By language extensions do you mean syntactic extensions or just adding
more primitive procedures? I agree that syntactic extensions are
generally to be frowned upon. OTOH macros being used to implement
obvious extensions to the syntax that are just wrappers around
existing syntax is, IMO, pretty benign.
 
> 
> Readability: Scheme has a way of leading to long expressions.  When
> you get down to it, any functional procedure will be made up of one
> (long) expression.  That's hard for humans to parse.  I like the
> highlighting that DrScheme does -- when you are at a open or close
> parenthesis, it highlights (without being obnoxious) the expression
> that is enclosed by that parenthesis.  The passive nature of the
> highlighting is what makes it nice.  (The static code analysis that
> DrScheme does is also nice, but I'm not sure it's helpful enough to
> warrant the complexity of implementing the interface)

Hmm, does any Emacs expert know if this can this be done with Emacs's
font-lock-mode?

> 
> Along the library side, it would be nice to think about the naming
> system for all the library procedures and coming up with a consistent,
> concise, and meaningful system for naming (this also helps mnemonics).
> I wish I could give a concrete example, but I haven't really worked
> with any complex libraries in Scheme.  However, what I haven't seen
> that I think is important is that Guile should be viewed as including
> not only the core language, but a set of modules which provide
> commonly-used functionality.  These are too essential to the language
> and it's practical use too simply be add-ons.

Well, I hope the normal Guile package/installation comes with a bunch
of useful extensions, but bundled as modules rather than built into
the library.

> 
> A rich set of datatypes could also be a powerful addition.  Make
> dictionaries, not hash tables -- mostly that's just a matter of
> terminology, but I think it's important.  As a programmer, I don't
> care about the fact that hash tables use hashing, I care about the
> fact that hash tables are a collection of values indexed by keys.
> 

There are a lot of data structures that are a collection of values
indexed by keys. Association lists and hash tables, to name two useful
ones, both have their place. Association lists are mainly useful in
that they can be treated as lists directly, and Scheme's list
operators can be used to do a lot of stuff quickly and easily. Hash
tables are nice in that they have constant-time access.

> If/when debugging in Guile becomes pleasant, allow people to actually
> use these reference implementations so that they can access everything
> their code does to a fine degree of granularity.  For instance, if you
> provide a bad hash function to a hash-table, the error (or worse: the
> lack of error) might pop up inside some hash-table function.  Being
> able to inspect the innards of that function will make it much easier
> to find the bug.

Are you referring to bugs in user code or bugs in system code? If the
bug is in system code, debugging a Scheme version of C code might not
help unless the Scheme code and the C code happen to have the same
bug.

> 
> Mnemonics: As with readability, some deep thought about the naming
> system will help.  Also, keeping the number of functions to a minimum
> is also important.  I think keyword arguments could be of great help
> here.  They allow a very general function (with lots of parameters) to
> masquerade as a less general function.  Also, argument order is almost 
> always arbitrary (and thus hard to remember): keywords help solve
> this.  I think they could be important enough to be used heavily in
> Guile, even if it is (in effect) a language extension.

Well, it's easy enough to provide support for optional arguments in
terms of more primitive syntax. I made it a set of `defmacro'-style
macros.

There would be advantages to putting support for keyword and optional
arguments directly in the interpreter though:

* Standard forms would all take optional arguments; no need to use
funny whatever* versions (I could actually make my module do this by
redefinind lambda as lambda* and so on, but this seems highly
unsatisfying and might break in unexpected ways).

* Better performance.

* Less psychological reluctance to use it in core libraries where it
would be useful.

> An object system could also help.  With dynamic typing, it seems so
> close... all the compromises are there (space, time, lack of static
> analysis), but the benefits are not completely reaped.  An object
> system allows a function which can work conceptually for many
> datatypes to actually do so, without cond statements (which are
> ugly).

Dynamically typed languages _can_ be mostly statically analyzed. A
good type inference engine can infer the type of almost any Scheme
expression (the general problem is uncomputable but I think the losing
cases are rare enough that you can just provide some fallback for such
cases). 

That's not really relevant to your point though. I agree that a good
object system is key. I actually have a pretty spiffy Guile project on
the back burner waiting for a good standard object system (I might
just break down and use the interpreted tinyclos to prototype).

> I guess, to me, it's a question of Guile being just a language (which
> it is) or an entire environment (which it isn't -- at least yet).
>
> My, I wrote a lot more than I was expecting to.

Your comments were very insightful however. As Guile moves towards
becoming an increasingly useful tool, we need to think hard about how
to make it useful for practical purposes while not losing the basic
elegance of Scheme.

 - Maciej