newLISP - Differences to Other LISPs

puts the fun

back in Lisp

Comparison to Common Lisp and Scheme

This page clarifies some differences between newLISP and Common Lisp and Scheme. Read the About page to find out more about newLISP's unique approach to LISP.

The purpose of this comparison is not to point out that one language is better than the other. Different styles of programming appeal to different personalities. Each approach has different strengths and weaknesses. The notion that there could be designed a single, all-purpose language is naive. The author of newLISP uses 'C' and Java concurrently with newLISP. Others use a different set of tools tailored to the character of their projects and style of programming.

newLISP's aesthetic, casual and flexible style has attracted not only the traditional computer programmer but many people from other professions. Many contributions to the design of newLISP come from people who are not primarily programmers. For many, newLISP is not only a programming language but also a tool for modeling and organizing creative thought.

Scripting language versus compiled language

newLISP is a scripting language designed not to be compiled but to be fully dynamic and introspective. Many of the differences to other LISPs result from this distinction.

Both approaches have their place in modern computing. For a discussion of this subject see here:

In Praise of Scripting: Real Programming Pragmatism.

For more about history, critique and other aspects of Lisp see here.

Openness and Transparency

newLISP is completely open. There are no hidden states. All language and data objects are first class. Although newLISP initially compiles program source into an internal s-expression-tree made of LISP-cells, everything can be serialized back to human readable text all the time. This includes the symbol environments in contexts (name-spaces) and lambda expressions. This openness eases interactive working and debugging and eases the understanding of the language. newLISP programs are fully self-reflective and can inspect and modify itself at all times.

newLISP is network aware regarding file resources in a transparent way. In most instances where files are used, URLs can be used instead. Files can be read, written and appended, programs can be loaded or saved using the same code for local or network access. This facilitates writing distributed applications.

Function application as in Scheme

Unlike Common Lisp, newLISP and Scheme evaluate the operator part of an expression before applying the operator to its arguments.

Lambda expressions

In newLISP, lambda expressions are constants evaluating to themselves. They are a subtype of the list data type, a first-class data object that can be manipulated like any other list. In Common Lisp and Scheme, lambda expressions, when evaluated, return a special function data type forming a lexical closure after binding its free variables to the current environment.

In newLISP, binding of free variables in lambda expressions takes place during application of the lambda expression without creating a closure.

Lambda expressions in newLISP can be treated as data all the time, even after being bound in a function definition:

(define (foo x) (+ x x)) => (lambda (x) (+ x x))

(last foo) => (+ x x)

Other LISPs use lambda closures to create functions with state. Maintaining state is a precondition for a programming language to allow object-oriented programming. In newLISP, lexically closed contexts (namespaces) can be used to write state-full functions. Like lambda expressions, contexts in newLISP are first-class objects. Contexts can be created and destroyed during runtime, passed as parameters, and referred to by symbols.

See the page: Closures and Contexts for a comparison of Scheme closures and newLISP contexts and see also the definition of def-static.

One symbol space

In newLISP and Scheme, variables, primitives, and user-defined functions share the same symbol space. In Common Lisp, function symbols and variable symbols each use a dedicated name space. This is why Common Lisp function symbols sometimes must be prefixed with the sharp quote #'. Symbols in newLISP are case-sensitive.

Dynamic scoping inside isolated namespaces

newLISP is sometimes criticized for using dynamic scoping and fexprs. These critics overlook that newLISP's namespace contexts shield function symbols from the traditional disadvantages of dynamic scoping and fexprs.

In newLISP, all variables are dynamically scoped by default. However, by defining a function in its own context, static/lexical scoping can be achieved. Parameter names used in fexprs are pushed with their value on a stack upon function entry and are restored upon function return. In newLISP, several functions and data can share a namespace. By enclosing functions in their own namespace, a lexical closure- like mechanism is achieved. Common Lisp and Scheme are lexically scoped by default and use lambda expressions as the closure mechanism. Common Lisp also offers special variables for dynamic scoping.

The problems of free variables in dynamic scoping can be avoided. In the rare cases when free variables must be used, you can partition code into namespace modules for easier control over free variables. You can then exploit the advantages of dynamic scoping. With dynamic scoping inside lexically-closed namespaces, newLISP combines the best of both scoping worlds.

newLISP has no funarg problem because it follows a simple rule: variables always show the binding of their current environment. When expressions with local variables are entered, newLISP saves the current variable state on a stack and restores it on exit of that expression. In newLISP, not only are function parameters and variables declared with let expressions local, loop variables in all looping expressions are local too.

The lisp cell and `cons`

In Common Lisp and Scheme, the cdr part of the lisp cell can be used to contain another LISP object, in which case we have a dotted pair. In newLISP, there are no dotted pairs. Instead, each newLISP cell contains one object and a pointer to another object if the cell is part of a list. As a result, cons behaves differently in newLISP than in other LISPs.

   ;; Common Lisp and Scheme
   (cons 'a 'b) => (a . b)   ; a dotted pair

   [a | b]

   ;; newLISP
   (cons 'a 'b) => (a b)     ; a list

   [ ]
    \
    [a] -> [b]

   ;; LISP cells in newLISP
   (+ 2 3 (* 4 3))

   [ ]
    \
    [+] -> [2] -> [3] -> [ ]
                          \
                          [*] -> [4] -> [3]

Function arguments are optional

In newLISP, all arguments to a user-defined function are optional. Unassigned argument variables will assume the value nil inside the function.

Implicit symbol creation

Logically, there are no unbound or non-existing symbols in newLISP. Any unbound or non-existing symbol is created and bound to nil in the current namespace when it is first seen by newLISP.

`nil` and `true` are Boolean constants

nil and true are Boolean constants in newLISP. In Common Lisp, nil has an additional role as a list terminator:

   ;; newLISP
   (cons 'x nil) => (x nil)

   ;; Scheme
   (cons 'x #f)  => (x . #f)

   ;; Common Lisp
   (cons 'x nil) => (x)

Scheme has the two Boolean constants #t and #f for true and false.

In newLISP first and in Scheme car will throw and error when used on and empty list. Common Lisp car will return nil in that case:

;; newLISP
(first '()) => error

;; Scheme
(car '()) => error

;; Common Lisp
(car '()) => nil

One Reference Only (ORO) memory management

In newLISP, every object is only referenced once (ORO), with the exception of symbols and contexts. newLISP's ORO rule allows automatic, stack-based, on the go memory management, without the problems of traditional garbage collection algorithms used in other scripting languages. newLISP's ORO memory management is faster and uses less resources.

newLISP passes parameters by value-copy and stores intermediate results on a result stack. The memory created for intermediate results gets recycled after function return. Like traditional garbage collection, ORO memory management frees the programer from dealing with memory allocation and reallocation.

To avoid copying data objects when passing by value-copy, they can be enclosed in context objects and passed by reference. The following code snippet shows reference passing using a namespace default functor:

   (define (modify data value)
      (push value data))

   (set 'reflist:reflist '(b c d e f g))

   (modify reflist 'a) ; passed by reference

   reflist:reflist => (a b c d e f g)

newLISP's automatic memory management is fully transparent to the programmer but faster and lighter on resource requirements when compared to classic garbage collection algorithms. Because ORO memory management is synchronous, newLISP code has constant and repeatable execution time. Programming languages using traditional garbage collection display sudden delays and pauses.

The combination of newLISP's value-copy passing style and unique memory management make it the fastest interactive (non-compiled) LISP available and one of the fastest scripting languages available in general. As shown above, reference style passing is available, too. For built-in functions reference passing is the default.

As a by-product of newLISP's ORO memory management, only the equal sign = is required to test for equality. Common Lisp requires eq, eql, equal, equalp, =, string=, string-equal, char=, and char-eq for equality tests of expressions, data types, identical objects, and referenced objects.

Fexpr macros and rewrite macros

In newLISP, special forms are created using fexprs defined with define-macro. Common Lisp macros use template expansion and compilation to create special forms. Special forms don't evaluate their arguments or evaluate only under special conditions. In newLISP, fexprs are still called macros because they serve a similar purpose to macros in other LISP dialects: they allow the definition of special forms.

Fexpr's created with define-macro completely control when arguments are evaluated. As a result, newLISP macros can work like built-in special forms:

   (define-macro (my-setq x y) (set x (eval y)))

   ;; as hygienic macro avoiding variable capture

   (define-macro (my-setq) (set (args 0) (eval (args 1))))

newLISP can initiate variable expansion explicitly using expand and letex:

   (define (raise-to power)
      (expand (fn (base) (pow base power)) 'power))

   (define square (raise-to 2))
   (define cube (raise-to 3))

   (square 5) => 25
   (cube 5)   => 125

Variable expansion can be used to capture the state of free variables. See an application of this here: The Why of Y in newLISP. newLISP frequently combines define-macro and template expansion using expand or letex.

Variable capture in fexprs in newLISP can be avoided by either enclosing them in a namespace, or by using the args function to retrieve passed parameters, i.e. (args 0) for the first (args 1) for the second and so forth. Either way, the resulting fexpr's are completely hygienic with no danger of variable capture.

In version 10.1.6, newLISP introduced rewrite-expansion macros in a loadable module. Since version 10.6.0, the same functionality is available with a built-in, native macro function working the same way:

   ; register a macro template

   (macro (cube X) (pow X 3))

   ; during code loading the macros are expanded.

   (cube 3) => 27

The macros expansion facility hooks between source reading/translation and the evaluation process. In the example every occurrence of (cube n) would be translated into (pow n 3). This way the overhead of fexprs is avoided.

Impicit Indexing

newLISP features implicit indexing. This is a logical extension of LISP evaluation rules overloading lists, strings or numbers with the indexing functionality available from built-in list and string functions like nth, rest or slice, i.e.:

   (set 'myList '(a b c (d e) f g))

   ; using nth
   (nth 2 myList) => c
   ; with address vector
   (nth '(3 1) myList) => e
   (nth '(3 0) myList) => d

   ; using implicit indexing
   (myList 2) => c
   (myList 3 1) => e
   (myList -3 0) => d
   ; with address vector
   (set 'v '(3 1))
   (myList v) => e

   ; implicit rest, slice
   (1 myList) => (b c (d e) f g)
   (-3 myList) => ((d e) f g)
   (1 2 myList) => (b c)

Using implicit indexing is optional. In many instances it increases speed and readability.

Links

Automatic Memory Management in newLISP

Expression evaluation, Implicit Indexing, Contexts and Default Functors in the newLISP Scripting Language

Closures and Contexts in newLISP

The Why of Y in newLISP

History, Critique and Aesthetic Aspects of Lisp several links, several authors

Scripting: Higher Level Programming for the 21st Century John K. Ousterhout

In Praise of Scripting: Real Programming Pragmatics Ronald P. Loui

About the newLISP the author Lutz Mueller.