Teeny Scheme Manual (Hairspray Queen Enemy+)

Teeny Scheme is a fork of TinyScheme with a focus on maintaining the original’s simplicity. (Even if only in spirit.) And allowing easier extensibility, maintenance, and use. This manual focuses of these. The goal is to help you understand how Teeny Scheme works and make you confident in tailoring it for your needs. That’s the best I can hope for.

Table of Contents

• Usage
  • Environment Variables
  • Compile-Time Defines
• Standard Compliance
  • Feature Identifiers
  • Macro System
  • Syntactic Extensions
  • Typing
• Non-standard Goodies
• Provided APIs
  • tgscheme.h—type-generic Scheme API
• Extending Teeny Scheme
  • Adding a New Data Type
    • Table of Existing Type Abbreviations
    • GC-ing the New Type
    • Adding Embedded Interface Entries
  • Adding a New Opcode
    • Helpers for Opcode Writing
  • Adding a New Base Syntax Form
  • Adding an Extension Library in C
    • Foreign Pointers
  • Extending Parts of Implementation with Hooks

Usage #

Teeny Scheme can be used both as an embedded interpreter for cheap Scheme evaluation. And as a standalone Scheme implementation with REPL, libraries, and script execution.

When used as an embedded interpreter, Teeny Scheme can be linked to an executable as a dynamic library or bundled as a static library. See the Provided APIs section for how to use it But there’s not much to say on that, as I didn’t really explore embedded use.

REPL use, however, works well enough. Teeny Scheme can be called on CLI in a variety of ways:

teenyscheme [-v] [-V] [-h] # 1
teenyscheme # 2
teenyscheme [<file1> file2> ...] # 3
teenyscheme [<file1> <file2> ...] -1 <file> [<arg1> <arg2> ...] # 4
teenyscheme [<file1> <file2> ...] -c <expr> [<arg1> <arg2> ...] # 5

1. Show version (-v and -V) or help (-h)
2. Start a REPL with a subset of (scheme r5rs) loaded. See environment variables for how to load more libraries.
3. Run the provided files as scripts and exit.
4. Load the initial files, then evaluate the provided file with (args) parameter bound to the args listed after it.
   • The -1 flag is intended for shell shebangs. If you specify #!/somewhere/teenyscheme -1 then teenyscheme will be called to process the file. For example, the following script echoes the Scheme list of its arguments:

     #!/somewhere/teenyscheme -1
     (display (args))
     

5. Same as (3), but evaluate an expression expr with (args) bound to args after it.

Return codes are:

0

Normal exit

1

Help printed

2

Cannot initialize

7

Eval limit was reached.

Anything else

User-provided quit code.

Environment Variables #

Run-time Teeny Scheme is mainly configured through environment variables. (Compile-time macro defines control the initial configuration.)

TEENYINITFILE (defaults to init.scm in the current directory)

Path to an init file. Given that the default Teeny Scheme comes with only a subset of R5RS, init file is needed to get it at least up to R7RS’ (scheme r5rs) or (aspirationally) (scheme base). Use with caution, defaults should be fine. To add more code, check TEENYLIBPATH instead. Also note that default build of Teeny loads R5RS-compatible init it is compiled with whenever the init file cannot be found.

TEENYLIBPATH (defaults to /usr/local/lib/teenyscheme, can be pre-set in config.h)

Path to a directory with loadable R7RS libaries. Directory structure should reflect the library name. So (srfi 160 f64) must be defined in $TEENYLIBPATH/srfi/160/f64.sld. teenylib is a good example of the structure for this.

TEENYCELLSEGSIZE

Size of a single cell segment, in bytes.

TEENYCELLNSEGMENT

Maximum number of cell segments.

TEENYEVALLIMIT

Number of steps to stop execution after, for debugging. Only works if you compile with -DEVAL_LIMIT=1

Description of the last three, courtesy of Rodion Gorkovenko:

CELL_SEGSIZE and CELL_NSEGMENT - expand available memory; memory is
allocated by segments, starting with 3, up to CELL_NSEGMENT; every
segment can hold CELL_SEGSIZE cells; before allocating more, GC will
try to free up existing. Defaults are 20000 and 12. Error code on
memory limit reaching is 9.

EVAL_LIMIT (if compiled with define of the same name) will stop
execution on reaching given amount of steps (with error code 7).

Compile-Time Defines #

A lot of Teeny behaviors are also mandated by C preprocessor macros defined as 1 or 0. These can be found in config.h, together with their default values and explanations. To override the defaults, provide the name of the macro with a value in FEATURES when compiling:

# Disable all the features enabled by default
make FEATURES="-DUSE_DL=0 -DUSE_MATH=0 -DUSE_UNICODE=0" teenyscheme
# Enable the most memory-hungry setup
make FEATURES="-DUSE_DL=1 -DUSE_MATH=1 -DUSE_UNICODE=1 -DCELL_SEGSIZE=100000 -DCELL_NSEGMENT=30" teenyscheme
# Disable _all_ features
make FEATURES="-DUSE_NO_FEATURES=1" teenyscheme

Note that the USE_* macros have to be defined as true or false (1 or 0.) The code checks them for their value and not them being (un)defined. If they are not defined, they are set to (sane) default values in config.h.

Standard Compliance #

Like with original TinyScheme, Teeny Scheme currently supports a subset of R5RS. Roughly corresponding with (scheme r5rs) library of R7RS. Things woefully missing:

• Full non-decimal flonum notation.
• Circular literals.
• Unwind-aware continuations, at least without current init—it defines dynamic-wind kinda properly.

Things that are not supported, and likely will never be, because they are not Teeny:

• Ratios.
• Unbounded bignums.
• Complex / imaginary numbers.
• Tail call optimization. To avoid blowing up the stack, use while or do and iterate.
• Re-binding of symbols like quasiquote and =>, as these are hard-coded too aggressively in the parser.

Feature Identifiers #

There are some features indicating what functionality is supported:

math

When built with USE_MATH. If it’s not defined, then many math operations will be unavailable and libraries relying on them will be severely broken.

full-unicode

When built with USE_UNICODE. Makes procedures like string->utf8 and utf8->string available. Also indicates Unicode support in strings and source code.

foreign-extensions

When built with USE_DL. load-extension is available.

big-endian, little-endian

Endianness

windows, android, linux, freebsd, solaris, hp-ux, aix, ios, osx, unix

OSes

x86-64, x86-32, i386, arm, arm64, riscv, mips, ppc, ppc64, sparc, m68k

Architectures

Macro Systems #

There are three macro systems coexisting in Teeny at the moment. Only one of them is native: macro form. There’s also a tiny wrapper around it, define-macro. The syntax is:

(macro (unless form) ;; 1
  `(if (not ,(cadr form)) (begin ,@(cddr form))))
(macro quasiquote ;; 2
  (lambda (l)
    something))
(define-macro (begin1 form . other-forms) ;; 3
  (let ([res (gensym "result")])
    `(let ([,res ,form])
       ,@other-forms
       ,res)))
(define-macro quasiquote ;; 4
  (lambda (l)
    something))
(gensym [string]) ;; 5
(with-gensyms (variables ...) body ...) ;; 6

Explanation:

1. This form is misleadingly procedure-like. However, it accepts only one argument—the form to expand. Including its head—macro name. Prefer define-macro to that, to avoid confusion.
2. This is a more honest version of the macro form, emphasizing that it binds a variable to a transformer.
3. define-macro is more featureful and procedure-like, in that its argument list is actually an argument list, allowing for positional and variadic arguments and ignoring the macro keyword itself, unlike macro. Not any more advanced though. Still unhygienic macros and symbol juggling.
4. Alternative form of define-macro synonymous to the matching macro form. Not used actively at the moment.
5. gensym is a vital helper for macro writing. Returns a new interned symbol each time it’s called. Taking string as the base when provided. Make sure to use it to bind forms that might be otherwise multiple-evaluated.
6. (Init-provided, trivial) with-gensyms makes gensym management slightly less annoying. It binds the provided variable-s to fresh gensym-s. If the variable is a two-element list (symbol string), where string is a gensym string instead of symbol name. Otherwise variable should be a symbol whose name will also be used for gensym.

Bundled init also provides a severely incomplete and buggy version of R7RS syntax-rules which is sufficient to write some simple macros, but can fail in obscure ways when left unattended. Prefer define-macro whenever you have the choice.

Syntactic Extensions #

There is a number of syntactic extensions made in Teeny:

Named lambdas

Providing any symbol after lambda arglist and before lambda body names the lambda with this symbol. This is heavily heuristic, but should be fine for most cases. At least until there are symbol macros.

Modifiable parameters

Calling a parameter on the value sets this parameter to the value, given converter passing.

Square brackets

Used as aliases for parentheses. Not configurable (yet?)

Typing #

One important extension that Teeny adds to Scheme syntax is type annotations / abbreviations. These can be used anywhere lambda formals are implied. Including in procedure-shaped form of define and in define-macro.

Every procedure argument that needs to be type-checked is represented as a list (name type) instead of a name symbol. The syntax of the type part is that of opcode type abbreviations, but represented as a string of 1+ chars. Only the first char is significant, the rest can be used for documentation purposes. So all of "s", "string", and "super-confusing" mean a type check for a string.

Same type annotations can be used in define-record-type, whenever it is loaded from either bundled init or teenylib. These types will be checked on record creation and on field modification. Extended syntax of fields then is:

(plain-field plain-accessor)
(plain-field plain-accessor plain-modifier)
(typed-field "type" plain-accessor)
(typed-field "type" plain-accessor typed-modifier)

Some examples:

;; https://dthompson.dev/posts/optimizing-guile-scheme.html
(define (magnitude [x "number"] [y "number"])
  (sqrt (+ (* x x) (* y y))))

;; From Teeny init
(define (cddr [x ". pair"])
  (cdr (cdr x)))

(define (get-environment-variable [var "string name"])
  (let ([val (assoc var (get-environment-variables))])
    (and val (cdr val))))

(define (struct-equal? [x "$truct"] [y "$truct"])
  (and (eq? (struct-name x)
            (struct-name y))
       (= (struct-length x)
          (struct-length y))
       (do ([i 0 (+ i 1)]
            [acc #t (and acc
                         (equal? (struct-ref x i)
                                 (struct-ref y i)))])
           ((>= i (struct-length x)) acc))))

(define (memq obj [lst "list (any)"])
  (generic-member eq? obj lst))

(define (filter [pred "/applicable"] [lst "list"])
  (do ([lst lst (cdr lst)]
       [acc '() (if (pred (car lst))
                    (cons (car lst) acc)
                    acc)])
      ((null? lst)
       (reverse! acc))))

(define-record-type time
  (make-time type [nanosecond "integer"] second)
  time?
  [type       "ymbol"   time-type set-time-type!]
  [nanosecond "number"  time-nanosecond set-time-nanosecond!]
  [second     "integer" time-second set-time-second!])

Return type checking and let-binding checks are possible, but not yet implemented. If you want to make them work, check OP_APPLY and type_mismatch function.

Non-standard Goodies #

Most of these descriptions are taken from TinyScheme Manual.txt.

(current-environment)

The environment in effect at the time of the call. Beware: this might not be the environment you want. It is ephemeral and changes all the time, so the bindings made with eval or load might be lost to oblivion.

(defined? symbol [environment])

Checks whether the given symbol is defined in the current (or given) environment.

(gc)

Performs garbage collection immediatelly.

(gcverbose [bool])

The argument (defaulting to #t) controls whether GC produces visible outcome.

(quit [num])

Stops the interpreter and sets the retcode internal field (defaults to 0). When standalone, retcode is returned as exit code to the OS. Whenever possible and provided by init, use exit.

(load-extension [filename-without-extension])

Loads a DLL declaring foreign procedures. On Unix/Linux, one can make use of the ld.so.conf file or the LD_RUN_PATH system variable in order to place the library in a directory other than the current one. Please refer to the appropriate man page. (This description is likely quite dated, update pending.)

(oblist)

Returns the oblist, an immutable list of all the symbols. Unlikely to ever be necessary nor useful, as it simply exposes an internal structure of symbol table.

(macro-expand form) (macro-expand-all form)

Returns the expanded form of the macro call denoted by the argument.

(define-with-return (procname args...) body ...)

Like plain define, but makes the continuation available as return inside the procedure. Handy for imperative programs.

(new-segment num)

Allocates more memory segments.

(procedure-code procedure)

Gets the code as Scheme data. The structure is usually (lambda args . body).

(procedure-name procedure)

Gets the name attached to the closure.

(procedure-arity procedure)

Gets the (minimum maximum) values describing the arity of procedure

1. minimum is always an integer
2. maximum is either an integer or #t for variadic procedures

Works on both opcodes and actual closures / procedures. In all other cases, returns #f #f

(make-closure code [environment])

Makes a new closure in the given environment.

(eval-count)

Count of main “evaluation loop” of interpreter, may be used as a time-independent metric of code (in)efficiency.

(struct? object)

Whether object is a structure / record.

(make-struct name size)

Create a new struct instance with type name and size slots

(struct-length struct), (struct-name struct)

Return the number of slots in the struct and its type name, respectively.

(struct-ref struct index), (struct-set! struct index new-value)

Getter and setter for a struct.

(set-parameter-unsafe! parameter value)

Set parameter value directly, ignoring parameter converter Use with caution.

(libdir [new-libdir])

Parameter returning the current libdir, can be set to an optional argument.

(load-path-name)

Current file name, whenever being loaded. Not a parameter!

(type-of object)

Returns the type of a given object, as a symbol. In case it returns unknown for you, you should probably report it as a bug.

(%cast! object type)

Cast the object to a given type (symbol, one of the symbols returned by type-of.) Beware: this opcode converts the object in place, so any use of the same object will now use it as the new type. Prone to breakages, use carefully and cast things back when done.

(make-immutable! object)

Make the object immutable and return it. Note that this is not reversible, so make sure to use it reasonably.

(reexport libspec ...)

Whenever used in define-library, imports the provided libraries and re-exports their exports from the current library.

(error [type] message irritants ...)

Extended form of standard error procedure, allowing for a symbol tag before message argument. This tag is in particular used for error subtypes and can be accessed with error-object-type.

(system command)

Passes a given command to the OS and returns the returned code.

(open-input-pipe command), (open-output-pipe command)

Makes a sub-process running the command and returns a port that allows reading or writing to it using regular Scheme APIs like display.

procedure-code is my personal favorite. Do you know of any Scheme that allows you to see the full code for a lambda, just like that?

Provided APIs #

All the provided APIs are available from teenyscheme/scheme.h header. Provided APIs differ based on whether USE_INTERFACE was true at build time. These C functions are provided regardless of its value:

teeny_scheme *scheme_init_new(void)

Allocate a new teeny_scheme object and return it.

teeny_scheme *scheme_init_new_custom_alloc(malloc, free)

Allocate a new teeny_scheme object and return it. But set its malloc and free attributes to provided functions. Useful in e.g. making allocation stricter with something like xmalloc.

int scheme_init(teeny_scheme *sc)

int scheme_init_custom_alloc(teeny_scheme *sc, malloc, free)

Initialize an already allocated scheme object, like above.

void scheme_deinit(teeny_scheme *sc)

De-initialize and cleanup the sc.

void scheme_load_file(teeny_scheme *sc, FILE *fin)

void scheme_load_named_file(teeny_scheme *sc, FILE *fin, const char *filename)

Load / evaluate the given file in sc. Use the *_named_file version whenever possible, so that error messages are more useful.

void scheme_load_string(teeny_scheme *sc, const char *cmd)

Evaluate expressions from the cmd. Note that this does not return any values. If you want return values, use scheme_eval.

teeny_object_t scheme_eval(teeny_scheme * sc, teeny_object_t obj)

Evaluate the s-expression contained in obj. Return the primary returned value.

teeny_object_t scheme_apply0(teeny_scheme *sc, const char *procname)

teeny_object_t scheme_call(teeny_scheme *sc, teeny_object_t func, teeny_object_t args)

Call the respective procedure, without or with args. scheme_apply0 is somewhat useless, because it accepts procedure name and not the procedure itself.

void scheme_set_external_data(teeny_scheme *sc, void *p)

Set the external data accessible to foreign functions.

void scheme_define(teeny_scheme *sc, teeny_object_t env, teeny_object_t symbol, teeny_object_t value)

Bind the given symbol to a value in env.

This is a pretty small set of operations. Not enough to deal with complex data for sure. That’s why Teeny also exposes data-processing functions when USE_INTERFACE is defined as truthy. All of these functions are documented in interface.h and are accessible through the sc->vptr field. So something like integer creation is sc->vptr->mk_integer(sc, 3).

make FEATURES="-D__NOCC__ -DUSE_MATH=1 -DUSE_UNICODE=1..." clean all

make FEATURES="-D__cplusplus -DUSE_MATH=1 -DUSE_UNICODE=1..." clean all

Many of these APIs deal with Teeny-specific C types:

teeny_object_t

Arbitrary scheme value

teeny_char_t

Representation of a character, changing based on Unicode configuration

teeny_fixnum_t and teeny_flonum_t

Types representing fixnums / exact integers and flonums / inexact reals, respectively.

tgscheme.h—type-generic Scheme API #

In case you use a C11+ compiler, you can alse use a tgscheme.h header. (Available as teenyscheme/tgscheme.h include.) It provides many of the APIs above, wrapped into short and convenient generic macros. Provided APIs are:

method(sc, name, args…)

Call the name-d (interface) method on the provided args. Note that sc has to be passed with all the other args, because some “methods” don’t use it. Providing it automatically would be rude to these.

mk(sc, literal)

mknum(sc, number)

Creates data from C literals. Beware: while mk is more generic, it does not have floating-point numbers support. Use mknum to create numbers. Also be careful when using these with non-literals—proper functioning not guaranteed.

mksym(sc, symbol)

Create a new symbol named by symbol. symbol can be a C string literal or a pre-existing symbol (teeny_object_t.)

define(sc, env, name, value)

name is converted to a symbol (as per mksym) and value is converted to a suitable Scheme value (as per mk) and then get assigned as per scheme_define.

eval(sc, expr)

Shortcut for scheme_eval. Whenever provided a C string as expr, evaluates the expr-named symbol and returns its value. Otherwise equivalent to scheme_eval.

length(sc, data)

Return the length (number of elements) in the given piece of data. For now, it’s implemented as a function instead of macro. Might change later.

apply(sc, fn, args…)

Applies the provided fn to args, which must be a C arglist of teeny_object_t-s, with the last one required to be a list teeny_object_t. So the behavior is much alike the Scheme procedure—variable arity, last arg is a list.

Here’s an example showcasing the type-generic API use:

#include "teenyscheme/scheme.h"
#include "teenyscheme/tgscheme.h"

int main(int argc, char *argv[], char *envp[])
{
	teeny_scheme sc_, *sc = &sc_;
	scheme_init(sc);
	teeny_object_t a = mk(sc, 1);
	teeny_object_t b = mknum(sc, 1.012321);
	teeny_object_t fn = eval(sc, "+");
	teeny_object_t res = apply(sc, fn, a, b, sc->NIL);
	printf("Result is %Lf\n", method(sc, rvalue, res));
	return EXIT_SUCCESS;
}

Extending Teeny Scheme #

This section is loosely based on the old TinyScheme hack.txt example of a memory block. Basically describing how to add a new data structure to Teeny Scheme.

Adding a New Data Type #

New type essentially consists out of some layout of things in memory and opcodes to operate on that.

First, this data type must have some memory layout and a place to belong. This place is struct cell in scheme.h (likely to be renamed or removed, but it’s here for now.) So add it there:

struct cell {
	unsigned int flag;
	union {
		struct {
			char *svalue;
			int length;
		} string;
		// New type added:
		struct {
                        // Technically, the same as string above
                        // But I prefer cleaner and stupider code
			char *data;
			size_t length;
		} memblock;
		struct {
			teeny_object_t *data;
			teeny_object_t name;
			int length;
		} struct;
		/* ... */
	} object;
};

Nicer macros can be added for easier access in scheme.c:

#define veclength(p)        ((p)->object.vector.length)
#define vecelems(p)         ((p)->object.vector.elems)

// Added:
#define memblocklength(p)   ((p)->object.memblock.length)
#define memblockdata(p)     ((p)->object.memblock.data)

After layouting, this type should be registered in types.h. This is the main source of all type info. The fields are:

Lowercase name

Used in predicate generation, the predicate gets the is_$lower name

Uppercase name

Used in type tag generation, type tag is T_$UPPER

simple?

Boolean for whether the value of this type can be easily printed and treated as non-compound. Make sure to add printing of simple values to simple2str.

Description

Human-readable description of a type, as seen in type mismatch error messages

Char-encoded type abbreviation

This are a bit cryptic. It assigns type check to a certain character to later be used in type checks in e.g. ops.h. Basically pick a char you like and see if there are compilation warnings.

Currently existing type abbreviations are:

We can add a record like this to generate our type tags and predicates and all.

_TYPE(parameter,    PARAMETER,    true,   "parameter",         ':')
_TYPE(boolean,      BOOLEAN,      true,   "boolean",           'b')
// Added:
_TYPE(memblock,     MEMBLOCK,     true,   "memory block",      'm')

Optionally, add printing of this type to simple2str:

static void
simple2str(teeny_scheme *sc, teeny_object_t l, int f, char **pp, int *plen)
{
	string p;

	*plen = -1;
	if (l is sc->NIL) {
		p = (string)"()";
	} else if (l is sc->T) {
		p = (string)"#t";
	} else if (l is sc->F) {
		p = (string)"#f";
        /* ... */
	} else if (is_struct(l)) {
		p = sc->strbuff;
		snprintf(p, sc->strbuff_size, "#<struct %s>",
			 symname(structname(l)));
	// Added:
	} else if (is_memblock(l)) {
		p = sc->strbuff;
		snprintf(p, sc->strbuff_size, "#<memblock %p [%lu]>",
			 (void *)l, memblocklength(l));
	} else {
		p = (string)"#<error>";
	}
	/* ... */
}

GC-ing the New Type #

Then we need to modify the garbage collection procedures to be aware of this memory block thing. The only mandatory place is the finalize_cell that frees up the memory:

static void
finalize_cell(teeny_scheme *sc, teeny_object_t a)
{
	if (is_string(a)) {
		sc->free(strvalue(a));
	} else if (is_vector(a)) {
		for (size_t i = 0; i < veclength(a); ++i)
			finalize_cell(sc, vecelems(a)[i]);
		sc->free(vecelems(a));
    	// Added clause:
	} else if (is_memblock(a)) {
		sc->free(memblockdata(a));
	} else if (is_port(a)) {
		if (a->_object._port->kind & PORT_FILE
		    and a->_object._port->rep.stdio.closeit) {
			port_close(sc, a, PORT_INPUT | PORT_OUTPUT);
		}
		sc->free(a->_object._port);
	}
}

In case your new type contains other Scheme data, you need to make it known to garbage collector too. Otherwise this data might be silently swiped away in the next round of GC. Marking the data involves only one change, in mark() function in scheme.c. In case your data structure is a sequence of Scheme datums, this struct incantation might be a good template:

if (is_struct(p)) {
	size_t len = structlength(p);
	for (size_t i = 0; i < len; i++) {
		mark(structdata(p)[i]);
	}
}

In case your structure is box-like, containing a restricted number of values, something like this might work:

if (is_parameter(p))
	mark(paramvalue(p));

The heuristic basically is: find all the Scheme data in your data structure and mark every one of them. GC will handle the rest.

Memblock example does not need marking, as its data is all C-side.

Adding Embedded Interface Entries #

You likely want to make the data structure you created accessible to the embedded users. Which means: interfaces and function pointers. At the very least, define a type predicate, type constructor, and type getters / setters for embedded user convenience:

interface teeny_object_t
mk_memblock(teeny_scheme *sc, size_t len, char fill)
{
	char *p = (char *) sc->malloc(len);
	if (p is nil)
		return sc->NIL;
	teeny_object_t x = get_cell(sc, sc->NIL, sc->NIL);

	typeflag(x) = T_MEMBLOCK | T_ATOM;
	memblockdata(x) = p;
	memblocklength(x) = len;
	memset(p, fill, len);
	return x;
}
/* is_memblock is defined by types.h */
interface char
memblock_elem(teeny_object_t p, size_t index)
{
	return memblockdata(x)[index];
}
interface void
set_memblock_elem(teeny_object_t p, size_t index, char value)
{
	memblockdata(x)[index] = value;
}
interface size_t
memblock_length(teeny_object_t p)
{
	return memblocklength(p);
}

And then add the respective entries to interface.h:

// Constructors
/* ... */
_INTERFACE(teeny_object_t, mk_memblock,       mk_memblock,        teeny_scheme *sc, size_t len, char fill)
// Predicates
/* ... */
_INTERFACE(bool,    is_memblock,       is_memblock,        teeny_object_t p)
// Value handling
/* ... */
_INTERFACE(size_t,  memblock_length,   memblock_length,    teeny_object_t p)
_INTERFACE(char,    memblock_elem,     memblock_elem,      teeny_object_t p, size_t index)
_INTERFACE(void,    set_memblock_elem, set_memblock_elem,  teeny_object_t p, size_t index, char value)

The fields are:

1: return type

Return type of the “exported” function, stripped from all the static et al.

2: exported name

The name it will be exposed by the interpreter under e.g. sc->vtbl.is_memblock, a callable struct member

3: implementation function name

It’s fine if it’s the same name as in (2)—separating these is more of a hack than a rule. Most of the time you just paste the same name here.

rest: arguments

Signature of a given function, basically. Useful in auto-generated type declarations.

That’s basically it. interface.h is used in a bunch of places to generate nice and proper types and fields. So just add things there and use them through sc->vptr!

Adding a New Opcode #

There are two main places to edit for new opcode:

• ops.h where metadata about opcodes resides, and
• scheme.c where the implementation / handler of the opcode goes.

First, ops.h. Add an _OP_DEF line there. The fields are:

1: opcode name

An optional string naming the Scheme-side symbol this opcode will be bound to. Can be set to nil in case the opcode is internal or otherwise unnamed.

2: minimum arity

Mandatory integer determining how many required arguments there are.

3: maximum arity

How many arguments can the procedure accept. In case it’s INF_ARG, the procedure is variadic and accepts any number of arguments after the required ones.

4: type checks

A string of type check abbreviations, format explained in type addition section. Strictly one check / char per argument. In case the procedure is variadic, the variadic argument should be typed too, with the last type check applicable to every of its elements. In case the procedure is not variadic but has optional arguments, these too can be typed.

5: opcode tag

This will become an opcode-identifying enum tag and will be used in handler definition.

So the example memblock opcodes are:

// Notice that the optional argument is typed too
_OP_DEF("make-block",                     1,  2,       "++",  OP_MKBLOCK)
// Note the use of the "m" type check
_OP_DEF("block-length",                   1,  1,       "m",   OP_BLOCKLEN)
_OP_DEF("block-ref",                      2,  2,       "m+",  OP_BLOCKREF)
_OP_DEF("block-set!",                     3,  3,       "m++", OP_BLOCKSET)
_OP_DEF("block?",                         1,  1,       "*",   OP_BLOCKP)

After the line describing the opcode is added, it’s possible to write a handler in scheme.c. (It’s also possible to write a handler and only then add an opcode metadata—it won’t compile without either of these anyway.) Two forms that help with that are DEFHANDLER and DEFSHORTHANDLER. DEFSHORTHANDLER is for simple one-line opcodes returning a single value. While DEFHANDLER is for more involved handlers. Respective syntax:

DEFHANDLER(OP_CODE_TAG, "scheme-procedure-name or freeform description", teeny_scheme *sc)
{
	some;
	statements;
	s_return(sc, ...); // or s_return_values(sc, values_cons);
// ENDHANDLER is mandatory for some macro magic
ENDHANDLER}
DEFSHORTHANDLER(OP_CODE_TAG, "scheme-procedure-name or freeform description", return_value);
// essentially equivalent to:
// DEFHANDLER(OP_CODE_TAG, "scheme-procedure-name or freeform description", teeny_scheme *sc)
// {
//	s_return(sc, return_value);
// ENDHANDLER}

DEFHANDLER always has a single argument, Scheme interpreter. And it implies a proper function body. While DEFSHORTHANDLER has this argument implied as sc. And always returns the value provided after description. See scheme.c for actual opcode definitions. I tried to make them as readable as possible. Shame on me if they aren’t.

First, the opcode to create memory blocks and the underlying function:

DEFHANDLER(OP_MKBLOCK, "make-block", teeny_scheme *sc)
{
	int fill = 0;
	size_t len = ivalue(car(sc->args));
	if (cdr(sc->args) != sc->NIL)
		fill = charvalue(cadr(sc->args)) % 255;
	s_return(sc, mk_memblock(sc, len, (char)fill));
ENDHANDLER}

Then, two short handlers—for block length and block checking. Preferably, put them into the same group with the other short handlers:

DEFSHORTHANDLER(OP_CURR_ENV, "current-environment", sc->envir)
// Added:
DEFSHORTHANDLER(OP_BLOCKLEN, "block-length", mk_integer(sc, memblock_length(car(sc->args))))
DEFSHORTHANDLER(OP_BLOCKP, "block?", mk_boolean(is_memblock(car(sc->args))))
// *INDENT-ON*

Now the gory part: element references

DEFHANDLER(OP_BLOCKREF, "block-ref", teeny_scheme *sc)
{
	char *data = memblockdata(car(sc->args));
	int index = ivalue(cadr(sc->args));

	if (index < 0 or (size_t) index >= memblocklength(car(sc->args)))
		error_named_1(sc, "out of bounds:", cadr(sc->args));
	s_return(sc, mk_integer(sc, data[index]));
ENDHANDLER}
DEFHANDLER(OP_BLOCKSET, "block-set!", teeny_scheme *sc)
{
	check_immutable(car(sc->args),
			"unable to alter immutable memory block:");
	char *data = memblockdata(car(sc->args));
	int index = ivalue(cadr(sc->args));

	if (index < 0 or (size_t) index >= memblocklength(car(sc->args)))
		error_named_1(sc, "out of bounds:", cadr(sc->args));

	char c = ivalue(caddr(sc->args)) % 255;

	data[index] = (char)c;
	s_return(sc, car(sc->args));
ENDHANDLER}

Note that, compared to the old tutorial, all the type checks are gone. They are performed by the interpreter based on opcode definitions.

With that, we have a new data structure with a proper memory layout. And with opcodes to work with it.

Helpers for Opcode Writing #

Some of these featured in the preceding section, some are slightly more esoteric. You’ll know it when you’ll need them.

s_return(sc, value)

s_return_values(sc, first, other)

These essentially return value(s) that the respective opcode should return to the caller. s_return_values gets an sc; first / primary value; and then a cons-ed list of other values. Use these instead of C return in handlers, as they manage stack and registers properly.

s_goto(sc, op)

This “jumps” to another opcode, in case you want to delegate execution to it. Make sure to set registers properly before using that!

s_goto_args(sc, op, args)

Same as s_goto but sets sc->args to its argument. While not strictly necessary, it’s too frequent of a pattern to just ignore.

s_save_args_code(sc, op, args, code)

This one is particularly useful when s_goto is used. It’s a way to chain opcodes so that the “saved” one is executed after the “goto-ed” one (in last-in-first-out order.) Might be useful if your opcode delegates some work to another opcode but then needs to process the results too.

sc->args, sc->code, sc->value, sc->other_values

These are registers of struct scheme that are used a lot in opcodes.

• sc->args is the cons list of arguments passed to the opcode.
• sc->code is somewhat arbitrary and is mostly used in syntactic opcodes, representing the code they process. But some opcodes use it in a loose “s-expression to process” sense. So meta.
• sc->value and sc->other_values are basically the primary value and the values after it. This is a weird design, but it’s there for backwards-compatibility and because the primary value is used too often to destructure conses all the time.

error_0(sc, msg)

error_1(sc, msg, arg)

error_many(sc, msg, arg_cons)

error_typed_many(sc, type_string, msg, arg_cons)

Use these as much as possible—opcodes should be as strict as possible, like all of Teeny Scheme. They error out in the REPL or call (error-hook) on error. With nice metadata like function or opcode name and file / line.

As an example of s_save_args_code and other helpers, call-with-values uses one additional opcode:

_OP_DEF("call-with-values", 2, 2, TST_PROCEDURE TST_PROCEDURE, OP_CALLWV0)
_OP_DEF(nil,                0, 0, nil,                         OP_CALLWV1)

And then delegates the work to OP_APPLY when applying value-producing thunk:

DEFHANDLER(OP_CALLWV0, "call-with-values (entry)", teeny_scheme *sc)
{
	s_save_args_code(sc, OP_CALLWV1, sc->NIL, cadr(sc->args));
	sc->code = car(sc->args);
	sc->args = sc->NIL;
	s_goto(sc, OP_APPLY);
ENDHANDLER}
DEFHANDLER(OP_CALLWV1, "call-with-values (apply the second procedure)",
	   teeny_scheme *sc)
{
	sc->args = cons(sc, sc->value, sc->other_values);
	s_goto(sc, OP_APPLY);
ENDHANDLER}

Adding a New Base Syntax Form #

On the example of a recently added while form.

First, add name detection to syntaxnum (yeah, this code sucks:)

static int
syntaxnum(teeny_object_t p)
{
	char *s = symname(p);
	switch (strlen(s)) {
		/* ... */
	case 4:
		/* ... */
	case 5:
		switch (s[2]) {
		case 'g':
			return OP_BEGIN;	// begin
		case 'l':
			return OP_DELAY;	// delay
		case 'c':
			return OP_MACRO0;	// macro
		// Added:
		case 'i':
			return OP_WHILE0;       // while
		default:
			return OP_QUOTE;	// quote
		}
		/* ... */
	}
}

Then, assign the syntax alongside all the rest in scheme_init_custom_alloc:

	assign_syntax(sc, "macro");
	assign_syntax(sc, "case");
        // Added:
	assign_syntax(sc, "while");

What’s left is writing opcodes processing this new syntax. First, declarations in opcodes file:

_OP_DEF(nil,                  0,  0,       nil,                      OP_WHILE0)
_OP_DEF(nil,                  0,  0,       nil,                      OP_WHILE1)

And then actual code in scheme.c:

DEFHANDLER(OP_WHILE0, "while", teeny_scheme *sc)
{
	s_save(sc, OP_WHILE1, car(sc->code), cdr(sc->code));
	sc->code = car(sc->code);
	s_goto(sc, OP_EVAL);
ENDHANDLER}
DEFHANDLER(OP_WHILE1, "while...", teeny_scheme *sc)
{
	/* ... */
ENDHANDLER}

Note that syntactic opcodes don’t get sc->args. They get sc->code, containing the forms after their keyword. So a form like (while (positive? a) (display a) (set! a (- a 1))) gets ((positive? a) (display a) (set! a (- a 1))) as its sc->code.

What’s left is figuring out how to write proper opcodes for my syntactic monstrosities.

Adding an Extension Library in C #

The user can add foreign functions and extension libraries in C when Teeny was compiled with USE_DL. For example, a function that squares its argument may look like:

teeny_object_t square(teeny_scheme *sc, teeny_object_t args) {
	if (args != sc->NIL) {
		if (sc->vptr->is_number(sc->vptr->pair_car(args))) {
			flonum_t v = sc->vptr->rvalue(sc->vptr->pair_car(args));
			return sc->vptr->mk_real(sc, v * v);
		}
	}
	return sc->NIL;
}

This function can then be bound to a symbol:

sc->vptr->define(
	sc,
	sc->global_env,
	sc->vptr->mk_symbol(sc, "square"),
	sc->vptr->mk_foreign_func(sc, square, "n"));

Foreign functions can use the external data (ext_data) in the scheme struct to implement any kind of external state. External data are set with scheme_set_external_data.

But for that foreign function to be accessible in the Scheme world, it has to belong to a library. This library is required to:

• Be accessible to OS-specific DLL loading facilities (dlopen on Unix-es.)
• Have a name (part of a file (base)name before the dynamic library extension.)
• Have an initialization function of the form init_$name. This init function must accept scheme pointer and return void. Put all the define-s there.

Foreign Pointers #

One can add foreign functions to wrap C functions on the Scheme side. However, there is a need for representing C pointers on Scheme side too. That’s what foreign pointers API does: representing and manipulating pointers.

(datum->pointer object)

Convert object to a pointer object with address matching the internal address of the object itself.

(pointer->integer ptr), (integer->pointer n)

Convert between Scheme integer and pointer having address equal to this integer.

(bytevector->pointer bytevector)

Converts a given bytevector to a pointer object pointing directly to its contents. Try (pointer-ref (bytevector->pointer #u8(1 2 3)) 8)

(pointer-ref ptr size), (pointer-set! ptr size value)

Getter and setter for pointer contents. Sets the first size bits of the ptr to value (positive integer.) Endianness depends on the system and is not penetrable.

(pointer+ ptr offset)

Add offset bits (can be negative) to ptr. In case you want to reference some data at a distance from a pointer, the combo is (pointer-ref (pointer+ ptr offset) size)

(pointer? object), bool is_pointer(teeny_object_t p)

Scheme- and C-side predicates to check whether something is a pointer object.

void *pointer_raw(teeny_object_t ptr)

C function to get the actual pointer from the pointer object.

Extending Parts of Implementation with Hooks #

A part of Teeny implementation logic can be overridden through the use of hook variables / parameters. These are subject to change, but useful nonetheless. The variables / parameters are:

parameter: (compile-hook [value])

Called on a lambda code—(args . body)—before it is evaluated. Might be useful in optimization.

parameter: (error-hook [value])

Called with a single argument—R7RS error object. (File and read errors, along with other error classes, are in progress.) Allows overriding the exception handling and preventing errors “bubbling up” to toplevel.

parameter: (sharp-hook [value])

Called whenever the implementation encounters an unknown piece of syntax prefixed with hash # symbol. Should be a zero-argument procedure handling read-ing on its own.

Contribute to Teeny!

I will be happy to accept contributions as long as they are small, focused, and have no AI-generated content. Visit Teeny Scheme repository to learn more and get accustomed with the code.