The OpenD Programming Language

arsd.script

A small script interpreter that builds on arsd.jsvar to be easily embedded inside and to have has easy two-way interop with the host D program. The script language it implements is based on a hybrid of D and Javascript. The type the language uses is based directly on var from arsd.jsvar.

The interpreter is slightly buggy and poorly documented, but the basic functionality works well and much of your existing knowledge from Javascript will carry over, making it hopefully easy to use right out of the box. See the examples to quickly get the feel of the script language as well as the interop.

I haven't benchmarked it, but I expect it is pretty slow. My goal is to see what is possible for easy interoperability with dynamic functionality and D rather than speed.

Public Imports

arsd.jsvar
public import arsd.jsvar;
Undocumented in source.

Members

Classes

NonScriptCatchableException
class NonScriptCatchableException

A base class for exceptions that can never be caught by scripts; throwing it from a function called from a script is guaranteed to bubble all the way up to your interpret call.. (scripts can also never catch Error btw)

ScriptCompileException
class ScriptCompileException

Thrown on script syntax errors and the sort.

ScriptException
class ScriptException

This represents an exception thrown by throw x; inside the script as it is interpreted.

ScriptRuntimeException
class ScriptRuntimeException

Thrown on things like interpretation failures.

Functions

interpret
var interpret(string code, var variables, string scriptFilename, string file, size_t line)

This is likely your main entry point to the interpreter. It will interpret the script code given, with the given global variable object (which will be modified by the script, meaning you can pass it to subsequent calls to interpret to store context), and return the result of the last expression given.

interpretFile
var interpretFile(File file, var globals)
repl
void repl(var globals)

Enhanced repl uses arsd.terminal for better ux. Added April 26, 2020. Default just uses std.stdio.

Detailed Description

A goal of this language is to blur the line between D and script, but in the examples below, which are generated from D unit tests, the non-italics code is D, and the italics is the script. Notice how it is a string passed to the interpret function.

In some smaller, stand-alone code samples, there will be a tag "adrscript" in the upper right of the box to indicate it is script. Otherwise, it is D.

Installation instructions

This script interpreter is contained entirely in two files: jsvar.d and script.d. Download both of them and add them to your project. Then, import arsd.script;, declare and populate a var globals = var.emptyObject;, and interpret("some code", globals); in D.

There's nothing else to it, no complicated build, no external dependencies.

$ wget https://raw.githubusercontent.com/adamdruppe/arsd/master/script.d
$ wget https://raw.githubusercontent.com/adamdruppe/arsd/master/jsvar.d

$ dmd yourfile.d script.d jsvar.d

Script features

OVERVIEW

  • Can subclass D objects in script. See [http://dpldocs.info/this-week-in-d/Blog.Posted_2020_04_27.html#subclasses-in-script
  • easy interop with D thanks to arsd.jsvar. When interpreting, pass a var object to use as globals. This object also contains the global state when interpretation is done.
  • mostly familiar syntax, hybrid of D and Javascript
  • simple implementation is moderately small and fairly easy to hack on (though it gets messier by the day), but it isn't made for speed.

SPECIFICS

  • Allows identifiers starting with a dollar sign.
  • string literals come in "foo" or 'foo', like Javascript, or raw string like D. Also come as “nested “double quotes” are an option!”
  • double quoted string literals can do Ruby-style interpolation: "Hello, #{name}".
  • mixin aka eval (does it at runtime, so more like eval than mixin, but I want it to look like D)
  • scope guards, like in D
  • Built-in assert() which prints its source and its arguments
  • try/catch/finally/throw You can use try as an expression without any following catch to return the exception:
    var a = try throw "exception";; // the double ; is because one closes the try, the second closes the var
    // a is now the thrown exception
  • for/while/foreach
  • D style operators: +-/* on all numeric types, ~ on strings and arrays, |&^ on integers. Operators can coerce types as needed: 10 ~ "hey" == "10hey". 10 + "3" == 13. Any math, except bitwise math, with a floating point component returns a floating point component, but pure int math is done as ints (unlike Javascript btw). Any bitwise math coerces to int.

    So you can do some type coercion like this:

    a = a|0; // forces to int
    a = "" ~ a; // forces to string
    a = a+0.0; // coerces to float

    Though casting is probably better.

  • var a = "12";
    a.typeof == "String";
    a = cast(int) a;
    a.typeof == "Integral";
    a == 12;
    Type coercion via cast, similarly to D. Supported types for casting to: int/long (both actually an alias for long, because of how var works), float/double/real, string, char/dchar (these return *integral* types), and arrays, int[], string[], and float[].

    This forwards directly to the D function var.opCast.

  • some operator overloading on objects, passing opBinary(op, rhs), length, and perhaps others through like they would be in D. opIndex(name) opIndexAssign(value, name) // same order as D, might some day support [n1, n2] => (value, n1, n2)

    obj.__prop("name", value); // bypasses operator overloading, useful for use inside the opIndexAssign especially

    Note: if opIndex is not overloaded, getting a non-existent member will actually add it to the member. This might be a bug but is needed right now in the D impl for nice chaining. Or is it? FIXME

    FIXME: it doesn't do opIndex with multiple args.

  • if/else
  • array slicing, but note that slices are rvalues currently
  • variables must start with A-Z, a-z, _, or $, then must be [A-Za-z0-9_]*. (The $ can also stand alone, and this is a special thing when slicing, so you probably shouldn't use it at all.). Variable names that start with __ are reserved and you shouldn't use them.
  • int, float, string, array, bool, and #{} (previously known as json!q{} aka object) literals
  • var.prototype, var.typeof. prototype works more like Mozilla's __proto__ than standard javascript prototype.
  • the |> pipeline operator
  • // inheritance works
    class Foo : bar {
    	// constructors, D style
    	this(var a) { ctor.... }
    
    	// static vars go on the auto created prototype
    	static var b = 10;
    
    	// instance vars go on this instance itself
    	var instancevar = 20;
    
    	// "virtual" functions can be overridden kinda like you expect in D, though there is no override keyword
    	function virt() {
    		b = 30; // lexical scoping is supported for static variables and functions
    
    		// but be sure to use this. as a prefix for any class defined instance variables in here
    		this.instancevar = 10;
    	}
    }
    
    var foo = new Foo(12);
    
    foo.newFunc = function() { this.derived = 0; }; // this is ok too, and scoping, including 'this', works like in Javascript
    classes: You can also use 'new' on another object to get a copy of it.
  • return, break, continue, but currently cannot do labeled breaks and continues
  • __FILE__, __LINE__, but currently not as default arguments for D behavior (they always evaluate at the definition point)
  • most everything are expressions, though note this is pretty buggy! But as a consequence: for(var a = 0, b = 0; a < 10; a+=1, b+=1) {} won't work but this will: for(var a = 0, b = 0; a < 10; {a+=1; b+=1}) {}

    You can encase things in {} anywhere instead of a comma operator, and it works kinda similarly.

    {} creates a new scope inside it and returns the last value evaluated.

  • functions: var fn = function(args...) expr; or function fn(args....) expr;

    Special function local variables: _arguments = var[] of the arguments passed _thisfunc = reference to the function itself this = reference to the object on which it is being called - note this is like Javascript, not D.

    args can say var if you want, but don't have to default arguments supported in any position when calling, you can use the default keyword to use the default value in any position

  • macros: A macro is defined just like a function, except with the macro keyword instead of the function keyword. The difference is a macro must interpret its own arguments - it is passed AST objects instead of values. Still a WIP.

Todo list

I also have a wishlist here that I may do in the future, but don't expect them any time soon.

FIXME: maybe some kind of splat operator too. choose([1,2,3]...) expands to choose(1,2,3)

make sure superclass ctors are called

FIXME: prettier stack trace when sent to D

FIXME: support more escape things in strings like \n, \t etc.

FIXME: add easy to use premade packages for the global object.

FIXME: the debugger statement from javascript might be cool to throw in too.

FIXME: add continuations or something too - actually doing it with fibers works pretty well

FIXME: Also ability to get source code for function something so you can mixin.

FIXME: add COM support on Windows ????

Might be nice: varargs lambdas - maybe without function keyword and the x => foo syntax from D.

Examples

This example shows the basics of how to interact with the script. The string enclosed in q{ .. } is the script language source.

The var type comes from arsd.jsvar and provides a dynamic type to D. It is the same type used in the script language and is weakly typed, providing operator overloads to work with many D types seamlessly.

However, if you do need to convert it to a static type, such as if passing to a function, you can use get!T to get a static type out of it.

var globals = var.emptyObject;
globals.x = 25; // we can set variables on the global object
globals.name = "script.d"; // of various types
// and we can make native functions available to the script
globals.sum = (int a, int b) {
	return a + b;
};

// This is the source code of the script. It is similar
// to javascript with pieces borrowed from D, so should
// be pretty familiar.
string scriptSource = q{
	function foo() {
		return 13;
	}

	var a = foo() + 12;
	assert(a == 25);

	// you can also access the D globals from the script
	assert(x == 25);
	assert(name == "script.d");

	// as well as call D functions set via globals:
	assert(sum(5, 6) == 11);

	// I will also set a function to call from D
	function bar(str) {
		// unlike Javascript though, we use the D style
		// concatenation operator.
		return str ~ " concatenation";
	}
};

// once you have the globals set up, you call the interpreter
// with one simple function.
interpret(scriptSource, globals);

// finally, globals defined from the script are accessible here too:
// however, notice the two sets of parenthesis: the first is because
// @property is broken in D. The second set calls the function and you
// can pass values to it.
assert(globals.foo()() == 13);

assert(globals.bar()("test") == "test concatenation");

// this shows how to convert the var back to a D static type.
int x = globals.x.get!int;

Macros

Macros are like functions, but instead of evaluating their arguments at the call site and passing value, the AST nodes are passed right in. Calling the node evaluates the argument and yields the result (this is similar to to lazy parameters in D), and they also have methods like toSourceCode, type, and interpolate, which forwards to the given string.

The language also supports macros and custom interpolation functions. This example shows an interpolation string being passed to a macro and used with a custom interpolation string.

You might use this to encode interpolated things or something like that.

var globals = var.emptyObject;
interpret(q{
	macro test(x) {
		return x.interpolate(function(str) {
			return str ~ "test";
		});
	}

	var a = "cool";
	assert(test("hey #{a}") == "hey cooltest");
}, globals);

Classes demo

See also: arsd.jsvar.subclassable for more interop with D classes.

var globals = var.emptyObject;
interpret(q{
	class Base {
		function foo() { return "Base"; }
		function set() { this.a = 10; }
		function get() { return this.a; } // this MUST be used for instance variables though as they do not exist in static lookup
		function test() { return foo(); } // I did NOT use `this` here which means it does STATIC lookup!
						// kinda like mixin templates in D lol.
		var a = 5;
		static var b = 10; // static vars are attached to the class specifically
	}
	class Child : Base {
		function foo() {
			assert(super.foo() == "Base");
			return "Child";
		};
		function set() { this.a = 7; }
		function get2() { return this.a; }
		var a = 9;
	}

	var c = new Child();
	assert(c.foo() == "Child");

	assert(c.test() == "Base"); // static lookup of methods if you don't use `this`

	/*
	// these would pass in D, but do NOT pass here because of dynamic variable lookup in script.
	assert(c.get() == 5);
	assert(c.get2() == 9);
	c.set();
	assert(c.get() == 5); // parent instance is separate
	assert(c.get2() == 7);
	*/

	// showing the shared vars now.... I personally prefer the D way but meh, this lang
	// is an unholy cross of D and Javascript so that means it sucks sometimes.
	assert(c.get() == c.get2());
	c.set();
	assert(c.get2() == 7);
	assert(c.get() == c.get2());

	// super, on the other hand, must always be looked up statically, or else this
	// next example with infinite recurse and smash the stack.
	class Third : Child { }
	var t = new Third();
	assert(t.foo() == "Child");
}, globals);

Properties from D

Note that it is not possible yet to define a property function from the script language.

static class Test {
	// the @scriptable is required to make it accessible
	@scriptable int a;

	@scriptable @property int ro() { return 30; }

	int _b = 20;
	@scriptable @property int b() { return _b; }
	@scriptable @property int b(int val) { return _b = val; }
}

Test test = new Test;

test.a = 15;

var globals = var.emptyObject;
globals.test = test;
// but once it is @scriptable, both read and write works from here:
interpret(q{
	assert(test.a == 15);
	test.a = 10;
	assert(test.a == 10);

	assert(test.ro == 30); // @property functions from D wrapped too
	test.ro = 40;
	assert(test.ro == 30); // setting it does nothing though

	assert(test.b == 20); // reader still works if read/write available too
	test.b = 25;
	assert(test.b == 25); // writer action reflected

	// however other opAssign operators are not implemented correctly on properties at this time so this fails!
	//test.b *= 2;
	//assert(test.b == 50);
}, globals);

// and update seen back in D
assert(test.a == 10); // on the original native object
assert(test.b == 25);

assert(globals.test.a == 10); // and via the var accessor for member var
assert(globals.test.b == 25); // as well as @property func

Meta

Authors

Adam D. Ruppe

History

November 17, 2023: added support for hex, octal, and binary literals and added _ separators in numbers.

September 1, 2020: added overloading for functions and type matching in catch blocks among other bug fixes

April 28, 2020: added #{} as an alternative to the json!q{} syntax for object literals. Also fixed unary ! operator.

April 26, 2020: added switch, fixed precedence bug, fixed doc issues and added some unittests

Started writing it in July 2013. Yes, a basic precedence issue was there for almost SEVEN YEARS. You can use this as a toy but please don't use it for anything too serious, it really is very poorly written and not intelligently designed at all.