this is my kid in a candy store wishlist of programming language features that are kicking around in my head:
- small and consistent
- functional w/eg lexical scope and closures
- mutablity allowed
- managed memory
- statically typed with full inference
- record types w/row polymorphism
- polymorphic variants
- multiple dispatch
- coroutines
- multiple backends, including its own VM
- well-defined interface to the host
- single threaded
admittedly some of these these are in tension 🤔
functional is non-negotiable to me. lexical scope, first class functions, anonymous local data structures, closures etc are the best building blocks to software I've ever used.
I say this in contrast to conventional object oriented programming which is, frankly, some non-sense with no theoretical underpinning. there are a few projects out there similar to what I'm thinking that are held back by their commitment to these debunked antiquated ideas.
that said I tend to want to make video games and things for which a very particular kind of performance is critical, and forcing immutable data structures and their overhead is a non-starter unfortunately. I know this from experience... in my experience I avoid the kinds of bugs you get from mutable state by keeping mutation local with coroutines and control flow, and by keeping this whole thing single threaded.
managed memory is critical because that's just not the level of abstraction I want to be working at when I am implementing creative ideas. it feels like trying to paint with a Ziploc bag full of loose bristles instead of a paint brush.
that said there is not really a one size fits all solution to memory. in my thinking this is a hosted language and questions about memory are moved to the embedding interface. for the VM it would be a combination of some reference counting the papers I've read.
I have not become more conservative politically as I've gotten older, but I have become more sympathetic to static typing. I do not tend to run into state bugs in practice but I *do* run into bugs that would have been prevented by a better type checker...
that said unlike a lot of type nerds I do not think annotations and declarations are "inherently good" and I'm much more drawn to systems that let me elide them and/or allow anonymous types. Roc really sent me off down this path.
related to that is my more recent reading on records with row polymorphism, which is what I think want out of a record type. I want to be able to create them anonymously wherever I need them and pass them around while not abandoning checking. JavaScript objects come *so close* but they have a bunch of weird edge cases around properties that are unfortunate, and TypeScript is limited in its checking and inference around them...
polymorphic variants are the row polymorphic version of algebraic data types which I found transformative to use in rust and f#. they offer such a nice compromise where an expression can be "one of a set of values" which to me feels like a really powerful midpoint between "every expression is an instance of one type" and "anything can be anything". exhaustive pattern matching is also really nice.
the classic type theory stuff really falls down around dispatch though. every function, generally, can only have one signature for type theoretic reasons. that means you end up with functions with names like map2 which is gross. more critically operator overloading becomes impossible, and in video games I'm doing linear algebra... all the time. so many examples of type inferencers take addition to be defined as an operation on numbers where in my world I need it to support vectors matrices etc
it's not just classical typed languages, this is also my biggest gripe with making video games in JavaScript. if I have to write Vector3.addition instead of + the languages barely usable for linear algebra. this kind of feature gets brushed under the rug generally because I guess most programmers are not doing that much arithmetic? the Julia language is my guiding light here
coroutines! absolutely non-negotiable feature for me to feel expressive making interactive software. I wandered the deserts of PLT looking for semantics I could paint with and I found them.
they require a compiler transformation to do right so the language needs to support them at a fundamental level. I expect multiple dispatch + static typing + coroutines is going to run into formal roadblocks but I'm willing to dream.
from my experience in clojure I'm wary about languages with multiple backends but I think it's worth pursuing. particularly if this language is explicitly designed to be hosted and as a result makes few assumptions about its host. once type checking and other transformations are done code generation to different backends is not intractable. this one is maybe a nice to have idk
I have thoughts for a standalone VM that would support this language too, probably something like a rust library. my thinking uses reference counting for managed memory and I think you can set it up to be more or less pauseless, basically by deferring decrement and release operations into chunks of work that can be run "in between" events while the system is idle. I also have a sketch of spaghetti stacks to implement the coroutines.
given that it is a hosted language then the interface to the host is critical. this is what makes Lua and Python attractive as embeddable languages.
the model I have in mind is more similar to the language Céu where the VM exposes very constrained inputs and outputs, almost like a shader but for behavior/state machines. it's basically an event loop that allows coroutines to block on events that the host can unblock, optionally passing in event data.
finally: it is absolutely single threaded, which often gets dismissed as a deficiency but the benefits you gain from simplicity and, critically, determinism cannot be overstated. it is so important to be able to build reproducible code when developing video games and interactive software. the ability to capture and replay input and reproduce state reliably is critical. having everything run on one thread makes that possible.
@nasser there is a lot of gamedev stuff like networking, physics and AI that is pretty much impossible to do single threaded. coroutines dont help when you have to wait on a blocking socket or churn through simulations in a performant way while keeping your game responsive