Motivation: Why a New Programming Language Is Still Needed
This summary is to explain why a new language is still needed.
What I care about is this: which historical lessons existing languages have failed to absorb, where history has regressed, and which design mistakes a language aimed at complex engineering, static analysis, systems development, and security should avoid.
Problems with Rust
Rust's biggest problem is that it builds memory safety on the myth of static checking.
Static analysis cannot predict future. Building memory safety on a compile-time model is itself a fantasy.
Rust's actual safety gains are far lower than advertised, while its language complexity is extremely high. Mechanisms make code very hard to read, write, and maintain. Many API designs become awkward, and many simple tasks become "compiler puzzles.". Its complexity substantially reduced development efficiency.
Rust even has macros, a major regression that seriously erodes safety. Rust also has variable rebinding, an unbelievably bizarre design. Rust's design is full of unbelievable and severely safety-damaging bizarre choices. Some of these absurd designs also depend strangely on context, constantly work against "safety," move in the opposite direction of "safety," and seriously conflict with the advertised claim of "safety."
More seriously, code complexity itself introduces security risks. Making code hard to understand in pursuit of safety can ultimately have a negative effect on safety.
Rust's safety myth has been over-promoted. Rust safety promise is unreliable in essence.
Problems with Go
Go's problem is that, despite having GC, it still preserves a large amount of C-like pointer semantics and nil problems.
Go still allows nil pointer dereference. This is a serious design flaw in a modern language.
Type aliases, interfaces, 'interface{}', 'any', and 'nil' are serious design flaws in Go, and using them in combination can be disastrous. These design flaws deal a serious blow to Go’s type system. And led to unfixable difficulties when generics were added later, leaving Go with an ugly generics implementation.
Another design flaw: keyword overloading. Not just overloading, but far too much overloading.
These design flaws seriously conflicts with Go's advertised "simplicity."
Go's advantage is that it is safer than C, but it is still several serious defects away from an ideal engineering language.
Problems with Python
Python is only suitable for small programs, scripts, and quick prototypes. Writing one hundred lines of code is fine; once it enters complex engineering, dynamic typing and excessive flexibility explode directly.
Python's core problem is that its semantics are too open. Mechanisms make programs difficult for static tools to understand.
Python has difficulty supporting reliable refactoring, rename, call graph analysis, dead code analysis, and large-codebase navigation.
Python's "flexibility" is, in essence, a sacrifice of static analyzability. It is convenient for small programs and disastrous for complex projects.
Problems with C / C++
The biggest problem with C/C++ is that manual memory management has already been proven by history to fail in complex engineering. On the most core issue, preventing RCE, C/C++ perform far too poorly.
The most typical real-world example is browser JavaScript engines, especially V8. Chrome / V8 is already one of the projects in the world with the highest security investment. Even today, the entire industry must still expect that V8, written in C++, will continue to produce RCE-class vulnerabilities.
C++'s problem is more serious than C's, it is not only unsafe but also extremely complex.
But C still has a reason to exist. At the lowest level, there must be a bare language for bootloaders, kernel bootstrap, and similar scenarios. The runtime of a safe language also needs to be started by some kind of bare language.
Therefore, C should not be completely eliminated, but should be restricted to a very small low-level trusted area. The bare-language layer must be extremely minimal, because it works in an unprotected environment. Once it becomes complex, failures are inevitable.
Problems with Scheme
Scheme's design is very unsuitable for engineering practice.
Scheme's token rules and identifier rules are too free. It allows variable names to use too many symbols. Scheme lexer rules do not merely tokenize source code; they participate in syntax, and even have bizarre context dependencies. This freedom provides almost no substantive help to expressive power, but greatly increases tool complexity.
S-expressions do not bring the imagined benefit of being "easy to analyze." Writing a Scheme parser is not simple at all, the language has too many special forms and variants. Almost every important syntax keyword has a pile of variants and different spellings. Multiplying those variants by the multiple ways each variant can be written makes error messages and AST complexity rise sharply. Add macros and special lexer syntax on top. The whole front-end becomes an engineering disaster. Yet these special variants multiplied by their different spellings add no expressive power; they only create trouble and confusion.
Scheme semantics also contain strange settings such as treating every value other than #f as true in conditionals, and treating '() as true in conditionals even though it is not the same value as #t. Strange settings fill almost every corner of Scheme. They are actively harmful to expressive power, and they do not even reduce typing.
The claim that Scheme is "small" is completely false; Scheme has a very large language surface area.
Scheme is dynamically typed. This is a hard flaw for complex program.
Macros are a major design flaw.
features such as call/cc and dynamic-wind have very low value in real engineering, yet they seriously increase compiler and runtime complexity.
Problems with OCaml / Standard ML
OCaml/Standard ML pay too high a cost for Hindley-Milner type inference.
OCaml/Standard ML takes syntax and feature stacking way too far. Tool support becomes unrealistic.
Problems with C#
- Type aliasing.
- Exception mechanism. Java's checked exceptions are imperfect, but at least they expose "this function may fail" in the API contract. C# removed checked exceptions, turning exceptions back into implicit control flow. Callers cannot know from the type signature what errors a function may throw.
- Type inference.
- Too many backdoors.
Problems with TypeScript
- Too many backdoors.
- Type system have too many features.
- Even the strictest setting isn’t strict enough.
Problems with Java
Java is one of the few languages that truly learned lessons from the engineering disasters of C/C++.
It removed macros, removed raw pointers, removed manual free, has no type aliases, binds filenames to public classes, keeps package/class structures stable, has clear nominal typing, and has very strong IDE and refactoring capabilities. Java's module and engineering structures have been validated over the long term by ultra-large codebases. Many later languages have failed to achieve this.
But Java also has serious flaws.
- null pointer dereference is a hard flaw.
- Java's generics implementation is poor.
- early Java lacked expressive power. It had no higher-order functions.
- Java was influenced by the OOP trend of its era and introduced inheritance as a core mechanism. Inheritance is a serious blow to the type system.
- Java took the bizarre "everything is object" route. It made arrays into a half-object, half-compiler-magic monster, seriously damaging the type system and adding unnecessary difficulty to later generics. "Everything is X" is usually the beginning of a disaster.
- Java depends on a VM. Many features must rely on the JVM runtime, so Java cannot serve as a truly native systems language and cannot be used to write operating systems.
- modern Java has also started adding some regressive mechanisms, such as local type inference.
Java was a major step forward, but it is not the final answer.
Incremental Compilation and Dependency Analyzability
File dependency analysis cannot be achieved by simply copying Java's import. It requires every aspect of the language to avoid features that cannot be analyzed. If just one weak link has a problem, the entire dependency analysis may collapses globally.
Languages with complex syntax, dynamic typing, or certain fancy type systems fundamentally do not have stable incremental compilation, they cannot even support reliable refactoring and renaming.
The key to compilation speed is dependency analyzability; single-file compilation speed does not need extreme optimization. Incremental compilation is a feature that complex engineering needs badly. If it must be sacrificed, the reason has to be unbelievably strong.
Why Fancy Type Systems Inevitably Fail
All kinds of fancy type systems are ultimately bound to fail for two fundamental reasons:
- type systems are extremely sensitive to features and can easily become undecidable all over the place;
- static analysis cannot predict the future.
A type system is not a syntactic decoration that can freely accumulate features.
Every added feature changes the system. Small features do not stay small. Interactions compound. Complexity leaks. Tooling degrades. The cost does not rise politely. It explodes.
Once the wrong combination of features is introduced, undecidability or near-undecidability can appear all over the place.
Static analysis can never predict the future. The claim that a particular type system and static checking can guarantee memory safety or concurrency safety is a computer-science version of "claiming to have built a perpetual motion machine.".
A good type system is limited and boring on purpose. Provide clear type boundaries, prevent obvious type mixing, make API contracts visible, and allow IDEs and static analysis tools to understand programs stably.
Why Fancy Concurrency Models Inevitably Fail
All kinds of fancy concurrency models are ultimately bound to fail for the same fundamental reason as fancy type systems: static analysis has two hard limits that cannot be broken.
- type systems and static models are extremely sensitive to features and can easily become undecidable all over the place.
- static analysis cannot predict the future.
These two hard limits cannot be challenged by any type system, any static check, or any compile-time model.
Over the past several decades, all kinds of models aimed at concurrency safety have appeared repeatedly. Each time there is new marketing claiming that concurrency safety has finally been solved. After each failure, a new set of terms is introduced and another cycle begins. These movements are the computer-science version of "building a perpetual motion machine."
Some people once criticized "concurrency as library," arguing that concurrency cannot merely be implemented as a library and must enter language design. At the same time, many languages started embedding mechanisms into the language core.
But this brings more serious consequences: once a concurrency model enters the language core, it interacts explosively with other language features.
Even worse, once these features enter the language, backward compatibility locks them in completely. Even if the design is later found to be wrong, removing it is impossible. The language carries a permanent complexity burden, with accumulated debt that becomes impossible to escape.
The current reality is that there is no public validated general mechanism, static or runtime, that guarantees concurrency safety.
A built-in concurrency model is a huge design flaw in a programming language and a serious blow to safety and must be expelled from core language design.
Concurrency models claiming to guarantee concurrency safety, and type systems or static checks claiming to guarantee memory safety, are all the computer-science version of "claiming to have built a perpetual motion machine." They are repeatedly proven to fail.
What Kind of Language I Want
The language I want should absorb historical lessons and explicitly address the most serious hard flaws in existing languages.
It should have the following characteristics.
- The core goal: prevent RCE
- GC, but explicitly triggered.
- No null pointer dereference
- Explicit nominal typing
- No type aliases.
- No type inference
- No built-in concurrency model
- No macros
- Context-independent typed literals and explicit type conversion
untyped literal and implicit type conversion are huge design flaw
- Higher-order functions and simple generics
- Tool analyzability must be a first-class design goal.
Dependency analysis, IDE functionality, and refactoring functionality must be as accurate as possible.
- Syntax surface area must be small
- Compile to native and be usable for systems programming.