Programming Languages DO Have Speed

Andrés Osinski

Your favorite language has a lot of differentiating characteristics that separate it from its different alternatives. We might say, naively, that if you consider a language to be just:

Syntax
Grammar
Evaluation rules
A standard library

you just might assume speed differences should be small or nil, given pieces of code from two different languages that represent the same operation. Such is not the case. Why?

Syntax and grammar limit or expand your level of abstraction

How does your language copy and array? Does your language say:

"perform this array copy operation using SSE3 XMM registers on memory-aligned data",
"for each element: copy into destination", or
"copy raw data as fast as possible on this machine"?

The answer to this question gives very different behaviors and optimization strategies for each operation: for the first, you'll have the fastest, most memory-efficient, least portable option available; or you get a useless binary because you're using a non-x86 CPU or you don't have SSE3 support, in which case your program won't run. Option 2 is the most portable alternative, and the slowest, as you'll be limited to copying things by your language's word size, which may or may not be optimal. Option 3 will probably be the fastest choice in addition to be the most portable and fail-safe, since it'll try the most optimal instruction, but it will be the most costly, having to do expensive machine-architecture checks that might just not be worth it.

Alas, if your language supports objects as memory references and the elements in question are such, that's not working right, because you're just doing a shallow copy, not actually moving all that data on memory. Do you have the semantics to support this detail? Should you actually need to support it?

Extremely high-level languages have the advantage that they can leave optimization to the compiler/interpreter/JIT of choice to find an optimal optimization strategy, but the overhead of this process and the lack of visualization of lower-level, potentially more optimal choices, means that you're missing out on what may be really, really important shortcuts, and you'll be consuming more RAM and time trying to find what works best.

And that is really not a trivial thing to say, yet most languages do not have support for annotating data in ways that may aid compilers, and for some extremely dynamic languages some static inferences of data are simply not possible. At the same time, some optimizations can only be performed during runtime, and it's unlikely that hand-coded lower-level code can be as optimal (witness some edge cases in JVM benchmarks, or the performance increase that Fortran arrays have over C arrays by avoiding pure pointer arithmetic). This leads to a second point...

Languages can specify implementation details

When a language sets in stone some of its capabilities, it's inherently describing some level of implementation details that all legal interpreters/VMs/compilers have to abide to. Capabilities that can have important effects on performance. If a language considers classes as objects, its internals will differ much from languages that use arrays of function pointers. If a language expects every function name to be available at runtime, there will be information that you must have somewhere in memory. If a language defines a calling convention, you must abide by it. A language that supports monkey patching needs a set of internals that's prepared to do so, even when it can be statically compiled. A lot of features simply incur a minimum level of overhead, whether they're used or not. Note the painstaking planning C++ required to make sure every interesting feature could be discarded and not consume resources (and how that makes it significantly less developer-friendly than many alternatives). On the other hand, look at how interpreted languages such as Python simplify the use of external symbols with the simple 'import' statement.

OK, so let's say we want to disregard everything and start over, not caring about the language's interop and minimizing the overhead of run-time capabilities so that they're irrelevant for all intents and purposes. But that takes us to the last point...

Languages Are Communities

Javascript, among others, could probably be turned into a fine systems programming language, yet you don't see people making efforts to bring V8 to do fast bit twiddling and instead turn their attention to the most common Web use cases. C++ could be made into a fine scripting language with sufficient macro magic, but most would scoff at the idea. That's because a language is more than what you get in the "Learning X Programming Language" book. It's an ecosystem of libraries, interop systems, VMs, compilers, coding styles and community. It's a way of doing things.

Who would write Python without considering the Zen of Python? Where would Javascript be without jQuery and mooTools? Why is it that Ruby first had Rails and RSpec and not some other language?

And some of these tools work as well as they do because they're not all fast at all. In fact, they work as well because usability and extensibility took precedence over performance and that design decision ends up paying off so well that the project lives on another day and gets sufficient critical mass to get people thinking about improving it in many ways. But if speed is not a concern, why worry, when it's just one of the many angles to the utility of our tools.

As much as we fantasize about having our favorite language be molded to fit into different performance and use envelopes, each community chooses where to take its language because they use them to get certain things done, while other tasks get different tools. And that's fine, actually, because trying to wedge language concepts and idioms where they don't fit ends up getting tiring, and sooner or later it's going to show and it will require dexterous hackery and many a sleepless night trying to understand just why something works the way it does.

This is another reason why alternative implementations of languages are hardly ever as popular as the standard implementation. The mainline Ruby VM is the most used VM despite it not being the most performant because it's the gold standard, and if code doesn't work adequately with it then it probably has little utility for the community. Mono includes an AOT compiler for C#, yet that just doesn't get used as much as the JIT, even when it might come in handy, because you lose the possibility of linking with other .NET libraries and using a lot of the goodness that comes from a managed environment.

Ultimately, a programming language has a set of rules, tools, communities and legacy that indeed gives it a "speed". Yes, a lot of them can be adapted for more performant tasks with C/C++ interop, and a lot of them can be made to be more useful for certain tasks with macros and some really hard thought put into them. But at the end of the day what matters is the speed you can get out of your tool's actual use, not the theory that a few microbenchmarks might say.