IronJS vs. Jint performance micro benchmark

I’ve been working on a Javascript implementation for the .NET CLR the last few weeks, I’m finally getting close to something release-able and wanted to post some initial performance benchmarks. I’ve measured the performance compared to another Javascript implementation available for the .NET CLR - Jint, which is not running on the DLR but is a custom interpreted runtime.

The specs on the testing machine are:

• 64-bit Windows 7 Enterprise
• 2.53Ghz Intel Core2 Duo
• 4.0Gb of RAM
• 4,5 Windows Experience Index
• .NET 4.0 Beta2
• Visual Studio 2010 Beta2

All assemblies were compiled using Release mode for Any CPU, Debug and Trace constants were not defined, Optimize code was checked and Debug Info was set to none. The benchmarks themselves are very low level measuring three different areas:

• Variable set performance
• Function call performance
• Member access performance

All benchmarks were done using the following timing code, 20 times and then averaged.

public sealed class Benchmark : IEnumerable
{
    private readonly Action subject;
    private Benchmark(Action subject) { this.subject = subject; }

    public static Benchmark This(Action subject)
    {
        return new Benchmark(subject);
    }

    public IEnumerator GetEnumerator()
    {
        subject();
        GC.Collect();
        GC.WaitForPendingFinalizers();

        var watch = new Stopwatch();

        while (true)
        {
            watch.Reset();
            watch.Start();
            subject();
            watch.Stop();
            yield return watch.ElapsedMilliseconds;
        }
    }

    IEnumerator IEnumerable.GetEnumerator() { return GetEnumerator(); }
}

Set variable performance

The overhead/performance of setting a variable was measured using the same for loop with two set variable statements in the body, iterating 1,000,000 times.

for(var i = 0; i < 1000000; ++i) {
    foo = 1;
    var bar = 1;
}

The results are (measured in milliseconds, lower is better):

Function call performance

The function call overhead was measured by iterating over a for loop 1,000,000 times and calling a user defined empty function each iteration

foo = function() {};
for(var i = 0; i < 1000000; ++i) {
    foo();
}

Which gave this result (measured in milliseconds, lower is better):

Member access performance

Measures the performance of accessing object properties by nesting four objects inside each other and accessing the deepest object 1,000,000 times using a for loop.

foo = { bar: { baz: { daz: {  } } } };
for(var i = 0; i < 1000000; ++i) {
    var stuff = foo.bar.baz.daz;
}

Which resulted in (measured in milliseconds, lower is better):

Conclusion

All in all I’m happy with the performance of IronJS, considering that I have not spent a single second optimizing it and have made conscious decisions to not use several DLR optimization techniques to keep the code as simple as possible up until the 0.1-release.