Compile timings

In order to fully compare the project compiler to NodeJs and RingoJs we need to take startup time in to account. This section will present the results we got when we meassured the startup times for the different JavaScript implementations.

5.3.5.1 The project compiler timings

The project compiler reports the time spent in each compilation phase after the whole compilation is complete. We have included the total time spent compiling and two individual phases: The lex and parse phase and the GCC compilation phase. The lex and parse phase is included for comparison with the NodeJs timings, since this is the phase that V8 needs to complete fully before any kind of partial compilation can take place. The GCC phase is included to get an idea about the time spent optimizing the final, native binary.

Table 5.17: The project compiler timings for the Delta Blue benchmark (in seconds)

No opt CP Inlining DCE TI

Total 35,04 39,13 39,44 48,75 51,16

In GCC 3,65 3,71 3,69 3,71 4,30

Lex and Parse 28,86 29,02 29,31 29,59 28,41

Other 2,53 6,41 6,43 15,45 18,45

The Delta Blue is the largest JavaScript file of the benchmark programs, so it is not surprising that the compilation time is relatively large. The time spent in the lex and parse phase, is disproportionally large - especially when considering that the GCC phase includes a lex and parse phase of its own and still only uses about 10 % of the overall time. Almost all of the time spent in the lex and parse phase is in the auto-generated code produced by the tool ANTLR. The overall time increases when adding optimizations which is to be expected.

Table 5.18: The project compiler timings for the Dijkstra benchmark (in seconds)

No opt Cp Inlining DCE TI

Total 8,824 9,955 10,698 13,388 14,362

In GCC 1,724 2,247 1,646 1,641 1,646

Lex and Parse 6,406 4,864 6,75 6,276 6,048

Other 0,694 2,844 2,302 5,471 6,668

Again we see that most of the time is spent in the lex and parse phase, but the results are less extreme compared to the Delta Blue test. Again the most expensive part of the rest of the compiler is the Optimizations, which, at the highest level, take almost as much time as the lex and parse phase. GCC, again, uses very little time compared to the rest of the compiler.

The loops program is tiny, so the fact that more than four seconds are spent in the lex and parse phase is very surprising. The size of the program also

Table 5.19: The project compiler timings for the Loops benchmark (in seconds)

No opt CP Inlining DCE TI

Total 6,213 6,407 6,52 6,712 7,709

In GCC 1,335 1,345 1,343 1,342 1,335

Lex and Parse 4,106 4,111 4,215 4,071 4,384

Other 0,772 0,951 0,962 1,299 1,99

reflects on the time it takes for the project compiler to optimize it - even with all optimizations enabled, the optimization phase uses little more than a second.

Table 5.20: Distribution of time spent in compilation Time spent

In GCC 11%

Lex and Parse 65%

Other 24%

The distribution of time across all the compilations measured shows that the time spent in the lex and parse is disproportionally large. GCC uses a significant amount of time, but still not very much compared to the rest of the compiler.

5.3.5.2 NodeJs startup timings

For NodeJs we have timed how much of the run-time is being used outside of the test. This does not correlate exactly with the time being spent compiling.

Since V8 is a JIT compiler, some of the compilation will be taking place during execution.

Due to the design of V8 the timings below are more accurately described as the time it spends on the lex and parse phase as well as compiling the outer-most function, since all other functions are not compiled untill they are called.

Table 5.21: NodeJs time spent outside the test No Opt IC All opt

Empty 0,045 0,045 0,046

Delta Blue 0,048 0,052 0,051 Dijkstra 0,129 0,064 0,068

Loops 0,047 0,049 0,052

The results do show that NodeJs spends very little time starting up. The over-head, as measured by the empty JavaScript program, is about 45 milliseconds.

This indicates that V8 is able to lex and parse the Delta Blue test in about 3 -7 milliseconds. The Dijkstra tests takes slightly longer for it to parse, but that might simply reflect that more code is in the outer-most function. The loops test is almost instantaneous with about 1-2 milliseconds spent lex, parsing and compiling the outer-most function.

5.3.5.3 RingoJs startup timings

For the RingoJs timings, the time spent outside the tests includes both the startup overhead as well as the compilation overhead. To try and differentiate the two, we have included timings for the empty JavaScript program. Rhino does ahead of time compilation, so the entire compilation overhead occurs before the test begins. We have included the interpreter as well to compare the time it takes to start up with the time it takes to first compile, and then start the program.

Table 5.22: RingoJs compile timings Interpret No opt All opt

Empty 1,971 1,005 1,005

Delta Blue 2,933 1,233 1,234 Dijkstra 3,944 3,462 3,461

Loops 3,434 1,046 1,057

It is interesting to note, that we see practically no difference between the time it takes Rhino to compile with and without optimizations enabled. Also quite surprising is that the interpreter actually takes longer to start than doing the compilation. The compilation times for the Delta Blue test are interesting when compared to the project compiler - the entire Rhino compilation is completed about two seconds faster than even just the GCC phase.