Click here to Skip to main content
Click here to Skip to main content
Go to top

An Objective Analysis of Language Performance - 1) Math Functions

, 26 Apr 2008
Rate this:
Please Sign up or sign in to vote.
This article, the first of a series of articles, presents an objective analytical analysis of language performance for Java and C# .NET

mathperformance5.jpg

Introduction

This series of articles presents an objective analytical analysis of language performance for Java and C# .NET (2.0, 3.0, and 3.5). The aim of these articles is too end the age old dispute of .Net is better than Java or vica versa. The article analyses performance of standard base mathematical operators common to all languages but specifically mathematical operators that form the basic components of most programs.

Scope of the Analysis

In this article the following functions are tested for performance;

Function Java Equivalent C# .Net Equivalent
Add + +
Subtract - -
Multiply * *
Divide / /
Power Math.power(x, y) Math.Pow(x, y)

Testing Procedure

Key Points

  • The testing is performed via a console all cases with system load at minimal prior to test start
  • All system auto-updates are disabled
  • Task manager is run and set to show cpu usage of all processes ordered by cpu usage descending; if any other process exceeds 5% usage then the test is restarted
  • The test process is set to a single core affinity (to reduce cpu effects)
  • All tests are performed 5 times to reduce data noise and increase accuracy
  • All tests are performed on the same multi-core system (reduces influences of other processes)
  • Each mathematical function is tested 1000000000 times to reduce inaccuracy in system time calculation and increase accuracy of measures
  • Effects of looping over 1000000000 times is removed in control case

Test System

CPU Intel Core 2 Extreme QX6700 - Kentsfield, 2.66Ghz
Motherboard ASUSTeK - P5WDG2 WS Pro
Memory DDR2 - 5120MB, Dual channel

Test Code

Control Case

double y = 0;
int controlStart = System.Environment.TickCount;
for (int i = 0; i < 1000000000; i++)
{
   y = y + 1;
}
int controlEnd = System.Environment.TickCount;

Console.WriteLine(y.ToString());

The control case is intended to be used to remove the effects of the loop over the test function. To prevent compiler optimisations from removing the loop the loop performs a simple add function which is used in all function tests (see below). Furthermore the value of the add function is printed at the end, again the prevent compiler optimisations from removing y due to non-use and then the loop itself. The control case is repeated 5 times and the average of all (controlEnd - controlStart) is used as the controlTime. y is reset to 0 at the start of each test.

Test Case

y = 0;
double x = 0;
int addStart = System.Environment.TickCount;
for (int i = 0; i < 1000000000; i++)
{
   y = y + 1;
   x = x + 1;
}
int addEnd = System.Environment.TickCount;

Console.WriteLine(y.ToString());
Console.WriteLine(x.ToString());

The test case is kept as similar to the control case as possible, only with the addition of the mathematical function being tested. In this example the add function is being tested, and like the control case the value of x is printed at the end to prevent removal of x due to compiler optimisations. Furthermore comparison of resultant x values may be used to confirm equal functionality between the various languages being tested. The test time is calculated as ((addEnd - addStart) - controlTime).

The other tests are performed as follows;

Function Code
Add x = x + 1;
Subtract x = x - 1;
Multiply x = x * 1.1;
Divide x = x / 1.1;
Power x = Math.Pow(x, 1.1);

Results Analysis

mathperformance4.jpg

All times are in milliseconds and are averages of the 5 test passes.

mathperformance1.jpg

mathperformance2.jpg

Lower values are better.

In these graphs relative performance of functions is shown, and counter to perhaps popular belief is that in all but the power and multiply cases Java actually appears to out-perform C# .Net languages. It is worth noting however that the cases of add and subtract for java are actually lower than 0 indicating that either the control cases have run particularly slowly or perhaps some optimisation has occurred. Since the control cases for Java show good cohesion, it is unlikely to be the former.

mathperformance3.jpg

Lower values are better.

In this graph the total time for completing the control cases are shown over the 5 passes. It is interesting to note that some form of optimisation has occurred in the .Net languages due to the consistent reduction of overall time from pass 1 to the latter passes; this effect cannot be seen in Java which appears to perform consistently better (perhaps due to the control case being based on the add function itself which according to the results above performs significantly better).

Summary

  • The control case, i.e. a loop and add function, performs consistently better in Java
  • .Net performs some optimisation on repeated functions, while Java does not
  • Java performs better at divide, add, and subtract functions (although further testing is required to confirm)
  • .Net performs better at multiply and power functions

Other References

Check back soon for other articles on comparison of disk access and interface update performance.

History

Version 1.0.0.1 - Testing +, -, *, /, and power.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Derek Bartram
Software Developer Rail Research UK
United Kingdom United Kingdom
I originally studied for a masters in engineering of software engineering at The University of Birmingham during 2000-2004, of which I received a 2:1. I continued at Birmingham University working with Civil Engineering and Rail Research UK where I am currently in my final year of a 3 year PhD project developing a Computational Intelligent Approach to Railway Intervention Planning. Although my work has a significant focus on railway engineering and associated practices much of my work is with data mining (on SQL Server 2008) and computational intelligence (CI) techniques. My key areas of expertise in CI are clustering algorithms (including Rival Penalised Competitive Learning) and evolutionary algorithms.
 
Outside of my formal work I enjoy testing the latest technologies such as .NET 3.5 and the many frameworks of which it comprises (mainly WPF). I have several projects on the go including a .NET and DirectX port of Quake 3 and many utility libraries. I also maintain an extensive website coded in Cold Fusion which is regularly updated; more information is available about me there.

Comments and Discussions

 
GeneralJava time warp! Pinmember leppie 26-Apr-08 10:16