All My Servers Are No Longer The Same Speed

I was planning to spend the weekend working with MongoDB, but those plans evaporated when it crashed and destroyed my test database. So instead I dug out my toy Python benchmark and ran it on Eineus. And just for fun, did the same in Psyco and Cython and Jython. Results are... Mixed. Yeah, that's a good word, particularly since the IronPython benchmark is still running.

System	CPU	Clock	Python	Loop	String	Scan	Total
Eineus	Phenom II 945	3.0GHz	2.6.4/32	0.950	1.483	0.437	2.870
Eineus	Phenom II 945	3.0GHz	2.6.4/Pysco	0.013	0.180	0.477	0.670
Eineus	Phenom II 945	3.0GHz	2.6.4/Cython	0.000	84.750	0.490	85.240
Eineus	Phenom II 945	3.0GHz	2.5.1/Jython	0.682	499.936	0.758	501.376
Nagi	Phenom 9750	2.4GHz*	2.6/IronPython/32	0.544	3502.652	1.541	3504.739
Nagi	Phenom 9750	2.4GHz*	2.6/IronPython/64	0.916	5399.020	1.264	5401.202
Potemayo	Core 2 Duo	1.8GHz	2.6/IronPython/32	0.559	1.632	1.567	3.759
Miyabi	Phenom II 945	3.0GHz	2.6.4/64	0.637	1.003	0.530	2.170
Akane	Opteron	2.0GHz	2.5	1.887	2.733	0.880	5.500

* Normalised to 3.0GHz for ease of comparison.

I'll paste in the IronPython results if it ever finishes. (Update: Done now.)

Some notes:

64-bit Python is now a good bit faster than 32-bit for many cases. It's actually a bit slower in string scanning; I don't know why.

A 3GHz Phenom II running Python 2.6 is 2x faster than a 2GHz Opteron running Python 2.5 from 3 years ago. Someone's been doing some good work, either the Python people or AMD or the Gnu compiler team.

CPython (the standard Python) has some really neat string optimisations that I depend on in Minx. These flow through nicely to Psyco, but are conspicuously absent from Cython, Jython, and IronPython, which are 60, 400, and a couple of thousand times slower for heavy string concatenation (as I said, that benchmark hasn't finished yet...) It's certainly possible to avoid that idiom, and instead, for example, create a list of substrings and then join them all in one operation.

Apart from that, Jython seems to perform fairly well; certainly, if you need to run heavily multi-threaded Python code and can avoid doing millions of concatenations of large strings, Jython could be a winner. The Python interpreter can only run one thread at a time, though other threads can be handling I/O or library functions. The Jython runtime is fully multithreaded, so if you have a multi-threaded application and more than two CPUs - which I do - then Jython can provide an overall performance boost even if the single thread performance declines somewhat. (And it's actually faster on one of the tests, so depending on your code you might win both ways.)

As for IronPython, well, the string concatenation results are just terrible. Looping is comparable with Python or Jython (but far behind Psyco or Cython), and string scanning is the slowest of the lot, though only by a factor of 2, not 2000. It should be fast enough for most tasks as long as you really avoid concatenating large strings. I wonder what list performance is like - I'll have to add a test for that.

Bumped and updated: Tested this just now on Potemayo with a StringBuilder implementation for the strcat part of the benchmark, which delivered about 2000x faster performance (!) on that part of the benchmark and 1000x on average, which would make a .Net deployment of Minx pretty feasible.  It would be nice if all these people deploying immutable string libraries could also deploy the Python concatentation performance trick, because StringBuilder is not a general-purpose string library, just a faster way to concatenate and modify a stringlike object.

Posted by: Pixy Misa at 05:09 AM | Comments (1) | Add Comment | Trackbacks (Suck)
Post contains 527 words, total size 5 kb.