Wow. Wild digs.
Tell me about it. My googoomomometer is going babies.

Sunday, April 26


Needs For Speeds

Testing various libraries and patterns on Python 2.7.9 and PyPy 2.5.1

Test Python PyPy Gain
Loop 0.27 0.017 1488%
Strlist 0.217 0.056 288%
Scan 0.293 0.003 9667%
Lambda 0.093 0.002 4550%
Pystache 0.213 0.047 353%
Markdown 0.05 0.082 -39%
ToJSON 0.03 0.028 7%
FromJSON 0.047 0.028 68%
ToMsgPack 0.023 0.012 92%
FromMsgPack 0.02 0.013 54%
ToSnappy 0.027 0.032 -16%
FromSnappy 0.027 0.024 13%
ToBunch 0.18 0.016 1025%
FromBunch 0.187 0.016 1069%
CacheSet 0.067 0.046 46%
CacheGet 0.037 0.069 -46%
CacheMiss 0.017 0.015 13%
CacheFast 0.09 0.067 34%
CachePack 0.527 0.162 225%
PixyMarks 13.16 40.60 209%

  • The benchmark script runs all the tests once to warm things up, then runs them three times and takes the mean.  The PixyMark score is simply the inverse of the geometric mean of the scores.  This matters for PyPy, because it takes some time for the JIT compiler to engage.

    Tests were run on a virtual machine on what I believe to be a Xeon E3 1230, though it might be a 1225 v2 or v3.

  • The Python Markdown library is very slow. The best alternative appears to be Hoep, which is a wrapper for the Hoedown library, which is a fork of the Sundown library, which is a fork of the unfortunately named Upskirt library.   (The author of which is not a native English speaker, and probably had not previously run into the SJW crowd.)

    Hoep is slower for some reason in PyPy than CPython, but still plenty fast.

  • cPickle is an order of magnitude slower than a good JSON or MsgPack codec.

  • The built-in JSON module in CPython is the slowest Python JSON codec. The built-in JSON module in PyPy appears to be the fastest.  For CPython I used uJSON, which seems to be the best option if you're not using PyPy.

  • CPython is very good at appending to strings. PyPy, IronPython (Python for .Net) and Jython (Python for Java) are uniformly terrible at this. This is due to a clever memory allocation optimisation that is tied closely to CPython's garbage collection mechanism, and isn't available in the other implementations.

    I removed the test from my benchmark because for large strings it's so slow that it overwhelms everything else.  Instead, append to a list and join it when you're done, or something along those lines.

  • I generally see about a 6x speedup from PyPy.  In these benchmarks I've been focusing on getting the best possible speed for various functions, using C libraries wherever possible.  A C library called from Python runs at exactly the same speed as a C library called from PyPy, so this has inherently reduced the relative benefits of PyPy.  PyPy is still about 3x faster, though; in other words, migrating to PyPy effectively turns a five-year-old mid-range CPU into 8GHz next-gen unobtainium.  

  • If you are very careful about selecting your libraries.  There's an alternate Snappy compression library available.  It's about the same speed under CPython, but 30x slower under PyPy due to inefficiencies in PyPy's CTypes binding.

  • uWSGI is pretty neat.  The cache tests are run using uWSGI's cache2 module; it's the fastest caching mechanism I've seen for Python so far.  Faster than the native caching decorators I've tested - and it's shared across multiple processes.  (It can also be shared across multiple servers, but that is certain to be slower, unless you have some seriously fancy networking hardware.)

    One note, though: The uWSGI cache2 Python API is not binary-safe.  You need to JSON-encode or Base64 or something along those lines.

  • The Bleach package - a handy HTML sanitiser - is so slow that it's useless for web output - you have to sanitise on input, which means that you either lose the original text or have to store both.  Unless, that is, you have a caching mechanism with a sub-microsecond latency.

  • The Bunch package on the other hand - which lets you use object notation on Python dictionaries, so you can say customer.address rather than customer['address'] - is really fast.  I've been using it a lot recently and knew it was fast, but 1.6us to wrap a 30-element dictionary under PyPy is a pretty solid result.

  • As an aside, if you can retrieve, uncompress, unpack, and wrap a record with 30 fields in 8us, it's worth thinking about caching database records.  Except then you have to worry about cache invalidation.  Except - if you're using MongoDB, you can tail the oplog to automatically invalidate cached records.  And if you're using uWSGI, you can trivially fork that off as a worker process.

    Which means that if you have, say, a blogging platform with a template engine that frequently needs to look up related records (like the author or category for a post) this becomes easy, fast, and almost perfectly consistent.  

Posted by: Pixy Misa at 01:28 PM | No Comments | Add Comment | Trackbacks (Suck)
Post contains 1403 words, total size 15 kb.

Thursday, April 23


And Now For Something Educational


Posted by: Pixy Misa at 10:39 AM | Comments (3) | Add Comment | Trackbacks (Suck)
Post contains 6 words, total size 1 kb.

Saturday, March 28


Azunyan Update

So, Cities Skylines at 4K and max settings on a low-end notebook GPU is not a good experience.  Turn down the settings and set the resolution to 1920x1080, though, and it's not too bad.  Not silky smooth, but playable.

The new version of IntelliJ IDEA, my IDE of choice, just came out, and features HiDPI support.  Tried it on my CrazyHiDPI notebook...  It Just Works.™

It's a lot heavier than Chika, but then Chika weighs all of 2lbs, so a 15" notebook with touchscreen and dedicated GPU would be expected to weigh a little more.

Screen is glossy, which is great for colour but not great for working in a brightly lit environment. 

So far, it works, it's fast, it has 4x the RAM and 2.5x the free disk space of Chika, and I'm very happy.

Posted by: Pixy Misa at 02:27 PM | No Comments | Add Comment | Trackbacks (Suck)
Post contains 138 words, total size 1 kb.

Monday, March 23


Azunyan Est Arrivé

I checked this morning and it hadn't shipped yet.

In fact, I checked just now and it hasn't shipped yet.

But it's sitting on my desk right now.  Apparently Star Track Express use a tachyon drive.

Posted by: Pixy Misa at 03:10 PM | Comments (2) | Add Comment | Trackbacks (Suck)
Post contains 39 words, total size 1 kb.


Six Years Counts For Something

So Azunyan will be showing up tomorrow with any luck, and I was wondering just how her GPU will hold up.  It's a Radeon R7 M270, which is a slightly tweaked R7 M265, and not a very fast chip.  It's roughly in line with AMD's better integrated graphics, or Intel's very best, but with the benefit of having its own 4GB of dedicated RAM, and drivers that work.

But according to this comparison, it delivers 80% of the 3D performance of my recently retired Radeon 4850.  And 80% of the 2D performance of my current Radeon 7950, which has no trouble at all with my 4K desktop monitor.

That's better than I expected.  I played Mass Effect 1 and 2 and Dragon Age (the original, accept no other) on that 4850.  Not at the full 2560x1440 resolution of my monitor of the time, but with full graphics details otherwise.

So at 1280x720 up to maybe 1920x1080, depending on the game, it should be adequate.  I'm going to give Cities: Skylines a try at 4K just to see what it looks like; it's not a graphically taxing game but I still expect it to over-extend the M270.

Posted by: Pixy Misa at 12:38 AM | No Comments | Add Comment | Trackbacks (Suck)
Post contains 201 words, total size 1 kb.

Wednesday, March 18


Time For A Change

So Mio is nearly five years old now.  Still working just fine, but five years is a long time in notebookland, and I'm looking at getting one of these.


3GHz 5th generation Core i7 (up from a 2.13GHz 1st generation Core i3), Radeon, Radeon R7 M270 (up from a Radeon 5650), 16GB RAM (from 4GB), 4GB video RAM (from 1GB), and a 256GB SSD (up? from a 500GB HDD).

And a 3840x2160 IPS touchscreen up from a 1920x1080 TFT nontouchscreen.  Touch I don't care about so much, but 4K IPS on a 15" notebook is very nice.

Only obvious problem is that it lacks dedicated page up/down, home, and end keys.  For editing code, that's a pain.

Update: Ordered.  Azunyan inbound, ETA Tuesday next.

Posted by: Pixy Misa at 05:28 PM | Comments (6) | Add Comment | Trackbacks (Suck)
Post contains 128 words, total size 1 kb.

Saturday, March 14


Five Good Things

The world's a mess, as it always is, and Moore's Law is coming to an end.  The automatic cycle of faster-better-cheaper that has helped drive the global economy for thirty years is coming to an end.

But engineers are smart people, and when one avenue for progress closes, they turn their attention elsewhere.  Here's five things to look forward to - some of them, in fact, already on the market.

  1. 3D Chips

    Flash memory is a near miracle combination of cheap, capacious, and lightning fast.  I have a Sandisk 128GB micro SD card; it cost me about $150, stores the equivalent of a million Apple II floppies, and is the size of my little fingernail.  I was afraid I would lose it before it was safely installed in my tablet.

    The problem with flash memory is that it wears out over time.  Not quickly - recent tests of consumer-grade SSDs showed that they last a long time even under appalling workloads.  But as technology advances and flash chips get smaller, an unavoidable consequence is that they wear out faster.  The newer your SSD or SD card, the shorter its lifespan.

    The answer?  Move into the third dimension.  To pack more capacity into its flash chips without making them less reliable, Samsung is now stacking up to 32 individual flash cells on top of each other.  The drawback is that this requires a level of control that we don't yet have in the latest process technologies - around 20nm.  Samsung had to go back to an older process, closer to 50nm.

    But by combining larger memory cells with 3D stacking, Samsung can keep increasing capacities without sacrificing lifespan or reliability.  The 850 Pro and 850 Evo SSDs, already shipping, use Samsung's second-generation 3D flash, and other manufacturers will follow in the next couple of years.

  2. USB 3.1 and the Type-C Connector

    Remember before USB?  When printers had printer ports, and keyboards and mice had keyboard and mice ports?  Modems had serial ports, joysticks had joystick ports, and every mobile phone in the world had a different charger.

    USB swept all that away with a single cable (cough) and a single connector (cough cough).  Yes, since then we've had USB 1, 2, and 3.0, in a total of four different speeds and at least eight standard connectors.

    USB 3.1 changes things just a little.  First, it's 2.4 times the speed of the already fairly zippy USB 3.0, thanks to a higher bit rate and improved encoding standards.  Second, it boosts the power supply standard from about 10W to 100W - from just about enough to charge a tablet to enough to charge a full-size notebook.

    And third, it introduces the Type-C connector.

    The Type-C connector is the same at both ends of the cable.  It works either way up.  It's about the size of the USB 2 micro-B connector, but more robust (at least, it's designed to be), and has up to 100 times the usable bandwidth.

    And if can carry a video signal to drive a 5K monitor at 60Hz.

    Look for it to do to things like HDMI, DVI, and Thunderbolt what USB 1 did to parallel and serial ports, and to notebook power supplies what the micro-B port did to phone chargers.

  3. NBASE-T

    That looks like gibberish if you don't know the history of Ethernet.  Early ethernet ran over shared coaxial cable, and used something called CSMA/CD to control the sharing of the bandwidth.  That worked fine if the network wasn't busy, but congested networks just plain sucked.  Plus, coax cable is a pain to work with.

    It was slowly supplanted by 10BASE-T, a standard for running a 10-megabit network over cheap twisted pair cabling.  That was supplanted by Fast Ethernet on the 100BASE-T (or, really, the 100BASE-TX) standard.  And that has been replaced by 1000BASE-T, gigabit ethernet.  

    There is a standard called 10GBASE-T - 10 gigabits over twisted pair.  The standard was published in 2006, and you can count the number of consumer products implementing it on, well, zero hands.  Even Apple's fancy Mac Pro doesn't have 10GBASE-T.

    The slow uptake has been due to a combination of factors - the power requirements to drive a signal that fast, the chip design required to handle the signal processing, and the cost of ripping out all your old cables because they don't meet the 10GBASE-T spec.

    NBASE-T basically says, well, if we can't do 10 gigabits, rather than dropping all the way back to the 1000BASE-T standard from 1999, let's see if we can run at 5Ggbits, or failing that, at 2.5Gbits.

    If that doesn't sound immediately exciting to you, think of how it would feel running any other piece of computer equipment from 1999.

  4. Retina Displays

    Like computer monitors for example.  The standard resolution - dots per inch - of computer monitors had remained largely unchanged for at least 20 years.

    The screens got bigger, something made possible by LCD panels, because a 30" CRT is not something you can conveniently sit on your desk.  But the number of pixels was tied directly to the size of the display.

    No longer.  Now, thanks to technology that started in the mobile phone and tablet markets, everything is all over the place.  At the common screen size of 27", you now have a choice of 1920x1080, 2560x1440, 3840x2160, or 5120x2880 - anything from 2 million to 14 million pixels.

    The change has been so rapid that it's left video standards behind; 5K monitors require two DisplayPort cables to run at 60Hz, and likewise, a 4K monitor only runs at 30Hz over HDMI unless both your video card and monitor support the very recent HDMI 2.0.

    And companies are forging ahead to 8K.  Which is where it will likely stop, because 5K is already pushing the point of diminishing returns.  Until we get to large, continuous workspace displays - where your desk is backed with a single curved sheet of glass a couple of feet high and six or more feet wide - 8K will do.

  5. 2.5D Chips

    Nvidia have just released their new crown jewel, the Titan X, which is one-and-a-half GTX 980s squeezed into a 600mm2 die.  With 3072 shaders, 8 billion transistors, and 12GB of RAM on a 384-bit memory bus running at 7GHz, it's a technological tour-de-force, and not surprisingly, it will set $999.

    But its reign as the fastest single-chip video card on the block may be short-lived.  AMD is tipped to release their Radeon 390X, which will have a memory bus running at only 1GHz...  But 4096 bits wide.

    This trick is made possible by the prosaically named high bandwidth memory, a process of stacking and connecting individual memory dies vertically, and then presenting a very wide bus - up to 1024 bits wide - to the CPU or GPU.

    The 390X is expected to come in 4GB and 8GB versions, with four memory stacks sitting right next to the GPU die on a tiny interposer board.

    Which leaves the rest of the video card to do, basically, nothing.

  6. Shingles

    As a bonus, Seagate recently released an 8TB disk drive.  It doesn't cost the Earth, either; Amazon list it at $270.  To bring that capacity down to that price, they've played a little trick.

    The read head on modern disk drives has for a while been much smaller than the write head.  After all, the write head needs to force the magnetic domains in the surface of the disk to change polarity; the read head only needs to sense the polarity as they zoom past.

    In fact, the read head is about half the size of the write head.  So in theory, you can read twice as much data from the disk as you can write.  Which is a dumb theory and makes no sense, and that's just what Seagate have done.

    The trick is this: The write head lays down a track of data - like tracks on a vinyl record, except they're invisible and largely intangible and they're concentric circles rather than a continuous spiral.  And then the head moves inwards half a track, and writes the next track of data.  And then it moves half a track again, and writes the next track.

    So the tracks overlap each other like shingles - hence the name.  The read head, being half the size of the write head, has no trouble reading just the half that isn't overlapped.

    The complications arise when you need to go back and change or delete something - you can't just overwrite it directly, because that would wipe out the half-track next to it.

    The solution is to put gaps into the tracks.  I don't know the exact numbers Seagate have used, but let's say they started with a 5TB drive, used this shingle trick to double it to 10TB, and then left every fifth track empty, giving an effective 8TB.

    When you want to overwrite something, you read four tracks from the disk into the buffer in the drive controller, make the change you want, and write them back again.  That means you get twice the storage in the same space, and reading and writing files is as fast as ever, but changing data takes four times as long.

    Since SSDs are phenomenally superior to regular disk drives for storing data that needs to be modified frequently, this makes for a good division of labour: Store your operating system, applications, documents, and databases on your SSD, and your media files - music, video, pictures, books - on the disk.  Because 8TB of SSD isn't cheap, even now.

Posted by: Pixy Misa at 10:04 PM | Comments (6) | Add Comment | Trackbacks (Suck)
Post contains 1602 words, total size 10 kb.

Thursday, March 12


Google > Apple

So two days ago, Apple announced the Macbook Nothing, which depending on your perspective is either a Retina Macbook Air with all the ports removed, or a 12" iPad Pro with full-size keyboard option.

It has one USB Type-C port.  This is the great new standard that will be showing up everywhere in short order.  It's about the same size as the USB 2 micro-B port found on most phones, but it supports 20 times the bandwidth and 10 times the power.

That's the good news....

And that's it.  One USB-C port.  No charging port; you charge over USB.  No SD card slot; you connect to storage over USB.  No video port; you connect to monitors over USB.

Which is a teensy problem if you want to do more than one thing.  Though Apple will happily sell you a $79 dongle that gives you an outdated video connection (no DisplayPort for you!) and another USB port.

Also, if that USB-C port breaks - and while we haven't seen USB-C in the wild before and don't know exactly how well it holds up, micro USB ports have a bad habit of working loose or failing entirely - you have a $1299 paperweight.

Google announced their new model Chromebook Pixel today.  It has a higher resolution screen and a faster processor than the Macbook, optionally more memory, and it comes with two USB-C ports.  One on either side, either of which can be used as a charging port or a video port...  Or a USB port.

It also has two regular USB 3.0 ports.  And an SD card slot.

Image borrowed from Ars Technica because those fuckheads at Google won't even let me view the product page without a US passport.

I don't know for sure what Apple were thinking when they released an expensive notebook with a single port for I/O, video, and charging, but I believe they were thinking "iPad" rather than "getting stuff done".

Posted by: Pixy Misa at 05:54 PM | Comments (8) | Add Comment | Trackbacks (Suck)
Post contains 328 words, total size 2 kb.

Tuesday, March 10


NBASE-T: When 2.5 > 10

I've been seeing references recently to 2.5 and 5 gigabit Ethernet, and was wondering what the hell was going on.

The general run-of-the-mill Ethernet port these days is 1 gigabit per second - Gigabit Ethernet or 1GbE.  Only cheap-and-nasty notebooks still leave you with Fast Ethernet, which is ten times slower.

The next step up is 10GbE - ten times as fast, more than ten times as expensive, and requiring all new cables.  10GbE specifies Cat 6A cables, where basically 100% of installed network cables worldwide are Cat 5e or Cat 6.

NBASE-T, as it is called, takes the technology developed for 10GbE and eases up a little, allowing it to run at 5 gigabits per second over Cat 6 cables, and 2.5 gigabits over older Cat 5e.

Because of the cabling requirements, 10GbE hasn't expanded beyond the server room, where you can easily run a short length of cable to the switch at the top of the rack, and fibre from there to your core router. 

NBASE-T may finally release us from the gigabit tyranny.  Gigabit Ethernet came out in 1999.  To put that in perspective, the fastest networking standard available to end users in 2015 is as old as this thing:


There's a reason they're called "clamshells".

Posted by: Pixy Misa at 11:54 PM | Comments (18) | Add Comment | Trackbacks (Suck)
Post contains 214 words, total size 2 kb.

Sunday, March 08



I love Kairosoft games.  I probably spend more time playing them these days than all my PC games put together.

The problem is that only about half their catalog is available on Android and in English.  They come out first on DoCoMo phones, and then get ported to Android later, and then translated into English later again.  

Which means that I can't play this:

And it looks awesome.

Posted by: Pixy Misa at 06:16 PM | Comments (1) | Add Comment | Trackbacks (Suck)
Post contains 69 words, total size 1 kb.

<< Page 1 of 121 >>
100kb generated in CPU 0.06, elapsed 0.089 seconds.
56 queries taking 0.0431 seconds, 288 records returned.
Powered by Minx 1.1.6c-pink.