The ravens are looking a bit sluggish. Tell Malcolm they need new batteries.

Wednesday, April 10

Geek

Daily News Stuff 10 April 2019

It's A Spreadsheet And Therefore Technically I'm Working Edition

Tech News



Ruby Ballroom

  • So today I ventured out beyond the Crystal standard library and...  Uh, yeah, things immediately got rather bumpy.

    Trying to do a shards install of crystal-lmdb failed until I had the bright idea of removing the version specification.  Compiling it failed because it couldn't find liblmdb.a located in /usr/local/lib.

    (LMDB was in /usr/local/lib because I had to build it manually because the version that comes with Ubuntu 18.04 crashes when you run the test script bundled with crystal-lmdb.  Well, good for the test script.

    Fixed that and shards build can't find the library I just installed with shards install. I have no idea why.  It congratulates me on finding a compiler bug.  Yay.
    Error: you've found a bug in the Crystal compiler. Please open an issue, including source code that will allow us to reproduce the bug: https://github.com/crystal-lang/crystal/issues
    I can just drop crystal-lmdb into my source directory and build it - and I did and it works and I can read and write a million (small) database records per second.  But the build tools and their error messages need, let's just say, a little TLC.


Social Media News


Disclaimer: You went full [Orwellian Ministry].  Never go full [Orwellian Ministry].

Posted by: Pixy Misa at 11:59 PM | No Comments | Add Comment | Trackbacks (Suck)
Post contains 525 words, total size 6 kb.

Geek

Film At Elevenish

There's a YouTube livestream of the announcement of the first direct images of a supermassive black hole in the galaxy M87.

It is spectacularly dull.

Posted by: Pixy Misa at 11:10 PM | Comments (1) | Add Comment | Trackbacks (Suck)
Post contains 28 words, total size 1 kb.

Tuesday, April 09

Geek

Daily News Stuff 9 April 2019

This Means You Edition

Tech News

  • Well, yeah, sure the First Amendment says "Congress shall make no law" but it doesn't say "this means you".  (TechDirt)

    Republicans this time, because the urge to control what people say is not just bipartisan but almost omnipartisan.

  • Netgear's new Nighthawk AX4 range of WiFi-6 routers shows the current problem with consumer and SMB networking: All but one model has significantly more WiFi bandwidth than wired bandwidth.  (AnandTech)

    So in theory you can do 6Gbps of WiFi, but you only have 1Gbps upstream to the internet or corporate network.

  • Dell's Precision 7000 workstation supports up to 56 cores, three Nvidia Quadro RTX graphics cards, 3TB RAM, 16TB NVMe SSD, and 120TB of local hard disk.  (AnandTech)

    I'll take two.

  • Has Google stopped indexing old web pages?

    Well, let's see.  I conveniently have a 16-year archive of stuff I can search for right here.

    https://ai.mee.nu/icons/PenguinInfestation.PNG?size=640x&q=95

    It found the current live archive page and direct link, and also the original post from Blogspot before it got imported first into Movable Type and then into Minx.

    Analysis: False.

Crystal Ballroom

  • Crystal is fast, but it's not C++.  Two years ago I experimented with building a runtime for a new programming language (because at the time Crystal wasn't really ready for production) and I was getting 500 million instructions per second on my iMac.  That's not bad for a simple implementation of a register-to-register bytecode interpreter.

    I did a (very) quick rewrite in Crystal and it looks like it's trending towards ~100M ops per second as I implement more opcodes.  Now, I am running the tests on Tohru (3.7GHz Ryzen 1700) rather than Taiga (4.2GHz i7 6700k) which accounts for some of the difference, but that's 15%, not 5x.

    On the other hand, that's plenty fast for an embedded scripting language that runs inside a statically compiled application.

    It might mean I need to use an array of pointers to inline blocks or something like that.  Let me see...

    Update: No, I think what's going on is that everything is bounds-checked (except, as my latest benchmark run just showed, integer overflow) where of course C/C++ says "You're on your own, mate.  Good luck."

    I've worked with Ada previously, years ago, and saw similar 5x performance differences between that and C.

    5x slower is a lot.  But: 

    • Crystal makes it 10x easier to write this code, which means that it might actually get done.  Code that exists is almost infinitely faster than code that doesn't.

    • Crystal itself provides the fast language I need - more than twice as fast as the best case for my C++ based interpreter - so this is just an engine for run-time features like scripts and templates.

    • I can trivially spit out the bytecodes as compilable Crystal code and it gives back that 5x performance and then some.  I got around 900M ops in a test run just now.  (And a faster CPU would easily exceed 1B.)  

      And that is still using the generic data structures of the bytecode engine.  So if you find yourself with a big piece of code that you need to speed up you can compile it and link it in to your binary, and mix and match Crystal, statically compiled bytecode, and dynamic bytecode however you want.

      Side note: If I translate my bytecode to use named variables rather than the bytecode engine's indexed register files, the Crystal compiler optimises it down to a constant and runs in zero time.  And if I turn off the optimiser, the named variable version of the code runs slower than the optimised register file version.

      So I'm not sure exactly how fast that would be in a general case.  The unoptimised named variable code is 15x faster than the unoptimised register file code, but that ratio would be impossible for the optimised version; the Ryzen CPU core can't actually issue that many instructions per second.

    • I'm installing Crystal on my iMac.  A little after midnight, and I hear this sound.  What is that?  It's the fan in my iMac, because the Homebrew install for Crystal builds the entire LLVM compiler framework from source.  That fan never, ever spins up.  I've had this iMac for three years and I don't think I've heard it before.

    • Crystal has a port of the Ruby Language Toolkit, which includes all the stuff you need to build lexers and parsers and compilers, and has a fully worked example of simple language somewhere between Ruby and Python in syntax.  So I can leverage all of that and don't need to start from scratch.

    • Oh, there's a thought.  Crystal is built on LLVM, as my iMac has pointed out rather volubly.  But apart from that, Crystal is written in Crystal (though the very early versions were written in Ruby) so Crystal needs an interface to LLVM, and it has one.  A very extensive interface.  And LLVM has a JIT compiler.  Only problem is reading through a couple of thousand pages of documentation.  Well, something for v2 if v1 ever sees light of day.


Disclaimer: 0th Amendment: This means you.

Posted by: Pixy Misa at 11:26 PM | Comments (5) | Add Comment | Trackbacks (Suck)
Post contains 855 words, total size 7 kb.

Monday, April 08

Geek

Daily News Stuff 8 April 2019

Big In Akkad Edition

Tech News

  • I mentioned yesterday that Python has a library for everything.

    Need to systematise your noun declensions in Classical Akkadian?  There's a library for that.

  • Crystal, as I mentioned, is single-threaded and has no serious inter-process IPC.  If you are building a server application that needs to scale beyond one CPU core that's a problem, unless you are talking over a socket as pretty much every server application does, and are running on Linux kernel 3.9 or later, as pretty much every server does.  In that case:

    • reuse_port to enable multiple processes to bind to the same port (SO_REUSEPORT).

    You can just set the socket and start up multiple independent processes, as many as you like, all listening on the same port number.  And basically that's it.  Client connections go to whichever process grabs them first.  It just works.

    Not on Unix sockets though.  Which means it is merely incredibly useful rather than perfect.  I wondered how the Amber web framework got its performance numbers - whether it was tested behind a load balancer of some kind.  No.  Or to put it another way, yes, but the load balancer is inside the Linux kernel.

  • Speaking of Linux kernels, 5.1 is on its way.  (Phoronix)

  • Optane DIMM pricing has leaked some more.  It is mostly in "if you have to ask" territory.  (Tom's Hardware)

    To be fair, buying 24TB of DDR4 RAM is not cheap either.

  • MIT Technology Review provides some uninvited woker-than-thou advice to Google.

    Burn academia to the ground.  No exceptions.


Social Media News


Disclaimer: The load balancer is coming from inside the kernel!

Posted by: Pixy Misa at 07:23 PM | Comments (2) | Add Comment | Trackbacks (Suck)
Post contains 296 words, total size 3 kb.

Sunday, April 07

Geek

Daily News Stuff 7 April 2019

Tweetle Beetle Bottle Battle Edition

Tech News

  • My adventures in Crystal land continue so far without the crushing realisation that it all, in fact, sucks.  Uploading binaries sucks because I'm still stuck on ADSL, but I'll get Crystal installed on my dev server shortly to sort that out.

    I wrote a little test app to compare performance of HTTP via the Amber framework, raw HTTP by the bundled server, TCP sockets, UDP sockets, web sockets, and Unix sockets.  Over a Unix socket with a single client and server process on my dev server I can get 50,000 requests per second.  That's mostly consumed by the kernel and not by the app itself, because the packets I'm sending are small and I'm just echoing them back again without any real processing load.

    Despite the fact that Crystal is statically typed and compiled, I can write code like this, compile it and run it without a single warning:

    def handle_client(client)
      while message = client.gets
          client.puts message
      end
    end

    def serve_tcp
      server = TCPServer.new("localhost", 3000)
      while client = server.accept?
        spawn handle_client(client)
      end
    end

    def serve_unix
      server = UNIXServer.new("/tmp/amble.sock")
      while client = server.accept?
        spawn handle_client(client)
      end
    end

    So TCP and Unix socket servers can share the same handler code transparently, and I haven't needed to define the type of anything anywhere.  It could easily be Ruby code, except that it's fast.

    The one notable limitation of Crystal right now is that it's single-threaded.  Python and Ruby are multi-threaded but bound by something called a Global Interpreter Lock (the infamous GIL), which means that only one thread can actually be executing Python or Ruby instructions at any time.  This actually makes things faster because otherwise you have to have locks controlling access to datastructures shared by the threads.

    Crystal (currently) just has one operating system thread (plus a second one for the garbage collector) and uses green threads a.k.a fibers for internal concurrency.  You can see that in the above code - every time a connection comes in to the socket I spawn a new thread.  Fibers only use 4K of RAM each so you can have trillions of them if you want.

    Internally it's using an event loop, but unlike a trainwreck like Node.js you don't care about that.  You spawn your little fibers and get on with your life.  Got 10,000 concurrent connections?  Who cares?  That's 40MB of RAM; it costs about 20¢ per month.  If you even stopped for a minute to think about it you just wasted more money than it is likely to cost you.

    And given that it's ten times faster than Python or Ruby for many tasks, it's like being given a multi-threaded language and a 10-core server for free.  Even so, improved concurrency is at the top of the development roadmap.

    I'm liking Crystal a lot so far, with the caveat that library support is (unsurprisingly) far behind Ruby, let alone Python.  Python has a library for everything.  Except VelocyPack.

    Update: Ooh, a brand new LMDB binding!  LMDB is perfect here because it supports multiple independent processes directly mapping the same database.  I did find a couple of earlier bindings but the projects had been dormant a couple of years while the Crystal language has been updated regularly.

    LMDB is kind of magical.  It's pretty good at handling concurrent writes (it's properly ACIDic), though it doesn't deliver the transaction performance of high-end database engines.  But being memory-mapped and copy-free it is phenomenally fast at reads.  I ran some tests in Python a while back reading 1GB per second on a small VPS.

    With LMDB it doesn't matter that your language has a GIL or uses green threads, because you can just run multiple processes.  It handles transactions and crash recovery for you, the database is a single file, and it comes with a hot backup utility where the backups are themselves LMDB databases.  That is, restore time once you have a backup is zero.

    This makes a project I've wanted to do for years suddenly very, very feasible.

  • TSMC has started risk production of 5nm devices.  (WikiChip Fuse)

    This extends the use of EUV throughout the production process and significantly shrinks the geometry, packing about 80% more transistors per square millimetre.  Performance and power improvements are expected to be smaller, on the order of 
    10%.

  • But MongoDB is web scale.

    A good roundup of terrible trends in web development.

  • Browsers are increasingly disabling disabling click tracking.  (Bleeping Computer)

    Except for Firefox among the major players.  Brave also still allows this, and it's likely that MoonPie and Vivaldi and other niche browsers do too.

    I didn't know that the ping attribute was at thing.  That's really useful and much better than using JavaScript to achieve the same end, but users should be able to turn it off.

  • Does Apple's "Magic Keyboard" still suck?  Yes.  (ZDNet)

Video of the Day



Just wait until he starts explaining how to construct five dimensional polytopes out of four dimensional ones.  He even has a model.


Bonus Video of the Day





Disclaimer: When beetles fight these battles in a bottle with their paddles and the bottle's on a poodle and the poodle's eating noodles... ...they call this a muddle puddle tweetle poodle beetle noodle bottle paddle battle.

Posted by: Pixy Misa at 11:42 PM | Comments (7) | Add Comment | Trackbacks (Suck)
Post contains 880 words, total size 8 kb.

Saturday, April 06

Geek

Daily News Stuff 6 April 2019

Crystal Ball Edition

Tech News

  • I bought and read Programming Crystal last night and sat down this afternoon and wrote the little app I needed.

    A bit of fiddling and refactoring and reading the docs for the Crystal standard library (which could do with some fleshing out) and it works.  The code is clean, it compiles quickly, the binary is portable, and it's only 6 megabytes.

    Okay, that last part isn't so great; as soon as I included the HTTP client library it yanked in 3.3MB of extra code.  But that's what static linking does for you.

    Don't concatenate large strings in a loop, though.  Python handles that very well because it reserves space at the end of each string and can reuse the memory.  Most languages don't do that trick and allocate memory and copy the entire string every time, so appending one byte to a one megabyte string is contraindicated.

    This is something you have to watch out for just going from CPython to PyPy with the same code, so it's something I check with every language.

    Oh yes.  As to the IDE question, while there is no convenient CrystalMine from Jetbrains, and the plugin that is available is meh, if you install VSCode and Windows Services for Linux and Ubuntu 18.04 under WSL and Crystal under Ubuntu 18.04 under WSL and the Crystal Language extension for VSCode (the one by Faustino Aguilar) and the Crystal Installer for Windows which does not install Crystal on Windows because Crystal doesn't run on Windows bu instead creates a .bat file pointing to the Crystal on Ubuntu on WSL and puts the .bat file in your path...  It mostly works.

    I also gave the Amber web framework a try.  After some initial swearing when I was editing the wrong copy of my source file, it basically worked.  I've looked at innumerable web frameworks over the last decade and this one is easy to pick up.  It compiles down to a portable binary (if you want).  But it does use the Code of Cancer.  Ugh.

    That crap aside, it's a lot faster than CherryPy, which I have been using for years, even CherryPy running under PyPy.  I'd previously tested Swift with the Vapor framework and it wasn't much faster than CherryPy under PyPy, certainly not enough to justify a switch of programming languages.

    Responding to 50,000 simple queries takes around 70 CPU seconds with CherryPy on CPython, 20 seconds on PyPy, and 1.5 seconds with Amber on Crystal 0.27.  That's a big difference.  (This is all on a small VPS with a single server process and default configuration.)

    This bears some thinking about.  

  • PostgreSQL is an enormously powerful SQL database built by perfectionists and designed by idiots that tells you to go fuck yourself at every possible occasion.

  • Optane DIMMs cost about half as much as RAM.  (AnandTech)

    A 128GB Optane DIMM costs around $900, the price of a 64GB DDR4 module, and a 256GB Optane DIMM at $2850 is about the price of a 128GB DDR4 module.  (Higher density modules are proportionally more expensive.)

    On the other hand, you can only use Optane DIMMs with very expensive CPUs, and they are significantly slower than DRAM (though still lightning fast compared to SSDs.)  So the market for these at the moment seems small.

  • "Ironically" too many video streaming services is leading to an increase in piracy.  (TechDirt)

    That's not irony, you idiots, that's arithmetic.

    All told, there are more than 300 over-the-top video options in the U.S.

    Zombies.  These businesses are funded and run entirely by zombies.  Only explanation.  (Ars Technica)

  • Lessons learned from porting 50,000 lines of code from Java to Go.

    From the publisher of such titles as A Walking Tour of the Sewers of Major European Cities and Leaving Ethiopia for Somalia.

  • A one-day bug bounty program run by Dropbox found 264 vulnerabilities.  (ZDNet)

    Uh, what?

  • A misconfigured Qt5 app and a malicious link can ruin your whole day.  (Bleeping Computer)

  • A sneaky hacker slipping a backdoor into a popular library can ruin your whole day.  (ZDNet)

    Fortunately this one was caught and removed the same day it appeared. 

Social Media News


5K 360° Video of the Day



For those of you with iMacs and gigabit internet access.



Disclaimer: No news is no news.

Posted by: Pixy Misa at 08:08 PM | Comments (4) | Add Comment | Trackbacks (Suck)
Post contains 741 words, total size 7 kb.

Friday, April 05

Geek

Daily News Stuff 5 April 2019

All Day Does Not Include 6:49 AM Edition

Tech News


Disclaimer: There is no 6:49 AM.

Posted by: Pixy Misa at 08:47 PM | Comments (5) | Add Comment | Trackbacks (Suck)
Post contains 217 words, total size 3 kb.

Thursday, April 04

Geek

Daily News Stuff 4 April 2019

Fuck This Shit Edition

Tech News

Social Media News

  • There's a lot of social media news to get to, but it all sucks, so I'm leaving it for tomorrow.  Or maybe the weekend, when I'll have time to work up a really good rant.

Four Dimensional Visualisation Trick of the Day


This video is cool because not only does it explain how Klein bottles work, it gives you a way to accurately visualise many kinds of four-dimensional structures just using familiar everyday concepts.  Ten dimensional hyperspheres not so much; the trick only really lets you jump from 3D to 4D, not beyond.


Disclaimer: Don't braid your flies.

Posted by: Pixy Misa at 11:23 PM | No Comments | Add Comment | Trackbacks (Suck)
Post contains 387 words, total size 4 kb.

Wednesday, April 03

Geek

Daily News Stuff 3 April 2019

Fifty-Six Is The New Forty-Eight Edition

Tech News


Disclaimer: When in trouble or in doubt, run in circles, scream and shout.

Posted by: Pixy Misa at 11:52 PM | Comments (4) | Add Comment | Trackbacks (Suck)
Post contains 511 words, total size 5 kb.

Tuesday, April 02

Geek

Daily News Stuff 2 April 2019

All The Bad Web Pages Are Gone Now Edition

Tech News



Sofa Optimisation Documentary of the Day


Also, if you pack 1024 10-dimensional hyperspherical unit-radius sofas into a length 4 10-dimensional hypercubic moving van (which fits perfectly) the space left over between the sofas is larger than the van.


Disclaimer: Yuh-uh.

Posted by: Pixy Misa at 07:26 PM | No Comments | Add Comment | Trackbacks (Suck)
Post contains 166 words, total size 2 kb.

<< Page 3 of 4 >>
114kb generated in CPU 0.0196, elapsed 0.1493 seconds.
57 queries taking 0.1355 seconds, 373 records returned.
Powered by Minx 1.1.6c-pink.
Using http / http://ai.mee.nu / 371