Friday, April 12

Geek

Daily News Stuff 11 April 2019

Ftruncate Does Not Truncate Mmap'd Files Edition


Tech News

  • HP's new Xeon workstations also go up to 56 cores and 6TB of Optane memory.  (AnandTech)

    Prices start at $2372 and continue up past $100,000.

  • The EU has identified a great amount of terrorist content hosted on archive.org including Project Gutenberg, a collection of Grateful Dead bootleg recordings, and CSPAN.  (TechDirt)

    Under the EU's new censorship law - no, a different one - archive.org would have one hour to remove that content or face criminal prosecution.

  • Firefox has removed Dissenter from their add-on gallery.  (One Angry Gamer)

    Mozilla were quite clear on this: If your extension is used by people to say things Mozilla does not like, it will be removed.

    Web browsers have to be neutral to retain users' trust.  Without that they are dead.  I don't think any of the major browser companies remember that fundamental rule.

  • Ethereum continues to be a pain in the ass.

  • Oh joy, an API method that returns success when the value is unknown.  Good work guys.

Crystal Ballroom

  • LMDB doesn't work properly on WSL, which is also a pain in the ass. There are a couple of known bugs in WSL's mmap that give you happy little segmentation faults in place of working code. That also applies to Docker on Windows.

    I was looking at building an app in Crystal that you could run on Windows either using Docker or WSL, but this blows that up. The other thing I could do is write a wrapper that emulates LMDB with SQLite, which has the advantage of running with 100% reliability on everything from supercomputers to vacuum cleaners.  Or just use SQLite as SQLite.  I know, radical idea.

    And static linking on MacOS doesn't work yet either.

  • On the other hand, when it does work, I can read and write 300,000 native Crystal records per second sequentially, single-threaded.  Or, and this is important, 180,000 random reads per second.  That's converting everything to JSON and back again, unpacking nested data structures, all that stuff.

    Random access time is around 2-2.5µs, with the same going for JSON decoding.  Throughput should scale linearly with core count but I just tried it and it doesn't, which is kind of annoying.

  • Hmm.  How fast is SQLite then?

    Read 2000000 records in 00:00:01.68

    Okay, that should do.  (Those are smaller records, so it's probably about the same speed as LMDB.)


Disclaimer: WSL's mmap behavior different from native Linux.  Native Linux work, but WSL(1438cool segmentation fault.  Here is sample code.

Posted by: Pixy Misa at 12:15 AM | Comments (4) | Add Comment | Trackbacks (Suck)
Post contains 425 words, total size 4 kb.

1 Did you WANT that smiley in your disclaimer?

Posted by: The Brickmuppet at Friday, April 12 2019 01:30 PM (xOgT9)

2 I think the smiley really ties this disclaimer together.

Posted by: Pixy Misa at Friday, April 12 2019 04:39 PM (PiXy!)

3 If I write a blog post that is critical of Mozilla, are they going to blacklist mee.nu in Firefox?

Posted by: Pete Zaitcev at Monday, April 15 2019 03:33 PM (LZ7Bg)

4 They're probably not that far gone yet.  But I'm going to be paying more attention to the smaller browsers now, that's for sure.

Posted by: Pixy Misa at Monday, April 15 2019 04:23 PM (PiXy!)

Hide Comments | Add Comment




Apple pies are delicious. But never mind apple pies. What colour is a green orange?




51kb generated in CPU 0.018, elapsed 0.1133 seconds.
58 queries taking 0.1014 seconds, 344 records returned.
Powered by Minx 1.1.6c-pink.