• Western Digital's 18TB Ultrastar DC HC550 and 20TB HC650 EAMR drives do not use a spin-torque oscillator.  (AnandTech)
  
  I know many of you were wondering.
  
  
• Intel's Tiger Lake has reportedly reached speeds of 4GHz.  (Tom's Hardware)
  
  Tiger Lake will potentially deliver 20% better IPC than current Coffee Lake cores, but runs at around 20% slower clock speeds.
  
  Oops.
  
  
• The UK, US, and Australian governments are working together to demand that people stop doing mathematics.  (Gov.UK)
  
  Ijits gonna ij.
  
  
• WTF Python.  (GitHub)
  
  Python is remarkably free of WTF moments compared to other languages, so here's  huge directory of some of the WTFiest.
  
  
• Step One: Ban all existing messaging apps.
  Step Two: Release your own messaging app that is really just spyware.  (New York Times)
  Step Three: Realise that this is more profitable than running a country.
  
  
• Andrew Yang on nuclear power scores 30% in a fact check.  (Bulletin of the Atomic Scientists)
  
  One true, one technically true but misleading, and three false.  Which is a good score for Yang.
  
  
• Star Wars: A New Hope did not hit theatres in 1977.  (Variety)
  
  The film was named Star Wars.
  
  Also, no, the future is not in streaming.  The future is in making movies that don't suck.
  
  
• People are apparently buying AirPods.  (9to5Mac)
  
  A population that will eat Tide Pods can be convinced to buy anything, it seems.
  
  
• The old "Help, I'm being held prisoner in a fortune cookie factory" has apparently come true.  (The Australian)
  
  Or very close.  One Christmas card in a packet sold in England had the extra message:

  We are foreign prisoners in Shanghai Qingpu prison China. Forced to work against our will. Please help us and notify human rights organisation.

I recently - as in, six months ago - mentioned in passing that Hedetniemi's Conjecture had been proven false.  Here's all the tasty deets, as the kids say.

Disclaimer: There was a merry passenger,

Posted by: J Greely at Tuesday, December 24 2019 02:59 AM (ZlYZd)

Hide Comments | Add Comment

It seems to be working pretty well.  I have multiple copies of MySQL running, plus MongoDB, Elasticsearch, Redis, and RabbitMQ, and while it has pushed stuff out to swap, it's not actively swapping, because no single test environment needs more than 4GB.  And it swaps back in pretty fast when needed.

Now I just need to make sure that all the nonsense I've done since yesterday hasn't wrecked the awesome performance I was getting.

Update: Seems to be just as fast as before - over 100 messages per second with a single worker, with all application features enabled, plus ZFS compression.  Read requests in the short-form test suite are slightly slower (up from 6.6ms to 7.7ms on average), but that might be because the PyPy jit hadn't fully optimised that code path.  Doing a full test run now.

Update: Yep, that gave me an average read request time of 6.6ms over the full run.  That's pulling the latest 20 messages from your feed along with all associated information.  With no caching - I have a caching module but it's disabled for now to make sure the core engine is fast.

Also took a look at the ZFS stats.  Both my deduplication ratio and my compression ratio are running at around 2:1, so for the 12.9GB used in my test environments so far, I'm using 3.56GB of actual disk space.

And that's when TokuDB writes its data in compressed format already.  ZFS helps by compressing the unused sections of the TokuDB files, because it allocates 2MB at a time by default.  Since every table and index on TokuDB is a separate file, that can add up.

So in short: The new VPS works great, it runs my test suite close to three times faster than before (which is very welcome given how many times I expect to run it over the next three weeks), ZFS works great, TokuDB works great, LXD works great except for the networking stuff, and even that is manageable now that I know the /etc/hosts trick, and it's all about $20 per month cheaper than the old server.

Posted by: Mauser at Monday, December 23 2019 05:29 PM (Ix1l6)

Hide Comments | Add Comment

Tech News

• The RX 5600 XT may fit right between the 5500 and the 5700.  (Tom's Hardware)
  
  The leaks suggest that the 5600 is not a fully-enabled 5500, but a partly disabled 5700.  The latter approach gives AMD a lot more room to manoeuvre, including giving it 6GB RAM on a 192-bit bus to go directly against Nvidia.
  
  
• Speaking of which. there are reports that the 5500 XT might not perform so well on PCIe 3.0 systems.  (Tom's Hardware)
  
  That is, if:
  
  

  1. You have the 4GB model (this does not affect the 8GB model at all), and
  2. You are playing a game that uses more than 4GB of VRAM.

  
  Apparently it only has 8 PCIe lanes enabled, which works just fine on a PCIe 4.0 motherboard even if you run out of memory, but is a limiting factor if you run out of memory on PCIe 3.0.
  
  So if that happens, turn down the graphics details a notch.
  
  
• Does SMT still make sense?  (Code Blueprint)
  
  One of the things I've had to deal with for a long time is that a server that shows itself as 50% busy has at best 20% of headroom.  That's because (a) the remaning 50% consists of the second thread on each core, and (b) pushing work onto those threads is likely to push your clock speeds down a step or two.
  
  But at the end of the day, 20% free performance is 20% free performance.
  
  
• If you have a Ring camera and haven't yet burned down your house, it might not be too late.  (Motherboard)

  At no point did Ring trigger any sort of alert, such as an email notification, to check that the IP address the system had never seen did indeed belong to the legitimate camera owner.

  
  
• The US Navy has banned TikTok from government-issued phones.  (Reuters)
  
  Good.
  
  
• Vox has fired hundreds of freelancers thanks to California's new law that would deem them employees.  (MSN)
  
  Unexpectedly.
  
  
• You can get Prison Architect and Cities in Motion 1 and 2 for $1 for the next couple of days.  (Humble Bundle)

The film Johnny Dangerously came out 35 years ago today.


Disclaimer: You can tell I'm a living legend, not some ordinary creep.

Posted by: Rick C at Monday, December 23 2019 05:35 AM (Iwkd4)

Posted by: Rick C at Monday, December 23 2019 05:39 AM (Iwkd4)

Posted by: Jay at Monday, December 23 2019 12:56 PM (vuQH5)

Hide Comments | Add Comment

Turns out it's that NVMe drive that makes all the difference.  Write latency is typically between 200 and 300 Âµs on my dev server, but it's consistently 80 Âµs on the Vultr node.

Given that performance and the low pricing, this looks like a good option for production.

I'll try configuring one with ZFS and see how that goes; the default install is EXT4.  Since Vultr provide free full-server snapshot backups and fairly generous storage (the standard $20 server has 80GB of space; the $24 high frequency server has 128GB) ZFS isn't critical if it proves irksome.

(ZFS is brilliant for the dev environment though. because spinning up a whole new virtual server uses about 3MB of disk space, so long as it's the same version of Linux as an existing virtual server.  ZFS can both compress files automatically and merge identical files.)

Update: Thumbs up for Vultr.  Installing a custom ISO couldn't be easier; they even have a list of ready-to-go installers that do a standard setup that will work with your Vultr VPS but let you customise your filesystems.

Update 2: So I ended up spending the afternoon moving my dev environments over to Vultr, because I'm getting around twice the CPU performance and three times the I/O performance.  It will probably work out somewhat cheaper too, since I can make do with a smaller server. 

I hope.  It's configured with 4GB RAM and 16GB of swap, because there's a lot of stuff that gets used maybe once a month, and the idea is it will get swapped out to that nice fast NVMe storage and then swapped back in when it's needed.  If I do need to go for the same memory size as before it will end up costing more, but will also be faster and have more disk space.

• If you put the names of your LXD containers in your hosts file or internal DNS, and match them up to LXD-managed IPs, LXD will actually do the right thing and assign those IPs, rather than its astoundingly unhelpful default behaviour of pulling an IP address out of its ass each time.
  
  If I'd known that a year ago it would have saved me, well, two or three hours, to be honest.
  
  
• It really does look like the Ryzen 4700U will have eight cores.  (Tom's Hardware)
  
  The problem is that it's a laptop part and supposed to run at 15W.  That's not a lot of power to run eight cores and 832 graphics shaders.
  
  
• The Intel 665p is a better version of the 660p, but is it better enough to fend off cheap DRAMless TLC NVMe SSDs?  (Tom's Hardware)
  
  The 665p is hampered by having only 2 PCIe lanes, meaning it can never break the 2GB per second barrier.  On the plus side, the 665p improves direct write performance (once the cache is full) by 50% over its predecessor.
  
  
• Nitter is an alternative front-end to Twitter.  (GitHub)
  
  It acts as a proxy server - your browser talks to your Nitter node, and Nitter talks to the Twitter API.
  
  It uses no JavaScript.  Not on the server, and not in the browser.  That's pretty impressive.
  
  And it's written in Nim, the Python-like compiled language I've mentioned a couple of times when talking about Crystal.
  
  
• Why Nitter?  Perhaps because the modern web is becoming an unusable, user-hostile wasteland.  (Abid Omar)
  
  Twitter is far from the worst culprit in that respect.  Well, the UI and API anyway, the content is another question.
  
  
• Design your web pages to last.  (Jeff Huang)
  
  A good list:
  
  
1. Use vanilla HTML and CSS.  If you need JavaScript, use it; if you don't need it, don't.
   
   
2. Don't minimise.  Your page will get compressed automatically by the web server, so minimisation makes the files unreadable while only saving a few percent.
   
   
3. Just have one great big page.  Uh.  Not so sure about that one.
   
   
4. Don't hotlink.  That includes sites like CDNJS, convenient as they are.  At least make sure you have a copy of anything you load from another source.
   
   
5. Don't use fancy webfonts.  Or if you do, put them on your own server.
   
   
6. Minify your SVG.  Wait, what?  Use WebP, an image format that works with about 20% of apps on a good day.  Um, no.  
   
   Rule 6: There is no rule 6.
   
   But don't embed huge PNG files right in the page.
   
   
7. Use a free monitoring service to tell you when your hosting provider has disappeared and your site is unrecoverable.
   
   So you can spend your weekend trying to extract it from archive.org.
   
   
• Things done fast.  (Patrick Collison)
  
  And things done not fast.  San Francisco has been building a bus lane for 20 years.  The Alaska Highway took less than eight months.
  
  
• Apple is working on top secret satellites.  (Bloomberg)
  
  I think people will work out what's going on when they launch the things into space on a huge fucking rocket.
  
  
• I missed this one before: Data for 1.2 BILLION people was left in an unsecured Elasticsearch server connected directly to the internet.  (WCCFTech)
  
  Elasticsearch, until recently, had no security at all in the open source version.  I mean, sure, MongoDB installs by default without a password configured, but it does nag you to set a password.
  
  Elasticsearch didn't even have passwords in the free version.
  
  
• Now data collected from million Facebook accounts by a third party has been found in another unsecured Elasticsearch server.  (WCCFTech)
  
  To be clear: This is not a server operated by Facebook.  Someone got their hands on the data somehow, put it in Elasticsearch to do, well, whatever, and left it connected to the internet with no password because until recently the free version of Elasticsearch didn't have passwords.
  
  
• San Francisco accidentally banned smartphones.  (TechDirt)
  
  First worthwhile thing they've done since they burned the place down.
  
  
• The Standard Model may be broken.  (Ars Technica)
  
  Have they tried turning the Universe off then on again?

Music Video of the Day

Posted by: Rick C at Sunday, December 22 2019 05:25 AM (Iwkd4)

Posted by: Rick C at Sunday, December 22 2019 05:30 AM (Iwkd4)

Posted by: Rick C at Sunday, December 22 2019 05:32 AM (Iwkd4)

Posted by: Pixy Misa at Sunday, December 22 2019 09:31 PM (PiXy!)

Posted by: Pixy Misa at Sunday, December 22 2019 09:39 PM (PiXy!)

Hide Comments | Add Comment

The back-end is complete and passes the test suite, and I've settled on the front end design.  I'll be posting the first previews in a few days.  

Posted by: Rick C at Saturday, December 21 2019 06:01 AM (Iwkd4)

Posted by: Pixy Misa at Saturday, December 21 2019 10:30 AM (PiXy!)

Hide Comments | Add Comment

• You remember how recently I had to block 175 servers that were crawling mee.nu and killing our performance?
  
  This is why.  (ScrapingBee)
  
  The article explains how to scrape websites and avoid automated blocking and filtering, and recommends using precisely the VPN service that I had to manually block in our firewall, one toxic IP address at a time.
  
  It got posted to r/programming.  People were not impressed.  (Reddit)
  
  
• AMD's Renoir APUs may still be Vega but will be Zen 2 with up to eight cores unless they're not.  (WCCFTech)
  
  It's a benchmark leak and not just a rumour so it looks like eight core mainstream laptops are a go for 2020.
  
  The article also suggests an increase from 11 CUs to 13 CUs on the graphics side.  The graphics benchmark results support that, with close to 20% better performance.
  
  
• ASRock has added support for registered DIMMs to their X299 motherboards unless they haven't.  (WCCFTech)
  
  I didn't know you could do that with a BIOS update.  You still won't get ECC support as I understand it, because Intel's HEDT processors have that permanently disabled.
  
  
• Multipass puts a Linux cloud on your Windows or Mac desktop.  (Multipass.run)
  
  On Windows Home it uses Virtualbox; on Windows Pro it can use the built-in virtualisation.  I think that means you can run Multipass and WSL on the same system, something that is impossible with Virtualbox and WSL.  (At least, if you want Virtualbox to run 64-bit containers.)
  
  It looks like it's essentially LXD wrapped up into a neat little bundle.  I like LXD containers quite a lot and use them for all my development work - I can start up a new Linux instance in a couple of seconds, and then throw it away when I'm done testing.
  
  LXD networking, on the other hand, can die in a fire.  OpenVZ got that right a decade ago.
  
  
• You can't use geometric fields with TokuDB, another limitation.  That one is slightly annoying - not for my current application, but for something else I had planned.
  
  
• France has fined Google $166M for its confusing ad rules that were implemented to placate countries like France.  (Tech Crunch)
  
  It's one of those cases where you wonder if there is a way for them both to lose.
  
  
• Hit indy game Stardew Valley is coming to a new platform: Your car.  (Tech Crunch)
  
  Well, if you drive a Tesla, you can now engage autopilot and spend your commute time harvesting pumpkins and whatnot.*
  
  
• Ghostery Midnight is an ad blocker for desktop apps that decide that they need to also jump on the annoying ad bandwagon looking at you Microsoft.  (Tech Crunch)
  
  Sounds great.

  After a seven-day trial, Midnight will cost $14 per month.

  They're doomed.
  
  
• The HP Elite Dragonfly is...  Wait.  (Tom's Hardware)
  
  Why would you put 32GB of Optane storage in a system with a 512GB NVMe SSD?  What the hell are you doing, HP?
  
  
• A comparison of AMD and Intel's current top-of-the-line HEDT CPUs.  (Tom's Hardware)
  
  This is not pretty for Intel.  Of course Threadripper with 24 or 32 cores is going to dominate the 18 core 10980XE in productivity, but it even wins many of the gaming benchmarks.
  
  This review highlights again that where the 2990WX was an interesting niche CPU for certain high-end tasks, the 3960X and 3970X are brilliant all-rounders.

* DO NOT DO THIS.

Video of the Day

Bonus Video of the Day

Music Video of the Day

• Broadcom is shipping the industry's first 25.6Tbps Ethernet switch.  (Tom's Hardware)
  
  That's 64 ports of 400GbE, so it's perfect for your next home networking project assuming that your home is the Large Hadron Collider.
  
  
• Intel has released the i3-9350KF.  (Tom's Hardware)
  
  Four cores, four threads, no integrated graphics, buy AMD.
  
  
• You can't index JSON arrays under TokuDB, only InnoDB.  Poot.
  
  Overall, though, TokuDB is a pretty clear winner for this application.  The database size is - hang on, let me measure it...
  
  For bulk-inserting 250,000 sample randomly-generated comments:
  
  InnoDB  168M  37.4 seconds
MyISAM  115M  25.1 seconds
TokuDB   79M  12.8 seconds
  
  InnoDB provides better transaction throughput than TokuDB on a high-end server, but TokuDB seems to be substantially better on low-end and virtual servers.
  
  It lacks some features (array and full-text indexes) but the benefits are too good to ignore.  And full-text indexes on InnoDB require a metric shit ton of system resources and are just not viable for my use case.
  
  
• FISA judge slaps FBI.  (TechDirt)
  
  Not directly tech-related, but illustrates why I track TechDirt.  Yes, they're all dirty rotten liberals, but they are principled dirty rotten liberals.
  
  
• Vivaldi now pretends to be Chrome on certain sites, because certain sites are retarded.  (Bleeping Computer)
  
  Shopify, for example.

Posted by: Rick C at Friday, December 20 2019 01:45 AM (Iwkd4)

Hide Comments | Add Comment

• The secret word of the day is DELAY_KEY_WRITE.
  
  If you create a full-text index in MySQL and start shoving data into it, MySQL will try to flush the new index entries to disk as fast as it can.  This can easily produce a situation where posting a single comment writes a megabyte of data to disk.
  
  DELAY_KEY_WRITE says don't do that.  The new index entries sit in the key buffer (in RAM) until they need to be written out to make room for something else.
  
  Downside is that if the server crashes you need to run an index rebuild.
  
  Upside is that - coupled with moving transactional data from the InnoDB engine to TokuDB - it reduced the I/O load by 95%.  Full-text search went from being a pain in the bum to manage, to being a pain in the bum to manage if and when the server happens to crash.
  
  I can live with that.  Need to decouple things a little so that if the search index gets corrupted everything except search will keep running while it rebuilds, but that will take maybe three or four hours.
  
  
• ADATA has 3GHz 32GB SO-DIMMs for all your 3GHz 32GB SO-DIMM needs.  (AnandTech)
  
  I want to try these in my Dell desktops, which currently only have 16GB.  I would upgrade, except that I have five computers with 16GB each so I just run things on different computers when I start to run out of RAM.
  
  Yes, it would have been a lot cheaper to buy more RAM.  I didn't plan to do it this way, it just sort of happened.
  
  
• The new Mac Pro is actually repairable.  (iFixit)
  
  I mean, it should be - it's just an expensive PC in an expensive PC case.  But this is 2019, and it's Apple we're talking about, so it comes as something of a surprise.
  
  
• That's not a whale.  (NewsBTC)
  
  Chinese scammers apparently stole - and then sold all at once - $185 million worth of Bitcoin.
  
  
• The idiots at Microsoft are putting ads in their apps again.  (MS Power User)
  
  You can submit this as a suggestion, Microsoft.
  
  
• Ghibli movies are now in the iTunes store.  (9to5Mac)
  
  I could wish they weren't copy-protected to Hell and back, but at least they are available.

Posted by: Jay at Thursday, December 19 2019 02:04 PM (vuQH5)

Posted by: Pixy Misa at Thursday, December 19 2019 11:06 PM (PiXy!)

Hide Comments | Add Comment

• If you have LXD running on a ZFS storage pool, you can adjust the storage parameters for each container individually.  So if you suspect that compression is slowing down your database container, you can just do a quick
  
      zfs set compress=off my_server/containers/db_container
  
  and future writes will not be compressed and existing compressed pages will be written uncompressed if and when they get updated.
  
  ZFS is one of my favourite things.
  
  It seems to have worked: I'm currently posting (and full-text-indexing) 80 messages per second on my test server (four core / 2.6 GHz / 8 GB RAM) with  less than 40% CPU usage.
  
  
• Samsung has announced two new mid-range phones: The Galaxy A51 and A71.  (AnandTech)
  
  They are very similar devices, with the A51 having a quad Cortex A73 processor and a 6.5" screen, and the A71 having a dual Cortex A76 processor and a 6.7" screen.
  
  2400x1080 display, 6 GB RAM, 128 GB storage, microSD, headphone jack, 32 MP front camera, 12 MP ultrawide camera, 5 MP depth camera, and 5 MP macro camera on both.  The A51 has a 48 MP main camera and the A71 has a 64 MP.
  
  The A71 is also 7 grams heavier.
  
  A51 is around €310, if you live in Poland or Vietnam.
  
  
• You can plug an iPad Pro into Apple's new Pro Display.  (WCCFTech)
  
  Even though the Pro Display is a Thunderbolt device and the iPad Pro lacks Thunderbolt, it actually works seamlessly.
  
  
  
  
  
• A high-end Threadripper deserves a high-end Threadripper motherboard.  (Serve the Home)
  
  Sure, the 3970X is a $2000 chip, and the Asus ROG Zenith II Extreme is an $850 board, but they handily beat the Intel Xeon W-3275 which costs $4500 by itself.
  
  Also worth noting that it uses about 100W less than the last-generation  2990WX while delivering much better (and much more consistent) performance.
  
  And this isn't even the fastest Thirdripper.  That comes next month.
  
  
• A new startup has raised $100 million to build a demonstration fusion reactor.  (The Telegraph)
  
  I looked around and everyone has exactly the same story, but someone at The Telegraph at least bothered to look up the company and add a quick paragraph on what makes this different from a Tokamak.
  
  General Fusion has a quick overview paper on their design.
  
  They've been at this for ten years already.  Fusion is easy, but practical, controlled fusion that produces net energy is not easy at all.

Posted by: Wonderduck at Wednesday, December 18 2019 05:49 PM (cTMj+)

Posted by: Pixy Misa at Wednesday, December 18 2019 07:36 PM (PiXy!)

Posted by: Pixy Misa at Wednesday, December 18 2019 07:38 PM (PiXy!)

Posted by: Wonderduck at Thursday, December 19 2019 10:38 AM (cTMj+)

Hide Comments | Add Comment