Oh, lovely, you're a cheery one aren't you?
Tuesday, May 05
Scenario:
At my day job we have millions of images stored in a shared folder. Each of around 800,000 objects is represented by about 20 images in different sizes and formats.
This is shared internally over Samba, because it works for what we need.
To cut a stupid story short, the problem is that Unix is case-sensitive and Samba is case-insensitive. If you are looking for a file and Samba finds an exact match, it's as fast as you'd expect. But if you are looking for a file and it's not there, Samba will scan the entire directory for case-insensitive matches, which is a disaster if you have millions and millions of files.
Solution:
Create a specific share for that directory and set
case sensitive = true
.Mount that in the appropriate place, and problem solved.
Posted by: Pixy Misa at
12:37 PM
| No Comments
| Add Comment
| Trackbacks (Suck)
Post contains 162 words, total size 1 kb.
Monday, May 04
Oops I Poked It Again Edition
Tech News
- I found a clean, simple, reliable way to add a public IP address to an LXC container - and inadvertently poked a hole in my firewall. Fortunately since this was in a test environment the hole led to a container running nothing but SSH and demanding a recognised private key from a list of exactly none, so no harm done.
I updated the documentation for this method.
Moral of this story is that having two nested firewalls like we do at my day job is not overkill.
TL;DR: Static routes take effect before firewall rules. You need to firewall your containers to be safe from future screwups.
- David Icke got banned from YouTube for spreading 5G conspiracy theories. (Newsweek)
Icke is a professional lunatic and his ideas are total garbage. YouTube is completely within its rights to ban him. And I am entirely on Icke's side here.
- The servers for LineageOS, which is the latest in a long line of pure open-source Android builds, got hacked, shut down, restored from backup, patched, hatched, scrutinised with an extreme scrute, and brought back online. (Bleeping Computer)
Apparently a Salt server was exposed to the internet. I'd say "why the hell was a Salt server exposed to the internet" but I just poked a hole in the firewall on my own server and didn't realise it for two hours so I'll give them a pass on that just this once.
- Melinda Gates is an idiot. (Politico)
- Redis 6.0 is out. (antirez)
This brings client side caching and, well, a bunch of smaller stuff. Client side caching means that cached data can be held in your own process space and Redis will push cache invalidation notices to you. For large systems that could have huge scaling benefits.
- The best way to get the right answer on the Internet is not to ask a question; it's to post the wrong answer.
The article is about a perceived case where Go was significantly faster than Rust, but turned out to be that the two programs being compared were not equivalent.
- Intel's Alder Lake will come in Socket LGA 1700 unless it doesn't. (WCCFTech)
Which the hell one is Alder Lake? Oh, it's the 8+8 core design, with 8 real cores and 8 crappy Atom cores. Which might work well for a laptop except that it has a 125W TDP so scratch that idea.
Anime Music Video of the Day
Posted by: Pixy Misa at
09:34 PM
| No Comments
| Add Comment
| Trackbacks (Suck)
Post contains 425 words, total size 4 kb.
The simple manual approach:
Let's say your public IP range is 1.2.3.0/29. This is a pretty typical allocation for a dedicated server.
The selected public IP should not be active on the host interface when you start. If you ping it you should get no response.
- Add the public IP address within the container.
root@container# ifconfig eth0:1 1.2.3.3 up
- Add a static route from the host server to the container. (But see step 4 below.)
root@host# route add 1.2.3.3/32 gw 10.1.1.3
- If you try reaching the container now, chances are it won't work, because the route hasn't been announced to the gateway. Fortunately, that's easy too.
root@host# arp -Ds 216.18.211.180 eth0 pub
In this caseeth0
will likely be something different. You want the main public interface for your server - in my case this wasenp35s0
.
- Make sure to set up appropriate firewall rules in the container. That static route will take effect before the filtering by ufw on the host. (That one surprised me too.)
- That's it. Done.
exit 0
.The advantage of this approach is that since you're not mucking about with netplan files the chance of you locking yourself out of your own server is greatly reduced.
I wish someone had explained this clearly somewhere back when I had Mari, before I had to give it up because I couldn't get networking to work properly and couldn't afford to keep it around but not in production. But Akane is a pretty nice server too. On the CPU side it is much, much faster (about 2.5x single-threaded, about 2x multi-threaded), and it's less than half the price.
Posted by: Pixy Misa at
05:28 PM
| No Comments
| Add Comment
| Trackbacks (Suck)
Post contains 399 words, total size 3 kb.
Someone found the author parameter and decided to try scraping every post on every blog on mee.nu. From twenty different servers at once. They've been blocked now.
That's comparing our current JIT compiler to the new one, not interpreted to compiled.
Now I just have to update the template engine so that it doesn't trash that performance. Accumulating the output using in-memory file operations should do the trick.
Update: Looks like PyPy is optimising some of the sub-tests into oblivion. Adding some randomness into the benchmark to prevent that reduced it to only 6x faster. Since the server also has twice the number of cores, that's 12x overall.
Posted by: Pixy Misa at
01:25 PM
| No Comments
| Add Comment
| Trackbacks (Suck)
Post contains 169 words, total size 1 kb.
Sunday, May 03
Your Sourcier Source For Newsier News Edition
Tech News
- Akane (new server) is nearly three times faster than Aoi (old server) even when Aoi is cheating by using the Psyco JIT compiler.
Psyco helps a lot in optimising loops and function calls - it's 20x faster on a benchmark that just loops and makes empty function calls - but can't speed up the Python code for functions like string manipulation. Akane wins by close to 6x on string scanning.
That's a lot more than the single-threaded difference between a Xeon E3-1230 and a Ryzen 3700X, so it's either a difference in cache sizes (8M vs. 32M) or virtualisation overheads. Aoi is stuck running OpenVZ within KVM, where Akane is running LXD.
I'll run the benchmark under PyPy, but I'll need to excise one of the tests. I test large string concatenations because that's how Minx builds HTML. That's very fast in Python (and Psyco) but offers nothing but pain and suffering under PyPy due to changes in the garbage collection.
- I mention LXD and LXC a lot, which might be a bit confusing. LXC is the container framework, and LXD is a management tool for LXC. So when you are using LXD, the containers are LXC containers and you mostly use the
lxc
command to manage them.
If you are using LXC directly you don't use thelxc
command because that's part of LXD. Instead you use commands likelxc-launch
where LXD useslxc launch
. And if you get the two mixed up you can scramble your configuration.
Clear?
Good.
I miss OpenVZ 6.
- Also, the ASRock IPMI interface on their Ryzen server boards works.
One of the long-standing problems with accessing IPMI over long distances (like Australia to the US) is keybounce on the KVM console. I've often run into cases where it's all but unusable. No sign of it here. Ping times aren't wonderful, but it works regardless.
This means that I can experiment safely with the network config on Akane to get LXD just the way I want it.
- No we don't. (Tech Crunch)
- Nvidia's Hopper architecture may be headed for Samsung's 5nm process or it may not. (WCCFTech)
This is supposedly the next generation after Ampere, which is the next generation.
Interesting point from the table is that the current top of the line Tesla V100 chip is larger than the estimated size of the rumoured 128 CU Navi part. Whether AMD is working on such a chip is still unknown, but TSMC is capable of manufacturing it if they are.
- A new California law requires that default passwords be unique for each unit manufactured. (Serve the Home)
The password is on the service tag.
The service tag is on the server.
The server is in a rack, in a datacenter, 10,000 miles away, and it's 3AM there.
Which is not to say this law is a bad idea, for all that it came out of the Global Bad Law Research and Development Center in Sacramento. But sysadmins will have occasion to curse, vehemently, in coming months.
- There needs to be a Kickstarter for network documentation for LXD BECAUSE THERE FUCKING ISN'T ANY.
There's a specific command,lxc network
, to manage LXD networks, and there's no documentation for it at all, not even a man page.
This is super painful when you are guessing your way to a solution because any time you screw up there's a good chance you'll lose access to your server. Which is why I was testing IPMI today.
Also, if you spin up an Ubuntu virtual machine rather than a container, it comes up with no working networking - and without the standard net-tools installed. And you can't install net-tools because you have no networking to reach the install repo. There is still theip
command but that is a pain in the butt.
Video of the Day
Posted by: Pixy Misa at
09:52 PM
| No Comments
| Add Comment
| Trackbacks (Suck)
Post contains 741 words, total size 6 kb.
Saturday, May 02
Okay You Win Edition
Tech News
- We've all had days like that.
- And who hasn't done this?
- Quibi is a failure while TikTok soars. (TechDirt)
This is because unlike Hollywood, China abandoned communism 40 years ago.
True, they abandoned it for fascism.
- Is Intel up to its old tricks again?
Ian Cutress of AnandTech tweeted - and then deleted - that his sources were telling him Intel was using financial incentives to stop AMD-based NUCs. Particularly now that AMD has a categorically superior chip for small form factor devices.
This and many other questions are discussed, though not necessarily answered.
- Currently running backups to the new server, after which I will figure out the details of the migration.
mee.nu can be switched from CentOS and OpenVZ to a current version of Ubuntu running in LXC behind a proxy with relatively little pain.
The main reason it's fiddly at all is that it uses Psyco - a precursor to PyPy - to speed up the code, but the new server is so much faster that we can do without for a while, until I can check that the code runs cleanly in PyPy,
Well, also that MySQL does stupid stuff every time we do a migration, and loses key settings that break things but only after I've tested it all and gone to bed.
The other stuff still runs on CPanel and not only requires CentOS but proper routing, which I have yet to get working cleanly with LXC.
The big advantage of this move - apart from the dramatically better hardware - is ZFS, which lets me instantly snapshot and back up the virtual servers. That will make future migrations much, much easier if we're willing to take a little downtime, because those snapshots can be restored on any other LXC host with everything intact.
I used to do this with OpenVZ, but OpenVZ 6 has been deprecated and OpenVZ 7 is not compatible.
Update: This is why I like LXC/LXD and OpenVZ 6.
I can tweak the memory and CPU capacities of the virtual machines live. Are we getting extra load on one app and need to give it another couple of cores and an extra 4GB of RAM for a while? Click click, done. No waiting, and most important, no reboots.
Anime Music Video of the Day
I hadn't seen this before, even though it's by Copycat Revolver who has done some of my favourite AMVs ever.
It does make me want to watch Sayonara Zetsubou Sensei.
Other Copycat Revolver AMVs you may have seen here before:
Disclaimer: So, that anime stuff... What's good right now? I have a 1.5TB MySQL database to migrate at my day job (to our new self-hosted cloud thingy) and could do with the distraction.
Posted by: Pixy Misa at
11:26 PM
| Comments (3)
| Add Comment
| Trackbacks (Suck)
Post contains 468 words, total size 4 kb.
I took a look at the SSD and it's a Samsung PM1725. (Storage Review)Ryzen 3700X64GB ECC RAM3.2TB NVMe SSD100TB bandwidth over 1GbUbuntu 20.04
It looks like this is a 48-layer TLC model, but it's an enterprise drive rated for 5 DWPD - which equals 45,000 4k random writes per second for five years. Since our current server averages, um, eight writes per second (though they're closer in size to 40k), I don't think we're going to have a problem there.
Posted by: Pixy Misa at
12:39 PM
| No Comments
| Add Comment
| Trackbacks (Suck)
Post contains 138 words, total size 1 kb.
3.2TB Of What Exactly Edition
Tech News
- So at my day job we're now basically our own cloud platform. Saves us about 85% in costs, except for bandwidth, where it saves us about 98%. Bandwidth charges at AWS, Google, and IBM are absurd.
Except for IBM's global private network, which is free. I have to admit, that is brilliant if you have servers on multiple continents.
- Intel has some new chips out. (AnandTech)
Yes, it's the 10th generation Comet Lake S range - or as the article points out, it's the 5th generation of Skylake, because the architecture hasn't changed for years.
Although it's a boring launch - partly because all the leaks were 100% accurate, and partly because the fastest CPU in the lineup already loses to AMD's second fastest mainstream processor - there are at least a lot of product codes, ranging from the $42 35W dual core Celeron G5900T to the $488 125W (yeah, right) 10 core i9-10900K.
Comparing Intel with Intel, that 10 core part represents about a 70% price cut over the past three years. But at my day job we're ordering nothing but AMD servers now, and that trend is just going to grow.
- A company has patented a general approach for banning social media posts that "spread misinformation". (TechDirt)
Which is great because now they can sue Twitter, Facebook, and YouTube into the ground.
And the EU as well.
Which is - wonder of wonders - actually standing up to China over the latter's propaganda efforts.
- .Org is off. (ICANN)
The planned sale of the .org domain for $1.1 billion has been killed after attracting the attention of everyone from ICANN board members to state Attorneys General.
- Is Xiaomi the new Huawei? (Forbes)
Well, for a start, don't use the Xiaomi browser. How much this affects the phone itself with a non-spyware browser isn't clear.
- Is there a server monitoring dashboard out there that doesn't look like a dog park after an outbreak of canine dysentery?
Thinking of rolling my own using the SmartAdmin template. It's not open source, but on the other hand it doesn't look like doggy diarrhea. I already wrote the monitoring agent - that was my test project when learning Crystal. The collector can be a simple Python + MySQL app.
- Font Awesome's Duotone icons are pretty cute. (FontAwesome)
These came out last year so I'm a bit behind. I was an early bird backer of their Kickstarter so I have not one but two perpetual Font Awesome Pro licenses. So you can expect to see these icons show up here at some point.
- Never let a good crisis go to waste. (Substack)
And if you can't find a crisis, you can always create one. Matt Taibbi weighs in against the neo-fascists over at The Atlantic.
- Mail in a Box is mail in a box. (Mail in a Box)
Basically it's a one-click mail server suitable for deploying on a $5 server at Digital Ocean, Vultr, or Amazon Lightsail.
It's not very configurable, but if you've looked inside the workings of something like Modoboa, that is a good thing. Email servers these days are a complete fucking nightmare. Every idea that everyone has had about email for the past fifty years is still in there, and the config files are infinitely recursive.
- Six lessons learned from a dead iMac. (Tidbits)
1. Target device mode is brilliant - you can boot one device off the disk drive in another over Thunderbolt.
2 through 6. If it was a PC you could just pop it open and swap the drive without going through all this agony. If swapping the disk drive in your computer makes the screen fall off - and you're thinking of buying another one just the same - then you have a problem.
- New server has been ordered. Ryzen 3700X, 64GB ECC RAM, 3.2TB Samsung NVMe SSD, 100TB bandwidth, $129 per month. Will be running Ubuntu 20.04 unless I hit a problem during installation in which case I'll fall back to 18.04.
Posted by: Pixy Misa at
12:37 AM
| No Comments
| Add Comment
| Trackbacks (Suck)
Post contains 682 words, total size 6 kb.
Friday, May 01
Looking at a couple of options. Not going straight for the 3960X given the terrible exchange rates at the moment, but a 3700X with
Update: They ran out of the drives that would have given me the 5TB configuration, and 128GB is a bit expensive if I want ECC (which I do). This configuration is still faster, has more SSD, and nearly as much RAM as all three current servers combined. Oh, and costs much, much less.
I still need to figure out how to get the networking right for CPanel under LXC. I think I know, now.
It doesn't need to be my dream config because they have similar pricing on different configs every week. The reason they're cheap is the same reason the config I wanted isn't available: They buy bulk lots of new-in-box but superseded enterprise components at about 25% of retail price, and when they're gone, they're gone.
At my day job we just deployed 48TB of MLC SSD with the same hosting company for surprisingly little cost - but that drive model launched at the end of 2016.
Posted by: Pixy Misa at
11:57 AM
| No Comments
| Add Comment
| Trackbacks (Suck)
Post contains 199 words, total size 1 kb.
51 queries taking 0.2101 seconds, 366 records returned.
Powered by Minx 1.1.6c-pink.