Tuesday, February 27
I had a couple of must-do items before I started the Minx beta tests:
- Switch from the built-in CherryPy session storage to a MySQL back-end
- Not run the darn thing as root
They should be simple.
In the case of the CherryPy session storage, the CherryPy docs are (a) incomplete and apparently (b) wrong. It looks like the only way to plug in your own session storage module is to directly hack the CherryPy source. It's not supposed to be that way, but if I have to, I can do that.
In the case of not running as root, well, there's a couple of ways to do it. The main problem is that Minx has to listen on port 80, and only root can listen on port 80, and running as root is a Bad Idea.
One way around this is to change the userid after startup, when the port is already established. I'm not sure why, but this causes Minx to lock up after the first request. Another is to use Apache and mod_proxy, but there's no way in hell I'm going to all that trouble. I'm doing all this in large part to get away from frigging Apache. Another way is to use a reverse proxy like Pound. I tried that, and it kind of worked. (It looked like it was broken, but that was actually due to the lockup problem I mention above. I expect if I try it again it will work flawlessly... Yep. Okay, so that at least isn't time wasted; Pound could well prove useful as we roll out more servers.)
Another way is to use iptables redirects. This is the simplest and most reliable way, and doesn't bloody work on my system. It works fine on my other Linux box, but not on the Minx test box, and not on the Minx production box. Why, I don't know. But iptables dnat does work, as it turns out, two hours of intense frustration later. I'm guessing that the redirect was redirecting to the wrong IP - localhost, or something. Minx binds itself to a specific IP so that I can run multiple instances per server, so that I can balance across the CPUs. (Python is multi-threaded, but only one thread can be executing Python code at any time. OS calls are multi-threaded, but the data crunching isn't. So to get the most out of Minx on our multi-processor servers, I'm using multiple instances of Minx and round-robin DNS.)
Uh, anyway. Currently working in a secured environment with file-based sessions. That will do for the beta test, and allows me to check off item 0 on my bug list. That leaves 22 items, plus one marked "bah".
Update: Bandwidth logging, which required using CherryPy callbacks, something I hadn't used before, worked second go. Much better.
Monday, February 26
Not so much reorganising the code to better fit the design changes, as reorganising the theoretical design to match the actual design.
We now have four types of site... And thirty-
sixeight types of folder. (Standard types. You can create your own arbitrary folder types, but they don't get any system support beyond the usual template magic.)
So site types (blogs, wikis, journals, forums, and so on) are just packages you drop on to your main site; each such function is represented by a folder (or a set of folders). The type of the site is merely the type of the default folder for the site. Which is the way I implemented it from day one... but not what the design document said.
Throws a small wrench into my already monkeyed implicit containers, and I will need another database change or two. Plus, it adds, let's see, 3449 new template tags. No, that's not right, I forgot the dot-ops. Well, in total, there are now 6878 template tags relating to folder-level operations.
I did mention that they are highly orthogonal, right? Good.
File uploads are working.
Drag-and-drop image insertering is working.
File management is working... ish. You can't delete files just yet, but that's not a big task to fix.* And if you try to rename a directory to have the same path as an existing directory, the results are undefined. At best. I think I'll just disallow that.
I still have a bunch of little things to fix. For example, I thought that the drag-and-drop image feature wasn't working, but that was because the
allow.html flag wasn't getting set on my test posts, so all the HTML in those posts was escaped, which made the images not work, which was what I was testing, so I thought that that was where the problem was, until I finally decided to check the data getting written into the database and found that that was correct even though the final HTML in the page was wrong, which meant that it was an errant filter causing the problem and not the WYSIWYG editor.**
Which is good, because the filters are a whole lot easier to fix than the editor. Cute editor, but it's more code than the whole rest of Minx at the moment.
Attachments aren't working just yet. It would be very easy for me to get them to work, but I really want to get them properly integrated with the file uploads, and there's a little conceptual difficulty there relating to the ownership of the files and the permissions to manage content on the sites involved. I probably have a couple more days of work to get that sorted out.
PS2 S-Video cable: Check
PS2 memory card: Check
PS2 gamepad: Check
FF XII: Check
PS2 power cord: Hmm. Looks like a standard figure-8 240V socket. Must have a few of those in the cupboard... Upstairs... Somewhere... Bah.
* Actually, you can't delete anything just yet. I have a set of delete functions defined, but they don't do nothing. I've been leaving them for a rainy day, which seems to have arrived.
allow.html flag should be cascading from the site (via the default or current folder auth record) and the user preferences... but it isn't, so it's off by default, which makes posts not work.
Uh, damn - I have to demo this tomorrow. I think I might just stick a 1 in there for now.
Saturday, February 24
Steven rereviews Popotan and comes to much the same conclusion as I did.
It's the dancing dandelions. It may take some time, but they will get to you in the end.
Picked up my copy of Final Fantasy XII today.*
I have sworn to finish the file management module before I unwrap it.
* It was released here in Oz yesterday - four months after its US release, and nearly a year after its release in Japan.
That's the one thing I really don't like about the console game market. PC games reach Australia either at the same time as the US release, or at worst within a couple of days. Console games can take six months or more. This is extra annoying with the Xbox 360, which is HD and therefore not limited by regional video standards - so Microsoft made their own, entirely arbitrary, regional fuckupification.
Thursday, February 22
The second most confusing and problematical aspect of Minx is the system of implicit containers.*
I've mentioned before that Minx has multiple hierachies, defined by Sites, Folders, Threads, Posts and Attachments. You can have as many of each as you want, and they can be nested any way you want, and can be intersected and unioned and what not in many strange and wonderful ways.
For a simple example, let's say you want to go to http://ai.mu.nu/blog/post/thats_going_to_need_stitches
The ai.mu.nu part specifies the site; the blog/post part specifies the folder; and the thats_going_to_need_stitches specifies the thread.** Which is all fine and dandy. That thread will likely have a category assigned to it, which is in fact another folder - in this case the "geek" folder - so that it can also be reached at http://ai.mu.nu/blog/geek/thats_going_to_need_stitches (or something like that). And that category, even though it's actually just another folder which is attached to the entry in question, is to your template an implicit secondary immediate parent structure, so it has to be automatically provided as an appropriate set of template tags.
With me so far?
The problem with this is context. My test site just blew up because the inline comments decided that they needed to be paged (so that only the first 20 comments were displayed inline, and for the rest you had to go to the next page); the problem being that because the page itself was a folder context rather than a thread context, the pager data for the comments (which are posts) wasn't set up. The fix was simply to have the comment paging code check first whether the requisite data was set up... which was a good idea in any case.
This just gets more complicated when you consider that certain objects - the Entry object is a good example - combine the aspects of two or more other objects. An Entry is essentially a Thread combined with a Post, but it also provides access to one or more Author objects and optionally a Folder object, and has to automatically create all the contexts involved. And since an Entry is always viewed inside a Folder context anyway, this can get a little fiddly.
For me. For the user, it's supposed to Just Work™.
One of the solutions I've put in place is explicit substructure tags to complement the implicit tags. Example: The [comments] tag creates a list of comments relevant to the current context, which might be a Folder, a User, a Thread, a Post, or possibly something else. It works out the applicable context automatically, prepares the necessary SQL queries, processes the data, and interprets your template for you. Easy-peasy, unless you wanted it to do something different.
So now, in addition to the semi-automatic [comments] tag and the fully-automatic [comments:here] tag, you have a set of tags such as [entry:comments], [user:comments], [folder:comments], [site:comments], and so on.
The problem with this is that there are, at last count, 12 specialised variants of the [sites] tag, 12 variants of [folders], 6 of [threads], 6 of [posts], and currently 9 of [attachments], and every one of them has to have a complete set of explicit substructure (and superstructure!) tags. So my structural tag table is now dynamically rather than statically generated.
The saving grace, or rather the two saving graces, are these: First, the code for this is largely O(1); the whole template engine works on hash tables, so this doesn't slow it down; and second, because all these variants are variants rather than explicit structures in their own right, this is all actually very well organised. The [category] structure is independent of the [folder] structure (and the [section] structure and the [area] structure and the [archive] and [subforum] and [directory] structures and so on...) but the fields in each one are identical. So as long as you know that [category] is a variant of [folder], you're all set.
How I'm going to structure the documentation for this I don't quite know yet.
* The most confusing and problematical aspect of Minx right now is the user interface customisation system. It doesn't give you a life-jacket, just a whole lot of rope. It's not hard to customise it to the point that you're effectively locked out of your own blog. I will be putting a failsafe in there that lets you log in to the maintenance system using the default settings, but that's not there yet...
** Well, in terms of a blog, it specifies the entry, but keep reading.
I probably still have a 10Base-T hub around here somewhere. We've come a long way, baby.
Wednesday, February 21
I hope to open the gate to the first group of beta testers later this week; large scale beta-testing will follow in March.
Tuesday, February 20
- Planets are small things, rarely supporting more than a handful of villages.
- The two most dangerous professions in the galaxy are Space Marine and Senior Medical Officer.
- All pre-industrial civilisations are blissed-out hippies.
- All post-industrial civilisations are stupid.
- All industrial civilisations are psychotic.
- You can get away with anything if you can plausibly lay the blame on a parallel universe.
The Age, 19th February 2007, morning edition:
WORLD opinion emphatically rejects the idea that Islam and the West are heading for an inevitable clash of civilisations, according to an ambitious poll of public attitudes across 27 countries, commissioned by The Age and the BBC World Service.The Age, 19th February 2007, afternoon edition:
India train blasts an 'act of terror'Seems that some people can't be bothered to read the paper.Twin blasts aboard a train bound from India to Pakistan that killed at least 64 people were probably "an act of terror", a spokesman for India's Prime Minister Manmohan Singh says.Southern Thailand bomb attacks kill threeAt least 28 bombs exploded in apparently coordinated attacks in parts of southern Thailand plagued by a Muslim insurgency, killing at least three people and wounding more than 50, the military said.Baghdad bombings leave 60 deadA double car bombing ripped through a crowded Baghdad market today, killing at least 60 people and wounding scores more in a first vicious blow to the city's new US-led security operation.Russian McDonald's rocked by blastAn explosion caused by a suspected bomb in a McDonald's restaurant in Russia's second city of St Petersburg on Sunday injured at least five people, officials said.*
* The Russian attack has not been tied to Muslim extremists, but from the same article:
Anti-Moscow rebels linked to Russia's Chechnya region have mounted a spate of bomb attacks on civilians in Russia but there have been no major attack outside the turbulent North Caucasus region for more than two years.
56 queries taking 0.3598 seconds, 371 records returned.
Powered by Minx 1.1.6c-pink.