Friday, August 18

Geek

Minus Oneth Normal Form

We used to joke about some of the database structures at my previous place of employment - for example, comma-separated lists of name=value pairs - as being in minus-oneth normal form. We were forced to do stuff like that because we were working with huge databases and impossible time constraints, and simply could not afford to take the databases offline to make the changes we really needed, so we had to stick data wherever it would fit.

(There's a good name for this sort of activity: deficit programming. I hate deficit programming.)

Why do I bring this up?

I've just been denormalising our database. We are running, right now, on a hacked-up copy of an ancient version of Movable Type. In preparation for moving off Movable Type, I am trying to get it into a form at least somewhat similar to the Minx structures.

Part of the problem is that MT is highly normalised. There is, for example, no record of how many comments are attached to a particular post. MT executes an SQL query to count them as needed. Minx embeds it within its queries. But being forced to embed a carefully grouped count function within a query that is assembled ad-hoc depending on parameters set by the blog owner and the individual template tags is something of a pain.

So I de-normalised the database. Now we have a comment count field.

There are only two pieces of code that add comments to the system, and I've patched those to update the new field. A database trigger would be a much better solution, but this is MySQL 4.1, and it doesn't have triggers.

Unfortunately there are also several things that delete comments, so I need to hunt those down as well. Or just set up a routine to recalculate the comment counts every so often. Until we move to Minx. Whenever that might be...

Update: And, uh, just breaking the entire blogging system in the process. Stupid MySQL.

Posted by: Pixy Misa at 10:56 PM | Comments (12) | Add Comment | Trackbacks (Suck)
Post contains 337 words, total size 2 kb.

1 So, Pixy, what's null+1?

Posted by: Pixy Misa at Friday, August 18 2006 10:57 PM (FRalS)

2 It's null, of course.  Which sucks.

Where are those bloody nulls coming from?

Posted by: Pixy Misa at Friday, August 18 2006 10:58 PM (FRalS)

3 Take that, evil nulls!

Posted by: Pixy Misa at Friday, August 18 2006 11:14 PM (FRalS)

4 It hasn't sped up the queries much (if at all), so obviously MySQL has some neat tricks to optimise those count() calls.  But it makes the joins a hell of a lot simpler.

Posted by: Pixy Misa at Saturday, August 19 2006 01:02 AM (FRalS)

5 What did speed things up, though, was keeping the modified_on field updated and then adding an index on that field.

Took the forum view from 15 seconds down to about 6 milliseconds.

Posted by: Pixy Misa at Saturday, August 19 2006 03:30 AM (FRalS)

6 So ... Dokuro-chan is still helping you with the database?

Posted by: Kristopher at Saturday, August 19 2006 10:46 AM (O5Ju8)

7

Reminds me of a story a co-worker of mine likes to tell.

Some kids, right out of college, wrote an embedded daztabase application for a place she worked.  They were all excited about "normalization" and had no experience with actual work and so they made the app in what they triumphantly described as 3rd normal form.  There was nothing that wasn't a database field instead of being a value.  Instead of a yes/no logical field, you got a table to hold the values "yes" and "no" for it to refer to.

Of course, as a result, the application literally could not do anything without referencing about 900 different tables.  It ran so slow most of the time no one could even tell if it was actually running.  It never did any useful work.

But this was at the height of the bubble and they were buddy-buddy with the CEO, so naturally they just blamed the hardware and still got paid.

Posted by: TallDave at Saturday, August 19 2006 03:09 PM (oyQH2)

8

I've just been denormalising our database.

Yeah.  Like anything around this place could be described as 'normal.'

Posted by: Wonderduck at Sunday, August 20 2006 01:06 AM (CJ5+Y)

9

So, um, Pixy ol' pal... gots a question for ya.

What's with the alphanumerics after every comment here, eh?  They're all different, except for yours (PiXy! indeed...); database stuff?

Posted by: Wonderduck at Tuesday, August 22 2006 12:13 AM (6YRS5)

10 It's a hash of the commenter's IP address.  I implemented that for Ace of Spades after the big sockpuppet kerfuffle.

Since you're on dial-up, yours changes all the time.

Posted by: Pixy Misa at Tuesday, August 22 2006 12:39 AM (FRalS)

11

I love you.

DB

 

 

 

Posted by: De_Bunk at Tuesday, August 22 2006 10:21 AM (NiWhx)

12 Aw, shucks. :)

Posted by: Pixy Misa at Tuesday, August 22 2006 09:39 PM (FRalS)

Hide Comments | Add Comment

Comments are disabled. Post is locked.
50kb generated in CPU 0.0151, elapsed 0.0992 seconds.
56 queries taking 0.0884 seconds, 349 records returned.
Powered by Minx 1.1.6c-pink.