1080p, HDTV and HD Ready makes me sad

In the old days, when I was a boy, it was usually the case that if you bought a monitor that was larger than your current one (diagonally larger screen), it supported more pixels as well.  These days, it’s sad to see monitor vendors sticking to the flawed idea that somehow, 1080 pixels is the new one size fits all.

If you buy a bigger monitor, you don’t get more pixels, you just get bigger pixels.

This is because monitor vendors have bought into the HDTV size of 1900 x 1080.  Why would anyone want to use anything different?  I think it’s actually because monitor vendors realised they were being dumb.  I mean, people spent thousands of pounds buying larger and larger televisions in the old days, and they never got any increase in resolution?  If people would pay top dollar for huge TV’s at the same resolution as 14″ portables, why the hell couldn’t they bring that business model to the LCD monitor market.

So they did.

There’s a good rant on this over here.

When I bought the LCD’s we use at the moment, I got 5:4 ratio LCD monitors.  People probably laughed.  They’re 19″ displays.  That means (sorry to switch units), that the actual screen is ~30.5cm high and ~37.5cm wide.  That gives about a ~48cm display (diagonal).  We were thinking of getting some new monitors, but I knew it wouldn’t be that easy so I made sure I had the measurements.  These monitors run at 1280 x 1024.  A 19″ widescreen (16:9) might give 1900 x 1080, but it’s vertically much smaller than the monitors we have.  That’s okay, 21″ widescreen?  Still shorter.  22″?  Still shorter.  23″?  Still shorter.  I’d have to buy a 24″ monitor, running at just 56 more pixels high, to give me roughly the same physical height as my existing monitor.  And the screen would be ~20 inches wide (or ~50cm).

To get 56 more pixels (vertically).

And that’s it – you have to be specifically looking to find anything higher than 1080 vertical resolution and you pay for it.  And there’s no good reason for it.  If I want to watch movies, I do that on my television.  So we didn’t buy any new monitors.

I want a choice of monitors, with a choice of native resolutions, in a choice of ratios.

Eight Years

Eight years ago I registered the gemmellmania.co.uk domain.  Sometime in 1999, I registered the gemmell-mania.org.uk domain (don’t go there now, it’s just full of links).  I ran various forums and web sites on those domains, hosted a Usenet FAQ (here, long out of date), ran mailing lists and wrote reviews.

The reviews eventually led to me meeting David, and to being a test reader on some of his later books, something I’m still astoundingly grateful for and proud of.

For a few years now, the site has had only a single page, my tribute to David.  However, the time has come for me to let the domain go.  The tribute is included on this blog as well (here).

The spirit of David’s writing lives on in The David Gemmell Legend Awards.

Debian, apache2, virtualhosts, FastCGI and PHP5

I’ve spent an amusing evening revisiting FastCGI under Apache2, in order to server PHP5 content through Apache’s threaded MPM (worker).  I set this up ages ago on my previous web server and then forgot about it.

It was fine for a long time, but I hadn’t really customised it and to be frank, wasn’t really sure what it was doing.  I just know at the time it was very confusing reading a lot of conflicting stuff on the web.  But it worked.  Until recently, when I noticed the server was running out of memory and processes were being killed.  I didn’t really spend much time looking at the cause though.

When I moved to the new server, I thought I’d try out the prefork MPM again, as per my previous posts and it seemed okay.  However, it’s not okay (although I may do some more load testing if I get a chance).  So I quickly switched back to the worker MPM and FastCGI.

Which is where I started getting frustrated again – I wanted to understand better what’s going on with FastCGI and make sure I was handling it correctly.

If you search the web, there’s a lot of stuff, much of it from 2007 – 2009 with conflicting information and stuff you might or might not need to do.

So, first some caveats,

  1. this is Debian Squeeze, other distributions might be different.
  2. I run PHP5 under FastCGI and nothing else, so my config changes only affect PHP5.
  3. I’m guessing about most of this stuff – so if I’m wrong, please feel free to provide constructive comments.

Here’s what I learned.

Two FastCGI Modules?

Debian comes with two FastCGI modules, those being libapache2-mod-fcgid and libapache2-mod-fastcgi.  They are binary compatible, I’m lead to believe, but fcgid is newer and works better with suexec.  So you should use libapache2-mod-fcgid unless you know you need libapache2-mod-fastcgi for some specific reason.  If you read examples talking about libapache2-mod-fastcgi you can probably just use libapache2-mod-fcgid instead.

Don’t install them both at once – you can do, but there’s no point and it’ll only cause confusion.  You only need one.

Some Fcgid settings are per virtual host.

I run with a low memory setup, so I wanted the PHP5 processes to shut down after a while, rather than hang around.  I couldn’t work out why they weren’t going away.  Also, I couldn’t work out why there were so many.  But it looks to me like you get at least one PHP5 process per virtual host, sometimes more (if the load is high, but remember, these are mostly vanity VPS’s, low load).  The default settings for fcgid processes is to start with none, and create as many as needed, and then drop back to 3.  But it looks like with the way I’ve got it configured (maybe all ways), that’s per virtual host.  I had to set the FcgidMinProcessesPerClass to be 0, so that on each virtual host, fcgid will close all the unused PHP5 processes after a while.

Wrappers?

Most of the articles online suggest you write a little wrapper to launch your PHP5 stuff via Fast CGI.  I couldn’t remember doing that on the previous server and spent a while looking for my wrapper script – until I realised I’d not created one.  You don’t need a wrapper script, but you do need to tell your virtual host to run PHP5 code using the Fast CGI module.  I have this in each of my virtualhost Apache2 config sections.

AddHandler fcgid-script .php
FCGIWrapper /usr/lib/cgi-bin/php5 .php
Options ExecCGI

You need to add the ExecCGI to the Options directive to ensure the PHP pages can be run as CGI apps, and the Handler and FCGIWrapper lines tell Apache2 how to run the PHP.  The default wrapper is just the PHP5 CGI binary (as shown above).  You can put a shell script there and set some defaults, but you don’t have to, it ran fine for over a year on my other server without doing so.

You can set values in fcgid.conf

Because I’m only running PHP5 stuff via Fast CGI, I can happily put settings in Apache’s fcgid.conf file.  Some articles suggest creating a PHP specific one, and putting the wrapper script stuff above in that as well.  I’m sure that works, but so does the way I did it (there’s always more than one way!).  Here’s my fcgid.conf,

<IfModule mod_fcgid.c>
 AddHandler    fcgid-script .fcgi
 FcgidConnectTimeout 20
 FcgidIOTimeout              60
 FcgidMaxRequestsPerProcess  400
 FcgidIdleTimeout            60
 FcgidMinProcessesPerClass   0
</IfModule>

The two Timeout entries ensure unused PHP5 processes are closed down.  The MinProcessesPerClass is required as mentioned above, otherwise it defaults to 3 per virtualhost.  The MaxRequestsPerProcess I’ve set at 400.  PHP will by default handle 500 and then shutdown, it can do that even if Fast CGI has already made a connection, resulting in a 500 error at the client.  If you force Fast CGI to stop PHP after <500 requests, you avoid that issue.  You can if you want write a PHP Wrapper script, and increase PHP’s max requests value, but you don’t have to.

There’s always another way

This is one way of setting it up, there’s always another way, and with Linux there’s usually another 10 ways.  I may do some more testing to narrow down some confusion I still have and see what the benefits of wrapper scripts may or may not be, and whether it’s worth moving some of the per-virtualhost config entries into the fcgid.conf file (like the handler bits).

Apache2 MPM’s

A couple of hours ago I wrote a post about migrating web services to a Debian VM running Squeeze, from one which had been running Lenny.  I said I’d switched to the prefork MPM under Apache2.

Well, if you’re reading this on my site, you’re reading it via the worker MPM once again – only a couple of hours later.  It became obvious pretty quickly once the site had real web pages, and real users, that prefork was not going to cut it.  The VM’s are small, only 256MB of memory and so I can’t run many Apache2 processes.  Although I tested a lot of accesses against PHP based pages using Apache’s AB, I missed doing some testing of both PHP content and the large amount of static content that goes with it (such as style sheets, javascript, images, etc.) at the same time.

Under those conditions, the server needed either so many Apache processes that it filled memory, or it reached the limits I had set and page loads took 20+ seconds.

So, I quickly switched back to the worker MPM and PHP running under Fast CGI, and the page loads are back down to 2 seconds or so on average.

I still have some work to do, to make sure I don’t start too many PHP5 CGI processes, but at least the sites are useable again.

Virtual Machines – taking the pain out of major upgrades

If your computers are physical machines, where each piece of hardware runs a single OS image, then upgrading that OS image puts your services at risk or makes them unavailable for a period of time.

Sure, you have a development and test environment, where you can prove the process, but those machines cost money.  So processes develop to either ensure you have a good backout, or you can make changes you know will work.

Virtual Machines have changed the game.  I have a couple of Linux (Debian) based VM’s.  They’re piddly little things that run some websites and a news server.  They’re basically vanity VM’s, I don’t need them.  I could get away with shared hosting, but I like having servers I can play with.  It keeps my UNIX skills sharp, and let’s me learn new skills.

Debian have just released v6 (Squeeze).  Debian’s release schedule is slow, but very controlled and it leads to hopefully, very stable servers.  Rather than constantly update packages like you might find with other Linux distributions, Debian restricts updates to security patches only, and then every few years a new major release is made.

This is excellent, but it does introduce a lot of change in one go when you move from one release of Debian to the next.  A lot of new features arrive, configuration files change in significant ways and you have to be careful with the upgrade process as a result.

For matrix (the VM that runs my news server), I took the plunge and ran through the upgrade.  It mostly worked fine, although services were out for a couple of hours.  I had to recompile some additional stuff not included in Debian, and had to learn a little bit about new options and features in some applications.  Because the service is down, you’re doing that kind of thing in a reasonably pressured environment.  But in the end, the upgrade was a success.

However, the tidy neat freak inside me knows that spread over that server are config files missing default options, or old copies of config files lying round I need to clean up; legacy stuff that is supported but depreciated sitting around just waiting to bite me in obscure ways later on.

So I decided to take a different approach with yoda (the server that runs most of the websites).  I don’t need any additional hardware to run another server, it’s a VM.  Gandi can provision one in about 8 minutes.  So, I ordered a new clean Debian 6 VM.  I set about installing the packages I needed, and making the config changes to support my web sites.

All told, that took about 4 hours.  That’s still less time than the effort required to do an upgrade.

I structure the data on the web server in such a way that it’s easy to migrate (after lessons learned moving from Gradwell to 1and1 and then finally to Gandi), so I can migrate an entire website from one server to another in about 5 minutes, plus the time it takes for the DNS changes to propagate.

Now I have a nice clean server, running a fresh copy of Debian Squeeze without any of the confusion or trouble that can come from upgrades.  I can migrate services across at my leisure, in a controlled way, and learn anything I need to about new features as I go (for example, I’ve switched away from Apache’s worker MPM and back to the prefork MPM).

Once the migration is done, I can shut down the old VM.  I only pay Gandi for the hours or days that I have the extra VM running.  There’s no risk to the services, if they fail on the new server I can just revert to providing them from the old.

Virtual Machines mean I don’t have to do upgrades in place, but equally I don’t have to have a lot of hardware assets knocking around just to support infrequent upgrades like this.

There are issues of course, one of the reasons I didn’t do this with matrix is that it has a lot of data present, and no trivial way to migrate it.  Additionally, other servers using matrix are likely to have cached IP details beyond the content of DNS, which makes it less easy to move to a new image.  But overall, I think the flexibility of VM’s certainly brings another aspect to major upgrades.

Moaning

I’m going to moan about life, you’ve been warned.

Firstly, I’ve not got much to moan about when compared to some people in the world – but we all live within our own context.  So in the global scheme of things, I’m super lucky.  But that doesn’t stop me being overwhelmed by rubbish stuff.  So, in an attempt at some cathartic release, here we go.

We bought our house about 5 or 6 years ago and haven’t done much to it since, so everything that’s ‘wrong’ with the house is a hangover from previous owners.

Driveway

Our driveway is tarmac.  It’s starting to perish and the rate of failure is increasing.  There’s a long crack from the front to almost the back, and about 2 feet of tarmac is going to slowly fall away.  I suspect when our next door neighbour had a long hedgerow removed to put in a fence, the contractor hasn’t backfilled tightly enough, or the ground has dried out, and so the drive has sunk a little.  Around a water access grate at the front of the drive it’s beginning to crumble, and near the back of the house where the tarmac joins a sheet of concrete patio, it’s sinking into a little hole.

Car

Car is due it’s MOT at the end of March and we know it’s going to fail with various bits.  If they come back and say it’s a few hundred, we’ll pay, if they say it’s a lot more we won’t, but if they come back in the middle ground, it’s always an annoying decision.  Pay and hope we get another year out of it, or don’t pay and just replace it.  Replacing a car isn’t easy for us either, we don’t know anything about them, and when you have no transport, getting around to buy another car is always a depressing experience.  I do not cope well.

Brickwork

A layer of bricks around the bottom of the house is crumbling.  This worries me in a sort of esoteric way, I’m not even sure how terrified I should be.

Electrics

The electrics in the house are a little worrying.  At some point, one of the previous owners did some ‘work’, so there are extensions running all over the place, plugs that appear to hang off other plugs, and various other things that seem clearly wrong to me.  But we’d need to get someone in and review it and then do a lot of work to fix it.  And we never have a single lump of cash to do that with.

Plumbing

The plumbing isn’t bad (especially compared to the electrics), but the bathroom sink hot tap has seized up, and although it’s decreasing in frequency, flushing the loo causes some of the pipes somewhere to vibrate.  Also, the silicon sealant around the bath and sinks has perished and really needs sorting.

Kitchen

The oven is starting to fall apart, and I try and avoid thinking about how the various bits of electrical equipment in the kitchen are actually wired in.

Windows

Our bedroom double glazed window has had a hole in the outside of the glass since we moved in.  The woodwork on all the frames needs serious attention.  One of the previous owners extended the kitchen, and the kitchen window’s outside wooden frame doesn’t actually fit into the brickwork properly.  So last year I found two 1.5cm diameter holes at the end of the woodwork, between the frame and the bricks.  Lovely.

Plaster

The paint on the wall in our bedroom under the window is flaking away.  Either it was put onto the plaster while it was wet, or we’ve got some damp action going on.

Decorating

The whole house is in serious need of internal decoration.