When will we be free from Internet Explorer 6?

Another standards-compliant site completed, another day or so consumed developing non-standard workarounds to make it work in Internet Explorer 6. I found myself asking, “why do I need to spend all this extra time deviating from standards just to accommodate one poorly-designed browser?” The unfortunate truth is always, “because over one third of all the site's visitors will be using it.”

When will IE6 finally die, and what will be the straw that breaks its back? Now that XP is no longer sold with new computers, and Windows Update is installing IE7 even on XP, IE6's market share decreased rapidly for a while, but now seems to have levelled off. Although its user base is decreasing, it's not decreasing at any kind of rate that would make me confident that I can stop supporting it when I develop sites.

For anyone who isn't aware of the issue, this browser, released in 2001, fails to respect countless web standards. Of course, some standards came along after IE6 was developed, but this is just no excuse, because every other browser has been patched to accommodate this.

What's very clear is that in the late 20th or early 21st century, a lot of businesses and corporations developed their own internal computer systems using IE5, 5.5 or 6 as a web interface. For example, a bank might have written a customer relationship system that only works with IE6 as a front end. Back then, standards were far less important to web design (because fewer browsers actively supported them all) and were far less… well… standard.

When IE7 came along, a lot of these corporations found that their internal software, on which they had invested thousands or millions, would no longer work, and other corporations were unwilling to upgrade from Windows NT, due to the cost, and therefore could not install IE7 at all. Many workplaces are stuck with IE6. IE7 is not available and other browsers cannot be installed whatsoever.

It might well cost a lot of money to upgrade from NT to XP, Vista or the forthcoming Windows 7, it's true. However, consider how many (wo)man hours, and therefore how much money, has been wasted unnecessarily on developing sites that work in IE6. Consider Microsoft's own 5-year support policy as the cutoff for IE6, which would be about 2006. In this 3-year period, just how much time could have been saved by developing all websites without IE6 support? This would easily outweigh the cost to businesses and corporations of updating their internal systems or operating systems.

What would it take to finally get rid of IE6? I don't think it would take too much. One large-scale site launches a banner on every page, for IE6 users, declaring that they should upgrade their browser or the site will no longer support it in 3 months. Another follows suit. Pretty soon, enough home users are persuaded to upgrade, and enough employees, disgruntled at not being able to check their eBay auctions or Facebook profile at work, pester their employers into upgrading.

The main problem here is one of competition. Putting a banner on your site claiming that you no longer support a browser used by a third of your customers will no doubt send a large proportion of your customers to another site. No large site is going to want to commit commercial suicide like that, because the amount of money they would lose in the short term would outweigh the amount of money they waste on IE6-specific development in the long term.

Instead, the task should be put to the non-commercial websites. Social sites like digg.com or online tools like Google Documents. The problem here is that while the sites aren't actively selling things, they do benefit from revenue brought in by large amounts of traffic, and certainly wouldn't want to take a hit in the number of visitors. I don't believe that the immediate loss of a proportion of traffic, especially from sites that are visited by those with a higher average level of technical awareness (and therefore less likely to be using IE6), would be that great, so I think it is a feasible idea. Digg will still be popular after dropping IE6. Facebook would still be visited by millions using IE7 and other browsers (and in fact, maybe more work would actually get done at work if people couldn't pointlessly update their status every 5 minutes while there).

Supposing one or more larger sites would actually agree to act for the greater good and commit to the outlawing of IE6, how should it be done? The idea would be to gently encourage the user to upgrade, rather than tell the user how stupid he or she is by not using a modern browser. People will respond far better if they think they are upgrading in order to get more out of their favourite site, rather than if they are insulted into upgrading. With that in mind, it could be beneficial to apply IE6-only drawbacks to using a particular site. Perhaps an eBay listing would have fewer pictures and no AJAXy interface in IE6, or perhaps the BBC would be unable to show more advanced, prettier interface elements. Whatever the case, users should be given a carrot and not a stick.

We need to start somewhere, at some point in time, to resolve to rid the world of IE6 for good. Not because it's the root of all evil, or because Microsoft sucks and should die, but because it's genuinely sucking up millions of hours of extra development time and holding back some really creative development techniques. We've already ganged together on the whole to create some fantastic things on the Internet. Wikipedia is a completely user-created free encyclopedia, as an example. Almost everybody in web design dislikes having to create special styles or rules for IE6, yet we seem to just accept that we're powerless to do anything about it, but that's just not the case. We should be putting on the pressure to get rid of an 8-year-old browsing relic. Are you still using the computer you bought in 2001? Would it even work nowadays, with today's Internet? It's about time we pressed a bit harder for change, and then get really creative with the web.

Drupal on Amazon web hosting

Cloud computing has been around for a while, but only recently have we, the general populace, had access to it. Amazon offer one such manifestation: an environment where it's possible to set up any number of virtual dedicated servers and use them for hosting, in our case, Drupal sites! I wanted to share some of my experiences (both good and bad) using the Amazon cloud, so you can make a better decision about whether it's right for your Drupal sites.

This is attractive compared to paying for a shared hosting account. One can never quite be sure what else is running on the system you're sharing, leading to potential performance woes. The Amazon EC2 (Elastic Compute Cloud), as they call it, appears more attractive than smaller companies offering VPSs (Virtual Private Servers) because of the sheer scale of Amazon. It is unlikely to disappear tomorrow and is backed up by the excellent S3 (Simple Storage Service) for backing up data.

Having said that, Amazon solutions can be costly. At the time of writing, Amazon's most modest offering, a reasonably specced dedicated machine with 1.8GB of memory, goes for 11 cents ($0.11) per hour. This doesn't sound like much, but there are 168 hours in a week, and based on the 4.3-week month, that's 722 hours per month, or about $80 per month. Writing from the UK, with the exchange rates as they are at the time of writing, this is about £55.

Still, the alternative for us was to purchase a new machine and buy some rack space (which is billable in advance, often for several months). Compared to Amazon, who bill for usage at the end of each calendar month, with no initial hardware cost, the choice seemed clear.

Starting out, it's striking how little documentation there is. Concepts like elastic IPs, keypairs and elastic block stores are very alien, even to the average techie, and whilst there is introductory material, it feels incomplete. Since the Amazon system is fairly new, and is quite pioneering in its approach, this is understandable, but doesn't make the task any easier.

One of the biggest surprises is that, at the time of writing, Amazon's own web interface does not allow the management of EU-based instances (virtual machines), despite allowing control over US-based ones. If there's one thing that really winds people up on this side of the Atlantic, it's that Americans (and often Canadians) are given preferential treatment with this type of thing. Nevertheless, we were soon able to locate the excellent ElasticFox firefox extension, which allows management of European instances, and we had our very own fresh copy of Ubuntu installed and running at the touch of a button.

This is incredibly powerful stuff, especially because in theory, you can launch as many virtual servers as you like with the click of a button. In practice, Amazon impose sensible limits (although you can apply for more if you genuinely need them) and after all, you're paying per hour for all these machines. We found Alestic's site very valuable indeed. It allows you to quickly find the right Amazon Machine Image (AMI) to use when starting your system, rather than poring over a huge list. A lot of these systems come pre-installed with all the things you're likely to need. There weren't any with specific Drupal installs, but since this is only a 5-minute job (download, unzip), it wasn't too much of an issue.

When you turn off (terminate) an instance, or the power fails at Amazon (very rare but can happen), the instance disappears completely, and so does its local storage. This is a concept alien to those unaccustomed to a VPS environment. After all, if I turn off my laptop right now, the data will be saved to the hard disk and when I start it up again, it's all there. This is not so with Amazon's hosting. The virtual machine powers down and all data stored locally, including program setting and even the operating system itself, is gone forever.

With this in mind, it's clear that a backup solution is needed. Luckily, there are some decent tools in the Amazon EC2 AMI tools package, which is pre-installed on many of Alestic's images. The idea is that you regularly take an image of the entire machine and copy it to S3 where it can be stored safely, and permanently.

Writing a simple script to do this proved more difficult, however. Firstly, it wasn't clear from the documentation that we needed to explicitly state that we wanted to back up the data to an EU S3 bucket. Without this option, the files were sent to a US bucket, taking a very long time indeed and costing $0.17 per gigabyte (a machine image is usually at least 1GB, if not more). Secondly, the bundling process, as Amazon calls it, is less than reliable. Sometimes it would just bail for no apparent reason. Sometimes it would bundle the image and fail during the upload process, again, for no apparent reason. I personally still don't trust the automated backup script I wrote because of these shortcomings, so I find myself checking manually a lot of the time, which diminishes the value of an automated system.

The next concept that was slightly alien was the Elastic Block Store (EBS), which is a system whereby it's possible to create virtual hard disks and mount them to your instances. This is much better than storing files on the instance itself, because if the instance dies, your data are safe. It's possible to take what Amazon calls snapshots of the volumes, enabling a simple backup system, and again, this process can be automated, but you will need to know your way around a shell script, since this is not a point-and-click affair.

The EBS makes it easy to split data into different volumes (database volume, websites volume, miscellaneous volume, etc). Initially we wanted to run MySQL and Apache on the same low-traffic system to see how good Amazon really was, but we always wanted the ability to migrate MySQL to a dedicated machine at a later date. It's a doddle with Amazon: you can simply unmount the database EBS volume from one machine and mount it to another.

We have used the EBS to store some of our more persistant configuration settings too, such as Apache configuration, Apache log files and configurations for the awesome Nagios monitoring system.

To administer these Amazon systems, shell access is needed at a minimum. Unlike other systems where it's possible to simply connect on port 22, Amazon uses a keypair system. Each instance must be created with a specific keypair, and then a key must be downloaded and used with the terminal application (such as PuTTY) before it will allow you to connect. Terminal is nice and all, but sometimes it's useful to do more with the system, like have multiple terminals open or use a GUI tool (like MySQL Administrator). For this, we set up NX, which is similar to VNC in that it provides an interface to the remote machine's desktop that you can use exactly as though it were your own desktop. We found a Google Groups article by Eric Hammond very useful in setting up NX, and thought it was preferrable to VNC because of the default encryption method and insistance on avoiding the root user.

Performance-wise, our Drupal machine has been running Apache 2, MySQL 5, and a host of monitoring software (Munin, Nagios and AWStats) so that we can keep an eye on things, and it has been running for around a month so far with no outages, crashes or other problems at all. The learning curve is pretty steep and the documentation is fairly sparse, but there is a very active community out there on the AWS forums and places like Google Groups. Overall we are very impressed with Amazon as a Drupal hosting environment and although not entirely convinced at the current time, will be looking towards moving more and more sites over there in the future.

Drupal 6: Core hacking for the simplest of things

I was recently faced with a problem concerning changing something in Drupal's core user module. This wasn't even a big change; it was as simple as changing the line of text that appears underneath the username field on the registration form. You know, the one that helps the user to choose an appropriate username.

The core has no option for changing this text, so the most obvious and worst thing possible is to open up user.module in the core and edit the string. This is bad because the first rule of Drupal club is do not hack the core. Actually, the second rule of Drupal club is... you get the idea. It would mean that when Drupal is upgraded, the change will disappear.

After ruling out the above method, I was faced with a number of opportunities to accomplish what I needed, but they all came with downsides.

Override the core

This involves making a copy of the core user module inside my site's modules folder. Drupal will then use the copy instead of the one in the core and changes can be made safely to this copy, because it is not going to be upgraded when a new version of Drupal is released, unlike the core itself.

My main concern here is that technically it's a core hack in disguise. In my case, I would be making a copy of Drupal 6.6's user module and then changing it. But, what happens when Drupal gets upgraded to 6.7? The core user module will be 6.7, but the site won't be using it. Any new or changed functionality in the user module will be missing from the site, and worse, this could introduce incompatibilities.

Suppose something has changed in 6.7's user module, and the other core modules expect this change to be in place? This would break Drupal, because it's not expecting to encounter the 6.6 user module here. How likely is this to happen? I'd say very unlikely in an upgrade from 6.6 to 6.7, but still possible. More than that, it's far more likely to happen once 6.7 becomes 6.8, 6.9 and so forth.

Locale

Drupal's translation system makes it possible to provide a translation string for the text underneath the username field, as though it were being translated into French or Italian, but this feels wrong. It's not a ‘real’ translation because we just want to change the string itself, not change it into another language.

String Overrides

There exists a module called string overrides. It provides a way of changing any text that is passed through the t() function, by saving these translations in the variables table. This seemed like an ideal solution at first, but on closer examination, there are a few drawbacks.

Firstly, since the overrides are stored in the database, we lose version control, which is never a good thing. Secondly, the more modules you add to a site, the more overhead there is, both in terms of memory and in terms of processing times.

Custom module

It would be possible to write a custom module specifically for the site in question. The module would mimic the functionality of the string overrides module, but store the overrides in a file instead of the database, thereby providing version control. The only slight downside to this is that it requires a lot more time than the other approaches. It seems like extreme overkill to have to write an entire module simply to change one string.

To conclude

In the end we decided to go with the core override method. I am not entirely happy with this, but I am sure I would not have been entirely happy with any of the solutions I've presented here. I can acknowledge and understand that there are many compromises to be made on the way to a completely custom Drupal site, but there is always room for improving Drupal to accommodate many of the more common requests. The questions are: is this one of the more common requests? and: what is the best way forward here?

Drupal: Pitfalls when converting CCK field modules from Drupal 5 to Drupal 6

I recently needed to convert a CCK field module from Drupal 5 to Drupal 6. There is a lot to take in, but I was faced with a particular problem:

warning: array_shift() [function.array-shift]: The argument should be an array in D:\wamp\www\drupal6\includes\form.inc on line 1320.

Oh dear. It's always difficult to debug this kind of thing because the problem lies in the code that called the function on line 1320 of form.inc, rather than there being a problem with form.inc. I used debug_backtrace() to see what parameters the previous functions in the call stack were using, and noticed that only the first two parameters in _form_set_value() were populated; the other two were NULL. This was the immediate source of the error. The following line was failing because $parents was NULL:

$parent = array_shift($parents);

Obviously you can't array_shift() NULL. I then went back further in the backtrace, to the form_set_value() function (note the function is not preceded by an underscore, like the last one). In this function, the second parameter ($value) was NULL. This was causing the NULL in the _form_set_value() function.

The solution

It turns out that I was using this code to handle the form widget's processing in my CCK field module:

/**
* Process the mcimage element.
*/
function mcimage_mcimage_process($element, $edit, $form_state, $form) {
 $field_name = $element['#field_name'];
 $field = $form['#field_info'][$field_name];
 $field_key = $element['#columns'][0];
 $value = isset($element['#value'][$field_key]) ? $element['#value'][$field_key] : '';

 $element[$field_key] = array(
   '#type'           => 'hidden',
   '#default_value'  => $value,
   // The following values were set by the content module and need
   // to be passed down to the nested element.
   '#title' => $element['#title'],
   '#description' => $element['#description'],
   '#required' => $element['#required'],
   '#field_name' => $element['#field_name'],
   '#type_name' => $element['#type_name'],
   '#delta' => $element['#delta'],
   '#columns' => $element['#columns'],
 );
}

My mistake? The function does not return anything! It was as simple as adding a return statement at the end of the function, so that the form element could be processed correctly:

/**
 * Process the mcimage element.
 */
function mcimage_mcimage_process($element, $edit, $form_state, $form) {
  $field_name = $element['#field_name'];
  $field = $form['#field_info'][$field_name];
  $field_key = $element['#columns'][0];
  $value = isset($element['#value'][$field_key]) ? $element['#value'][$field_key] : '';
  
  $element[$field_key] = array(
    '#type'           => 'hidden',
    '#default_value'  => $value,
    // The following values were set by the content module and need
    // to be passed down to the nested element.
    '#title' => $element['#title'],
    '#description' => $element['#description'],
    '#required' => $element['#required'],
    '#field_name' => $element['#field_name'],
    '#type_name' => $element['#type_name'],
    '#delta' => $element['#delta'],
    '#columns' => $element['#columns'],
  );
  
  return $element;
}

Drupal: Creating the perfect production environment

For small-time developments, it's simple: buy an account with a Drupal hosting provider, set up your single install, then create the site. This is a live server setup and is probably the simplest way to do things. While this works fine for simple sites that can be thrown together in hours, what is the best way to create an environment in which many sites can be independently designed, created, tested and hosted, while keeping track of large-scale developments? I want to share the method I use. It's not perfect by any means, but hopefully it will be useful to someone, and maybe you'll let me in on how you do things, too.

Version Control

Version control is essential for medium-scale projects upwards and projects where multiple individuals collaborate. Most people have experienced a situation where hours have been spent making a particular change, only to find that it's a mistake, and things need to be changed back to the way they were. A living nightmare under normal circumstances; a two-minute job with version control.

Also, I think a significant number of people are familiar with a situation where multiple individuals have edited the same file, overwriting one another's changes and generally becoming frustrated at the lost time and effort. Difficult to avoid under normal circumstances; a breeze with version control.

I use Subversion for version control, and use the excellent TortoiseSVN for Windows as a client. There is a learning curve involved: for the most part, terms like checkout, merge, branch, and tag will be less than obvious at first, but the ability to easily go back to any version of the file you're working on is soon indispensible and I often wonder how I ever did without it. I wish more things could come with version control. Maybe I've decided I don't like the redecoration I've done in the kitchen. It would be great if I could just revert it at the touch of a button!

Beyond this, version control allows you to keep logs for each commit (each time something is changed) so you know why things were altered, and even to blame each line in the file on the person who committed it, so when you see that line 134 in style.css is messing up your entire page layout, you know who wrote it, when it was written, and why.

The Subversion server I use is its own entity: it is an Ubuntu box that exists purely to serve version-controlled files from the repositories. The repositories themselves are not even on the Subversion server; instead, they're on a NAS (network-attached storage) device, so if the Subversion server dies, the repositories can be retrieved and used with a replacement server. Of course, the repositories are backed up too, because if the NAS device fails and there is no backup, it would be catastrophic.

Individual Development

Each individual working on Drupal projects has a WAMP server on his or her local machine. Drupal is installed here and runs under Apache 2. One important decision here is that there is only a single Drupal installation running multiple sites. These sites reside in subdirectories in the sites folder, and each site has its own repository.

Site-specific modules are installed in the site's own modules folder, whereas modules that will be used on all or most sites (such as cck, webform, etc) are kept in the sites/all/modules folder, which means there is just one codebase for each of these modules. This folder is kept in its own repository, so when modules are updated, changes are committed and a tag is created.

This means that only one member of the production team handles module updates. The others simply update their working copy of the sites/all/modules repository and everything is up to date.

Potential Problems with a Shared Environment

The alternative to having developers work locally on their own machines is to have a shared development server with something like a Samba share that the production team use to access and edit the files. The main drawback with this approach is that it's all too easy to have two individuals editing the same file at the same time, potentially overwriting one another's changes. The style.css and template.php files in Drupal themes are especially vulnerable.

Using local copies and version control all but eliminates this kind of thing. Let's say two people have edited style.css. Person A makes her changes, updates her working copy (this should always be done before committing) and sees that there are no changes, so she commits. Person B makes his changes, and updates his working copy. Assuming that A and B have edited different lines in the file, person B will see that person A's changes have been merged into his changes, so when he commits, the combined efforts of both will be saved.

If A and B have edited the same lines in the file, when person B updates his copy, he will see that there has been a conflict, but will be offered both versions of the file and shown the exact lines that are conflicting. He can then select which side of the conflict ‘wins’, or if neither will be correct, he can alter the conflicting line to incorporate both edits. Nothing will get overwritten unless B wants it to be.

Development and Testing Server

When changes are ready to show to a client, a dedicated development and testing server is used. The working copy on this system is simply updated, and the client is given a special development URL on which to view the work that has been carried out.

Shared Development Database

When I said that the production team work locally on their own machines using WAMP, this is true for the site's files, but not for the database. It's not possible to keep a database under version control, so consider this:

Person A is working locally with a local database and creates a node. There were 10 nodes already, so this is node 11. Person B is working locally with a local database too, and is working on a different aspect of the site. B creates a node too, but in B's database, this is node 11. Therefore, node 11 exists in two places, and is actually two separate nodes. Now imagine this kind of thing happening quite often as various team members work on different bits of the site.

When it comes time to merge all these databases together to create a final testing version of the site, all hell breaks loose. It simply isn't possible to automatically handle situations where the same ID in two different databases refers to two different things.

For this reason, I use a MySQL server on the development and testing system as the ‘central’ development system. Each individual is running his or her own copy of Drupal and has his or her own copy of the files, but utilises the same database. That way, when person A adds a new node, view, user or whatever else, it is instantly visible on person B's copy of the site too.

There are problems with this approach, for sure. Suppose person A has created a view with some templates. The view exists in the database, so it is visible on all of the production team's copies. Person A has not yet committed the templates to the Subversion repository, so they don't exist on person B's copy of the site. The view will almost certainly look very odd to person B until the templates are committed.

Another situation can arise where modules differ. Suppose person A is using a special module on the site to import some content. Once it's in the database, the module can be removed, and nobody else on the production team needs to do anything, because they're all sharing the same database. The module implements hook_menu() to provide an import page where settings can be chosen. When the module is installed on person A's copy of the site, the menu cache is updated to include the path to this import page.

All is well so far, but suppose that person B clears the site's cache for some reason. The import module does not exist on B's copy of the site, so the regenerated menu cache will not include the path to the import page any more. Person A will suddenly find that the menu option for the import page has disappeared! Luckily this is easily resolved (person A simply needs to clear the cache again) but this shows the type of unexpected event that might arise as a result of sharing a Drupal database across multiple copies of the site.

With all that in mind, the benefits of sharing a single database far outweigh the drawbacks, so this is my preferred method, rather than having multiple copies of the database and attempting to merge them together.

Multisite Hostname Problems

One big step I had to take in getting to grips with the multisite approach combined with each member of the production team running their own copy of the site was overcoming the problems associated with Drupal's handling of sites in a multisite environment.

When using a single Drupal install and single copy of Apache for multiple sites, all of Apache's virtual hosts use the Drupal install as their document root, and Drupal selects the appropriate site (and therefore the database, via settings.php) by examining the hostname.

For example, suppose we want to run two sites on the same copy of Drupal: www.foo.com and www.bar.com. In Drupal's sites folder, we create one folder called foo.com and one called bar.com. When a user's browser makes a request for www.foo.com, Apache handles this request, serves the document from the Drupal install folder (with Drupal, it's almost always index.php), and Drupal knows that because the request asked for www.foo.com, it should use the foo.com site and not the bar.com one.

If we apply this to our development environment, our main development and testing server (example.com, for argument's sake) can be accessed from either foo.example.com or bar.example.com, to see each site's development and testing copy. Great, but how does person A access her local copy of the site? She could add an entry to her hosts file to redirect foo.example.com to 127.0.0.1, but then how would she see the copy on the development and testing server? She would have to remove or comment the hosts file entry, which will get confusing, because it will be easy to forget which version of the site she is looking at during any given moment.

Luckily, there's a better way. The folder sites/foo.example.com exists on the development and testing server and also on person A's local copy. Person A alters the Apache virtual host entry for her local machine from foo.example.com to local.foo.example.com. She then adds an entry into the hosts file, directing local.foo.example.com to 127.0.0.1. Now, she can access her local copy by putting local.foo.example.com into her browser, and the development and testing copy by using foo.example.com, all the while keeping the structure of the sites folder the same in both places.

Keeping up with Site Files

Quite often, clients will want to populate their new sites with stories and articles before the site launches. Sometimes, I will want to give them this privilege before all areas of the site are finished. A situation arises where the client has uploaded files such as thumbnails or PDFs. These exist on the development and testing server, but not on any of the local copies used by the production team.

Luckily, SyncToy can help with this. Simply create a folder pair that echoes the development and testing server's files to the local system. Each time any member of the production team notices that their local copy is missing some thumbnails or other files, he or she can just synchronise the folder pair to receive all of the latest files. Alternatively, a schedule can be created to do this automatically at regular intervals.

Of course, this will not help in situations where the production team has uploaded files to their local machines that need copying to the development and testing server, but this is probably going to happen much less frequently, and there is always the option of synchronising both ways for this eventuality.

Live Server Setup

The live sites run on a single Drupal install on a dedicated Apache server, while the live databases run on a separate system that does nothing but run databases (and has MySQL in dedicated mode for best performance). I would always recommend separating these functions out on medium loads upwards, so that Apache does not slow MySQL and MySQL does not slow Apache.

I have not seen any viable clustering solutions for hosting Drupal sites, but luckily I am in a position where all the sites can be run from one web server and one database server. Even so, I would be interested to hear about any successful clusters out there.

Amending the Live Site

Unfortunately, making amendments to the live site is probably the biggest hurdle left in this particular production environment. If there is a rogue CSS class, that's fine. The change is made on a production team member's local system, and tested. The change is committed, then the development and testing server is updated, so that it receives this change too. The change is tested there. Assuming it is ready to be made live, the live server is updated and everyone is happy.

The real problem comes when, for example, a new node needs to be added. Suppose that the client has asked for a new webform. First, the live database is copied down to the development and testing server. A member of the production team produces the webform on her local system. This shares the database with the development and testing server, so unless template or CSS changes are required, the job is done and it can be tested.

The problem is that suppose there were 100 nodes before the work began. On the development and testing database, the webform is node 101. Now suppose that in the time taken to create and test the webform, 5 nodes have been created on the live site. The development and testing database cannot simply be copied back over, because this would wipe those 5 nodes!

One solution is to put the live site into maintenance mode while the webform is created. This way, the database will be locked off to the general public, so it can be copied down to development and testing, worked upon, then copied back up without the risk of wiping content.

This might be fine for a 5-minute job, but what if the work requires 4 hours? The site cannot very well be put into maintenance mode for 4 hours during the working day just so a small change can be made!

The answer at the moment seems to be to write everything down. Write down the settings you use to create the new webform. Once it has been tested and is ready to go live, just create the webform again on the live server. This is not sophisticated or classy in any way, but appears to be the best option.

Your Input

I would be interested to hear thoughts on the way this particular production environment is set up and how it might be improved, especially where I have noted problems. I'd also love to hear how you do things and why, because I think I learn a lot from other people's experiences.

Drupal 5: user_save and profile fields

I was recently required to import a large number of users into a Drupal 5 site, so I wrote a simple import module to take rows from a CSV file and pass them to user_save(). In addition to the basic user information in the {users} table, I needed to create several profile fields too. This was incredibly complicated, but probably shouldn't have been.

The first thing I noticed is that the documentation for user_save() is not exactly stellar.

$account The $user object for the user to modify or add. If $user->uid is omitted, a new user will be added.

Fine, but is this where I should be putting my new user's information, or perhaps I should use the next parameter?

$array An array of fields and values to save. For example array('name' => 'My name'); Setting a field to NULL deletes it from the data column.

Ok, fine. Maybe I should use this one for adding my new user data. This doesn't mention anything about the profile fields though, or explain what I should be doing with the $account parameter. Maybe there's something else?

$category (optional) The category for storing profile information in.

What? Category? Now I'm really confused. There's nothing obvious in user_save() that suggests how the profile fields get saved, or even where to put the profile fields. My only real clue is the call to user_module_invoke() towards the end of the function. This calls hook_user() in all the active modules on the site, and the one I'm interested in is the profile module, so my next stop was profile_user(). In turn, this calls profile_save_profile() with the details from the original call to user_save().

It was at this stage that I noticed that $category must refer to the various groups that profile information can be put in. For example, you can create a category for personal information, and one for notification preferences, and doing so will split the fields onto different tabs when the user edits his or her profile. Unfortunately, $category is a string, not an array, so for each call to profile_save_profile(), only one category can be changed.

Because profile_save_profile() is only called once per user_save(), it appears that when creating a user, it is only possible to create profile fields in one group! This causes a problem for me because I needed to import lots of profile fields in several groups.

My solution was to temporarily move all the profile fields into a single group. Once I had done that, I could populate $array with the information destined for the {users} table and the profile fields (this was not documented anywhere). It turns out I could just use NULL for $account (again, this was not documented).

Surely this is not the ideal way of creating new users programatically. My solution does work, but it is annoying and time-consuming. Is there another way to create users with profile fields in multiple categories?

Drupal 5: Don't rely on the node's path attribute

I recently noticed that some of the nodes on a site I was working on were not linking properly from the teaser to the full node view. It turns out that the hyperlink tag's href was empty. Checking the template for the node teaser, I found that it was using this:

print l($node->title, $node->path);

This appears to work fine for nodes with a path alias set up (via the path or pathauto modules), but not for other nodes, because the $node->path part was empty. I believe the following code should have been used, to always start with the base URL for the node, and let the l() function choose the most appropriate alias:

print l($node->title, 'node/' . $node->nid);