On the uselessness of array_merge_recursive()

I guess that most people have run afoul of the monstrosity that is array_merge_recursive(). In short a merge of array('a'=>2, 'b'=>3) and array('b'=>4) gives you array('a'=>2,'b'=>array(3,4)) which isn't the result that one would expect.

So, as usual, I've created a gist that outlines the problem and gives a more sensible alternative.

http://gist.github.com/271920

A future for Services - 3.x?

Services is a module that makes it easy to provide web services using Drupal. What services doesn't do is to allow the definition of separate API:s. This is something we need to fix if we want to turn services into a real web-service framework.

Consider the following: Today all installed services are always available on all installed servers and they all have to use the same authentication method.

This is what makes services practically useless as a building block for other modules, and what makes is nigh on impossible for API:s to coexist in one Drupal install. This worked fine when services only was used for integration with a single Flash/Js/external application; when it was installed by the developer/admin to solve one specific task.

Now we're nearing a point where both blog api and pingback implementors are looking to leverage services for their modules. This is great, as our goal must be that everybody implementing any kind of web service should look to services (core inclusion on the far horizon). Unfortunately it's not possible with the way services is constructed today.

The proposed solution

What I propose, and have gotten quite far with implementing, is to introduce the concept of separate endpoints. Endpoints allows modules and administrators to define separate API:s. By this I mean that each endpoint will provide the following information:

• which services should be exposed
• with what server and on which path they should be exposed
• how should the clients authenticate themselves

The endpoints can be customized completely independently of each other. Endpoints can also be created directly through the administrative interface. The plan is to use ctools for the endpoint information, which will give us exportables for API:s, correct handling of loading of definitions from code and database, overrides et.c.

I'm really exited about the possibilities that this provides, and how it ties in with features. I see it as absolutely essential for services continued development, and not as a good-to-have extra. That is not to say that it needs to be implemented exactly as I've done it. But we need the core functionality.

The current state of 3.x

I'll illustrate with some screenshots from a site running the endpoints-branch from github:

This is the new (when say new throughout the post I mean "my proposed new") first-page for services admin. It lists the services endpoints in the installation and some brief info about the path, server and authentication module that's used, and it lists the active services for the endpoint. The services that are listed may be fully or partially enabled.

This is a shot of the edit endpoint page. Here you can choose the path for the endpoint and which server and authentication module that should be used in the endpoint. Here you can see that the oauth module has gotten a new configuration which is OAuth context, described pretty well in the form: "The OAuth contexts provides a scope for consumers and authorizations and have their own authorization levels. Different services endpoints may share OAuth contexts and thereby allow the use of consumers and tokens across the services endpoint boundraries."

This page allows you to select which services and controllers will be exposed at the endpoint, and allows the authentication module to supply options for each controller.

What I like the most is the ability to define a services endpoints in code. Here we have the conglomerate module that I'm currently working on that defines it's own endpoint. This way modules can expose their own API:s and own authentication settings without disrupting each other.

The same goes for oauth_common, where modules now can define their own oauth contexts. Both these are perfect candidates for chaos tools integration, and ties in perfectly with the features-way of thinking.

Endpoint branches:

• Services
• REST Server
• OAuth Common
• Services OAuth

24hbc

På torsdag och fredag ska Christer och Hugo delta i 24 hour business camp där man ska bygga en internetapplikation på 24 timmar.

Eftersom vi tycker skolutveckling är ett både intressant och viktigt vill vi göra något kul inom det området. Möjligheten att ta lärandet ut i naturliga miljöer är spännande och vi vill därför bygga en iPhoneapplikation som kan användas i en framtida skola.

Hugo och Christer tänker bygga en iPhoneapplikation med utgångspunkt från vad man kunde göra i skolan om:

• Vissa elever hade en iPhone eller liknande enhet, eller om skolan hade några iPhones.
• Lärandet flyttades ut från klassrummet till naturliga miljöer.

Vår avsikt är att bygga en applikation där elever utifrån en uppgift given av läraren kan ge sig ut i sin stad eller naturen och i grupper dokumentera sin lösning på uppgiften. Det kan röra sig om att hitta exempel på vissa typer av arkitektur inom en stadsdel eller att hitta ett antal träd eller andra växter i en park. Eleverna ska kunna ta bilder och skriva text och får det loggat tillsammans med sin position. Deras vandring blir en virtuell vandring på en sajt, som man väl tillbaka i klassrummet kan ta upp på storbild och diskutera i klassen.
Kudos till @Larmats för input om konkreta mobila experiment i Svenska skolor fram till idag.

Målet är att ha en fungerande prototyp klar på fredag och att sedan så fort som möjligt lansera en gratisapplikation i appstore.

Co-Maintainers Wanted!

The list of modules that I maintain has become quite long, and in the beginning of next year I'll have a little daughter (if the nurse guessed right on the gender). So the time that I have for being a good maintainer will be very limited.

If you feel that you'd like to help maintain any of the following modules, I would be very grateful!

• Cobalt
• Node form columns
• OAuth Common
• Services OAuth
• Simple Geo
• JSONRPC Server
• Query builder

Modules not on DO

• Services OOP
• CSS DRY

Experience of using git, or the willingness to learn, is kind of a requirement, as all my development is done with git. The alternative is a patch-based workflow.

Copying Ubuntu VM:s and eth0

Note to self: when copying virtual Ubuntu servers between machines the MAC address will be changed and a eth1 device will be added with the new MAC address. But it won’t be properly configured. Edit `/etc/udev/rules.d/70-persistent-net.rules` and remove the old `eth0` device and rename the `eth1` device to `eth0`. Then reboot and everything should be fine.

If you’re running a local bind9 server, remember to update the zone file so that the A-records match your new IP-address (For me that was `/etc/bind/db.vmdev`) and restart bind: `sudo /etc/init.d/bind9 restart`.

Dynamic vhost configuration

I've recently tried out a new way define my virtual hosts on my development machine. I've always had a configuration file in my home dir (that gets included from httpd.conf) that looks something like this:

Ensuring that values are in an array

I've run across situations where I want to ensure that some values exist in a non-associative array (without getting duplicates). The way I've usually solved it is:

if (!in_array('nid', $fields)) {
  $fields[] = 'nid';
}
if (!in_array('title', $fields)) {
  $fields[] = 'title';
}

This is kind of ugly, and the more values you want to add the uglier it becomes, so it evolved to:

$ensure = array('nid','title');
foreach ($ensure as $f) {
  if (!in_array($f, $fields)) {
    $fields[] = $f;
  }
}

This is great if you have a known array of values that always should exists. But if you have third party code that could have requirements for certain fields to be there, or logic that's more spread out that determines the fields that needs to be ensured, this is the way I've settled for:

$fields = array_fill_keys($fields, TRUE);

// Add a required field
$fields['nid'] = TRUE;

// Add a set of required fields
$fields += array_fill_keys(array('title', 'created'), TRUE);

// Add another set of required fields
$fields += array('modified' => TRUE, 'teaser' => TRUE);

$fields = array_keys($fields, $ensure);

This way we get away with not using loops or conditions by first turning the values into keys in a associative array, modifying them, and then turning the keys into an array again. I guess this is a matter of personal preferences, and this last approach might have the drawback that it's not immediately obvious what's going on for the uninitiated. The other two approaches are more readable.

Convenience scripts for Solr

This is a set of convenience scripts for daily drupal use of solr in a dev environment. When you use the apachesolr module in many projects it becomes somewhat boring to set up solr again and again. And when you have a whole team doing the same thing, this script becomes a time-saver.

This little project lives at github (as usual): http://github.com/hugowetterberg/solr_nightly. Clone or download and execute get-nightly from the terminal (when inside the solr_nightly directory) to download the latest nightly and set it up with the drupal-specific configuration files.

$ ./get-nightly

Then execute start to get solr up and running.

$ ./start

Now your drupal install should be able to use solr through the apachesolr module. Press ctrl+c to quit solr. If you want to clear out all files that's been created for the search index and other runtime files (to use it with another drupal install, or just to start over), run:

$ ./reset

Remember to delete the index from the drupal admin when you have a site that expects that there is a index. Otherwise your nodes won't be re-added.

Re-run the get-nightly script if you want to download the nightly again.

Caching the results of your functions

Sometimes when you use info-hooks to collect information from a arbitrary number of modules, or perform some other expensive or "unknown cost" operation, you'll start to feel the need to cache your results. If your function will be called several times during one request caching in a static variable will take off some of the load, combining this with the Drupal cache could help even more.

Caveats: you probably know this already, but as with all caching it's only effective/sensible to cache stuff if the cost to calculate the results from scratch exceeds the cost of retrieving the results. This is especially important if the database backend is used for caching. If APC or another fast, memory based caching backend is in place, the list of stuff that could benefit from caching grows a bit longer. The other consideration is hit-rate - will your function really be called often enough for it to be necessary for it to be cached? If your cache is invalidated regularly because of writes this becomes even more of an issue, if the cached never is used, or at least not used enough to justify the cost of the actual caching, you will just have written unnecessary code, or in the worst case unnecessary code that just makes your website slower.

Now, the reason for this post: sample code & potential best practice here at Good Old. The following snippet collects information through module_invoke_all() and then lets other modules alter the very same info in a subsequent drupal_alter(). The results is cached in both statically and in the Drupal cache:

Terminal tips and tricks

This is a collection of terminal (not as in life-be-gone) tips and tricks - good things to do with your terminal and bash. It originates from our old google code wiki and contains things picked up from http://blog.macromates.com/2008/working-with-history-in-bash and other places. There are some good tips over here too: http://programmingishard.com/code/tags/bash