Yesterday Marcus and I made our way out to Walnut Creek to meet with Marc Canter in order to discuss the lay of the land, as well as introduce him to the Open Data Definition. Marc is an industry luminary who is excited by the possibilities of social software (when it's done right) and pushing very hard for what I believe are the technological developments needed for it to reach its potential.

As he's made evident in his subsequent blog post, he gets completely what we're trying to do with Open Data Definition. In particular, this is the core part of our thinking:

OpenDD not only covers all the schema stuff and what’s needed to facilitate data inter-change but its also grounded in a pragmatic approach to updating and synchronizing disparate system’s data structs together which are often dynamic and which are thought of completely differently by each system.

All too often, standards are opinionated: they force you to adopt their definitions. That's all and well and good when you're building secondary software (tools that rely on an upstream API from a particular application), but when you need to bridge two applications with equal weighting, you can't pin down the terms. It's unrealistic to get two applications on a platforms with different featuresets to use the same vocabulary across the board (although clearly for subsets of data it's possible: see RSS).

We're very excited about the possibilities here. The conversation is developing; we've had some very positive feedback, and we're exploring integrations with Atom, OAuth and XRDS-Simple. As ever, we encourage you to go and join the mailing list and get stuck in.

I've guest posted over at The Social Web:

The Open Data Definition is a new format for the import and export of data from within social applications. Rather than an academic exercise, it’s driven by necessity; hopefully, software companies and individual developers who feel the same pressures will join the conversation and build it with us. Data portability is an important issue, and it needs to be solved with practical solutions that work in the real world.

Read more.

This week we released a draft specification for something called the Open Data Definition, an extremely simple format designed to allow for easy data portability between services.

Like all the best things, Open Data Definition came about because we needed it ourselves. It's already built into the Elgg 1.0 core. Rather than replace the existing formats espoused by groups like DataPortability.org, the format sits comfortably alongside; we also plan on supporting every single logo you see on the DataPortability site in our own software. Each standard represents a facet of openness required for interoperability between services, and Open Data Definition fills an important gap: generic import and export.

In theory, the web is supposed to work as an interlinked network of connected resources that each sit in their own space. In practice, however, there are many times that a user might want to export all their data from one service and import it to another. Here are some:

• The service you're using to host a particular piece of data is shutting down, or isn't working the way it should.
• You were hosting some data on a third party service, but now you want to host it yourself.
• You're running a web-based intranet and you need to move twelve employees from one group to another, which has a completely different intranet.

Generally speaking, you expect software on your desktop to be able to be able to read and write files that another similar piece of software has created. Web applications should be no different.

Because the data schema of every application is different, we boiled the model down to the essentials: Users, which are people; Objects, which are things that people create and interact with; Sites, which are web services. Each of these can take on a subtype label ("blog post", "photo"), have unlimited amounts of arbitrary annotations and metadata, and be linked with relationships. A set of extensions for various applications can then sit on top; for example, one for social networking applications, which defines "friend" relationships.

Each descriptive item of data is atomic. That means that you can take, for example, a piece of metadata - "Ben Werdmuller likes sushi" - and sit it in an ODD file by itself. That file will still be readable by a parser, which can find out more about Ben Werdmuller if it needs to. You could therefore have a stream of these atomic updates and use them as a changes syndication file, like a social version of RSS, or even synchronise two services by having them broadcast a continuous stream of ODD updates to each other.

There are a couple of things ODD doesn't do by design. It doesn't define its terms. It's true that "friend" on Service A might mean something different to "friend" on Service B, but we think that's probably up to Service A and Service B to negotiate on their own terms. Similarly, no fields or field types are concretely defined for metadata; if you want to stick an XBox Live ID or where you went on your last vacation as one of your profile fields, you can export that as ODD without having to worry about defining namespaces for them or anything too complicated. (ODD does use namespaces, but only to define the broad extensions I described above. ODD no longer uses namespaces.) It's up to the service you're importing your data into to decide what to do with it.

The format is absolutely open for development discussion. There's a draft spec available for download from the OpenDD site, or you can join the mailing list. We'd love to hear what you think.

Philanthropy on the Commons, an article over at openDemocracy that was brought to my attention today, is though-provoking:

At the end of his essay - and in his accompanying book, Just Another Emperor: the Myths and Realities of Philanthrocapitalism (Demos/Young Foundation, 2008) - Michael Edwards asks what he calls the $55 trillion question: how will we use the vast amount of new philanthropic resources that will be created in the next fifty years? My instincts tell me that Wikipedia, open source and peer-production may hold part of the answer. The world of the commons has used openness, participation and community to create real and (hopefully) lasting public goods. Why not apply these same principles to improving education, creating low-cost housing or evolving our democracy?

The irony is that in many ways these principles are democracy, or at least, a more accessible version of the idea than we traditionally see at work in wider society.

I think to really understand what people are doing with ideas like crowdsourcing and wikis, you need to look back to what was happening in Silicon Valley forty years ago. To me, it feels like there's a direct link from the likes of Wikipedia (a free encyclopedia that anyone can edit) back to the likes of People's Park (a plot of land in downtown Berkeley ostensibly for everyone, at least until Governor Ronald Reagan sent in 250 state troops and claimed it back). In the sixties, the battles were huge - civil liberties, the draft, the tipping over of stale political and social ideas that had dominated for decades. Echoes could still be felt in the seventies: when Steves Jobs and Wozniak put personal computers together in a garage, they had traditionally been the realm of large companies and the very rich. Their generation allowed information technology to become more accessible to the mainstream, and similarly, our generation is allowing an immense amount of information to become accessible to the mainstream. Often, this is information we should always have had, like the kind hosted on TheyWorkForYou or Wikileaks.

The web works like this: anyone can stick up a page containing whichever content they like. Anyone can read it. If they like, they can also stick up a page, and link to the other pages they happen to think are good. The resulting graph is the basis for a lot of very lucrative commerce - it forms the basis of Google's success, for example - but it's also incredibly democratic, and again, very radical if you look at it as opposed to the model we used to have. Any idea can be published. As that concept moves beyond text, through the academic establishment and entertainment industries, the established gatekeepers have to adapt or die. The publishing houses used to control the printed word; the record labels are finding that they have less and less control over what music people listen to; it's only a matter of time before film and video also succumb to becoming a democratic market. And from there, there's nowhere to go but out.

Fifteen years ago, I logged onto the Internet for the first time. Many people had never heard of it at that point; it took the mass market a few more years to start surfing the web. But since then, it's caught society's imagination, and like wildfire has become the basis (or at least, an extremely important part) of everything we do. It's only a matter of time before the principles that are redefining business will also begin to redefine our society, in a very similar way to how the radicals forty years ago once believed they should.

People's Park photo by Areta, under a CC Attribution-No Derivative Works 2.0 Generic license.

We're having a couple of events in various places over the coming months, where we'll get together to talk about Elgg. In all cases, look for a devilishly handsome group of gentlemen clad in the fetching black tees you see pictured to the right. Come say hi; we'd love to talk to you.

San Francisco Elgg Meet: Wednesday, May 7 3pm - 8pm
Thirsty Bear Brewing Co., 661 Howard Street (at 3rd)
http://upcoming.yahoo.com/event/473595/

Elgg Jam 08: Wednesday, June 18, all day
Roxy Bar and Screen, 128-132 Borough High Street, London
http://elggjam.com/

Marcus Povey will also say a little about the forthcoming Open Data Definition at the Oxford Geek Night on April 22nd:
Jericho Tavern, 56 Walton St, Oxford
http://upcoming.yahoo.com/event/432766/

I identify strongly with Marc Canter's attitude towards his ancestry; I'm not in the slightest religious, but am also interested in the traditions of the various aspects of my family. I've got a very mixed background; parts of my ancestry left Ukraine during the time of the pogroms, parts are directly descended from occupants of the Mayflower (William Brewster, I think), parts are Swiss, Dutch and Indonesian. However, I'm interested in it all, and like Marc, actively proud of it.

Although I was actually kind of resistant to my eclectic background as a teenager (like many kids that age, I was desperate to fit in), I think the time has come for me to study those traditions, and the direct history, more closely. Yet another way that the Internet can really enrich our lives.

There's been some discussion about social publishing platforms, which Dave has covered well over on his blog. It's maybe worth taking a stab at defining the various types of web publishing systems out there:

Content Management Systems allow a person or organisation to easily modify static, content-driven websites, without resorting to programming at the authoring end. Generally these incorporate workflow capabilities, which allow moderators to have final approval over content, and as such need to be altered to fit the structure and requirements of each organisation. A CMS, by and large, requires a fair amount of programming to get off the ground. For example, in Microsoft Content Management Server (now incorporated into Sharepoint Server), each site is a compiled application, and each page template is a .net file that includes a fair amount of underlying Visual Basic or C# code.

Examples: Microsoft Content Management Server, Plone, Socialtext

Social Networking Systems represent the connections and interactions between people. They allow users to establish links to other users, which might represent existing organisational or social connections, but might just represent an interest in following content they produce. Commonly, blogging and photo-sharing tools are supplied. Usually, these are hosted by a third party, are impossible to modify programmatically to fit an organisation's structure or feature requirements, and act as a bubble away from the rest of the web. Because they are free and advertising-based, their operators rarely allow user data to be exported. However, there are some social networking scripts that can be downloaded and installed on a server; these generally allow some limited customisation, for example visually, but usually have a fixed set of features.

Hosted examples: Facebook, MySpace, Bebo, Ning (and about a million others)

Installable examples: PHPizabi, People Aggregator

Our own Elgg is also capable of acting as a simple installable social network, if you like.

Social Publishing Platforms are a combination of the above. They might have a centrally hosted version, but they are largely made up of installable scripts. These allow you to customise the workflow and functionality to fit your organisation's requirements, although generally they also come with a standard configuration that doesn't require any initial programming. Search is usually via a bottom-up ontology like tagging, and the emphasis is on informal content from a personal point of view. Often, functional changes come via plugins, which in some ways is an efficient way of expanding a system's possibilities, but also means that modifications are quite generic in nature (as a plugin needs to work for everyone who downloads it, regardless of context). Further modifications can be difficult.

Examples: Drupal, WordPress

Again, Elgg is capable of running as a social publishing platform. Its workflows can be modified and extended to an infinite level of complexity, and links between users can be established to allow for social content tracking. It also

Social Application Engines allow an organisation to determine their requirements first, and then fit the software to these rather than the other way around. Although plugins are provided and are feature-packed and flexible enough to work great as-is, they can also be used as starting points. Each individual internal piece of code is considered to be a feature, and built and documented accordingly; these can then be combined in flexible ways to create entirely new pieces of functionality or modify existing ones. All the while, the very core of the software manages data storage, the logical entities that make up the system (users, objects, relationships and so forth), access permissions and session management in a standard way. Although the application has been built in a bespoke way to fit a particular set of needs, because the core keeps everything managed in a standard way, aspects like full import/export, APIs, mobile applications, RSS, FOAF and semantic web formats all come for free.

Examples: Elgg

One of the things I'm both really proud of and excited about is how Elgg 1.0 straddles these categories. On full release there will be downloadable packages that work perfectly out of the box with no programming, and modifying these visually and with plugins is easy; however, it is also simple to either change the underlying infrastructure with these, or to build your own socially-aware application with Elgg as its core. Either way, it acts as a fundamental part of the social application stack, much like a database, while allowing you to create something that fits your needs in as technical or high-level a way as you like. I genuinely think it will change the way in which both web applications are built, and web platforms are produced.

Last year the University of Brighton organised the ElggJam, which was a roaring success. This year we're doing it all again; this time we'll be at the very cool Roxy Bar and Screen in central London on June 18th for discussion, beers and various talks. Speakers include myself and Dave Tosh, as well as Stan Stanier from the University of Brighton, Alberto Nardelli from UnLtdWorld and RuckU's Will Carling (who should really need no introduction).

Attendance is free, and it turns out there is such a thing as a free lunch; all you need to do is register at the Elgg Jam site. Places are limited, and given out on a first come, first served basis. There are also several sponsorship opportunities; if you're interested, get in touch!

Introducing the Triffid Book! Next month Packt Publishing release Elgg Social Networking, by Mayank Sharma. This is kind of a milestone: the first book written about the Elgg social networking platform. It's also a useful how-to guide for the classic Elgg codebase, which will probably come in handy for newcomers to the scene. Pre-orders get 20% off the full retail price.

The book deals with the codebase architecture that was introduced in 0.6 and persists through 0.9x, and you'll be pleased to hear that there will be a simple upgrade path from there to 1.0. More details will be released closer to the time.

Very competitive salaries offered. Can you code in PHP? Can you point to an app you've written? Let me know. My email address is ben at curverider dot co dot uk.

Skype. Chances are, if I'm on the phone, I'm talking on Skype. It's invaluable for talking to people overseas, even if they don't have their own accounts, because of the cost; it's an excellent IM and videoconferencing platform across Windows, Mac and Linux; it's even useful for recording the odd podcast. With services like the 3 Skypephone coming out, it seems to have cemented itself as the VOIP provider of choice (even if eBay did pay too much for it). Links: Skype, Skype 2 for Linux beta.

Eclipse PDT and Zend Studio, with XAMPP. I use both for different tasks, but I'm particularly impressed with their SVN integration, code completion and how much more efficient they make me as a PHP programmer. Okay, so Eclipse refused to work on my Dell, punily specified as it was with only a gigabyte of RAM, but my current machine (a Philips Freevents with an ample 2Gb) takes it well. XAMPP, meanwhile, is an all-in-one web server in a box for various operating systems, incorporating Apache, PHP, Perl, MySQL and more, which makes it really easy to maintain a fully featured local testing platform. Links: The Eclipse PDT Project, Zend Studio, XAMPP.

Paint.net. Although purists love it, I've traditionally found GIMP to be a beast to use - something that's unfortunately common in applications with Linux-based origins. Don't get me wrong, it's feature packed; mostly my gripes are to do with the way the interface works. Lots of separate windows float around with no dock, which tend to open in random places on my multi-monitor setup. On the other hand, Paint.net is incredibly functional, works well, and is similarly free. Nothing beats Photoshop, but this is a close second for my purposes. Link: Paint.net.

ElggVoices. It sounds like company spiel, but we've been using this religiously as we've developed it, and it's actually improved company communication in our distributed team by a factor of 10. I'm loving it, but there's little else I can say just now.

The Blackberry Curve. Blackberries have a bit of a reputation. In fact, I informed my girlfriend sometime last year that if I ever became the proud owner of one, she could kick me in the face. The thing is, though, it's incredibly well designed; it's the most responsive and feature-packed smartphone I've ever had. The QWERTY keyboard is invaluable for a whole bunch of tasks, from simple SMS messaging to logging into servers and writing full emails. It's got GPS, and a camera, and all the usual stuff, but its connectivity functions are akin to a palm-sized terminal, and the navigational nubbin is both satisfying to use and ergonomic. The one fly in the ointment is the web browser, which sucks in an almighty way, but honestly? I'll live. The ability to properly stay in touch wherever I am means that I have more freedom to be anywhere.

... And one I need to find:

Reliable, secure, cheap offsite backup. Last week my home was broken into, and all my computers were stolen. Aside from the fact that we were in the house at the time, and that it was hugely terrifying, the most annoying aspect was that a bunch of my personal files are gone forever. (Company stuff is backed up religiously.) I do keep some personal backups, but I'm not very good at it, so for example, all my photos that I haven't uploaded to Flickr from about August onwards are missing. Omnidrive looks like it might give me what I want, but are there other, perhaps better options? And is there something that will allow me to back up my Windows documents folder automatically every day, in an encrypted, secure way?