Automatic categories & Technorati tags 1: principlesWednesday, January 18, 2006
This 2-parter describes one possible way to include categories for your blog posts automatically with Technorati tags, if you're using a blogging platform (most notoriously Blogger) which doesn't support categories. (Previously I explained how to implement a manual categories method which is quick and easy for users but involves much more ongoing work for the blogger, though CoLT makes that a lot easier.)
Clearly many of us have been thinking along similar lines lately, e.g. see my fellow Web Corante Hub contributor John Tropea's recent post. A popular semi-automatic method has been to use Del.icio.us, see e.g John of Freshblog's post, but I was too lazy to go and tag all my old posts on Delicious, so I never tried that out myself. This new method, which I've been trying out since I discovered that Technorati have introduced much more powerful tag searching, should involve much less work going forward (always a good thing in my book), and it can even pick up your old tagged posts, but it still takes some time and thought to set up.
If you're curious, you can see the new system in action on this test blog (opens in a new window). It looks much like the manual system at the right hand side of this page, but peek under the skirt (as some would say!) and it's quite different. Why am I not using it on my main blog yet? Because I want to see how it goes in terms of speed and reliability and accuracy from day to day and different times of the day, and I don't want to risk slowing down my blog for my readers or confusing them with error messages by having it there until I'm completely happy it will work smoothly.
I'll outline the basic principles behind this method with some of its pros and cons, and then provide a practical step by step guide on how to implement it on your own blog. (I'll probably split the howto out into a separate post for length reasons as there are some twists involving how Technorati seem to have changed things on their site recently, which can hide certain things you need - I'll explain how to find them).
Tagging and tag searching You need to spend time thinking carefully about what categories you're going to have and the tags for the posts that you want to be listed in each category, how to construct your search so as to pick up previous posts (if you've not been tagging with consistent keywords in the past - which means combing through your old posts, checking what tags you used for them and trying out various search combinations), and of course you need to be consistent in tagging future posts that you intend to file in a particular category, though that'll be a heck of a lot easier than figuring out how to rope in all your old posts. You won't be surprised to know I've been working on this on and off for a couple of weeks, even though I've only got blog posts going back about a year.
Also note that because this system uses Technorati tag searches, someone else's posts from another blog could get listed in your categories if they've used exactly the same tag combo you've set your search up for (that's why meblogging tags are a good thing, and this system relies on them - and also assumes others won't use your meblogging tags, not much anyway!).
Reliance on TechnoratiIf a post isn't on Technorati's tag pages, e.g. you didn't tag it with the right tag or Technorati's tag pages haven't clocked it for whatever reason (which is a common problem for many blogs), it won't get categorised automatically. (Obviously, new posts won't be listed in your categories until spidered by Technorati, which should only take hours or less these days.) So you still need to keep an eye on Technorati's tag pages periodically (e.g. do a tag search) to check your new posts have been properly picked up on there, and sort it out with Technorati support if not (or just add the missed posts manually, see below) - which is counter to the "automatic" aim, but at least that's all ongoing work you'll have to do, and it's easier than manually hardcoding in every single one of your new posts. If Technorati could only solve the issues that stop them adding all tagged posts to the right tag pages consistently without fail, and beef up their systems so that users will never again see the dreaded "too many searches try again later" message instead of the desired search results, this method would be so much easier, and indeed it would become my own personal favourite.
Furthermore, again this also means that if the site you use for the conversion is slow, or down completely, then your categories list or even your whole blog could be affected, e.g. if the converter somehow doesn't pick up the Technorati feed. One major downside of this system is that it relies on two separate services, Technorati and the feed converter both, so if even one of them is up the spout then your blog could be scuppered.
Combine with manual categories?However a major benefit is that you can combine this method with manual listings of posts, so you have a lot of control. I've done that in my test blog (new window), which still makes use of my show/hide method.
Why Technorati?Now some of you may wonder why you need to use Technorati for this - why not e.g. rival blogosphere search engine Icerocket? Icerocket have their own tag pages, called "blog topics", with tag and author combo searching and a feed for every conceivable search result.
But - they only introduced tag pages around mid-2005, so they haven't got as many of my posts on their tag pages as Technorati do, despite my (unsuccessful) attempt to get more tagged posts on Icerocket; their multiple word tag searching and AND/OR tag combo searching doesn't work as expected, from my tests; plus, on the can't beat 'em join 'em front (that's me being all dry and sarky), Icerocket have also started having problems crawling/indexing my blog (all my posts for the second half of December 2005 are missing from Icerocket for example) - and the situation is worse with Icerocket than with Technorati. With Technorati, while a number of my posts are missing from their tag pages, at least they're still there somewhere on their database, because you can find them via a simple search on Technorati if not a tag search - whereas with Icerocket, my posts just haven't been indexed on there at all; they're not there, period. So personally I'm sticking with Technorati.
If your blog is newish and you're sure all your tagged posts are on Icerocket, if you want to then feel free to use them instead (their searches do seem to be faster at the moment); but I'm only going to provide a detailed howto on Technorati in the second part of this tutorial.
So those are the principles behind this automatic categories system. In part 2 I'll outline the practical steps involved in implementing this system, and go into more detail on how to address some of the cons I described above, with some advice and tips on the different stages.