A small update of Cuzimatter, our social bookmark utility, went online today. The improvements are,

• Polished user interface.
• Support for Yahoo! My Web 2.0 bookmarks.

We know there are a few utilities like this one out there, but we still think Cuzimatter is among the easiest to use. There are no plugins to install, nothing to download. Just click and go.

Recently, Wikipedia announced that they would add the nofollow tag to all outbound links from its site. Ostensibly this was to take away the incentive to use Wikipedia for spam, e.g. where shady companies would post unrelated links to themselves on Wikipedia pages. But as a side effect, this will greatly increase the page rank of Wikipedia, at the cost of every other site on the internet. In fact, Wikipedia has stumbled upon an amazing tool for spam and Search Engine Optimization. Read on to find out how it works.

Most websites these days are very concerned with being prominently visible in search engines. For Google, the number one measure of a page’s importance is called ‘PageRank’, and thus this is what everyone wants to have more of. Presumably both Yahoo! and MSN Live Search use similar ranking techniques.

PageRank (abbreviated PR) is loosely based on the number of incoming links to a webpage. More precisely it depends on how many pages point to a particular page, and what PR those linking pages have in turn. The best known and most detailed explanation of this seemingly circular definition might be Ian Roger’s “The Google Pagerank Algorithm and How It Works”.

At the end of Mr. Roger’s article the following advice is presented,

If you give outbound links to other sites then your site’s average PR will decrease (you’re not keeping your vote “in house” as it were). Again the details of the decrease will depend on the details of the linking.

So if you want to have a high page rank, part of your strategy may be to not link out. This is easier said than done though. A website that could get away with no outbound links whatsoever, and still be interesting, would be a rare website indeed. The very idea behind the WWW, World Wide Web, is that it is like a web of links. Without links, a website wouldn’t be more interesting than an ordinary printed sheet of paper. So you have to link, and losing your hard earned PageRank thus seems to be an inevitable consequence of making a normal website.

The challenge a website owner is faced with is this:

1. You have a website and you want it to be popular.
2. You want to have a higher PageRank because then people will find your site.
3. Using outbound links reduces your PageRank.
4. You must have outbound links or your site will be rather boring.

So what to do? Simple. Link a lot to yourself and not a lot anywhere else. To illustrate how this is normally done, I will briefly describe two traditional techniques below.

Most blogs allow users who make comments to add a link to their own website or blog. Since this leads to many outgoing links, a comment section can be described as a ‘PR drain.’ The technology blog Engadget works around this by not letting the user link their name to a website when commenting. While I won’t say that Engadget uses this technique intentionally, this is an example of something that helps to keep page rank bottled up inside of a site. Engadget and its sister blogs also employ enormous link lists at the bottom of every page so that even if there are a few outbound links, they are dwarfed by the large number of links pointing back at Engadget itself or to other sites within the network.

Some sites, like Ars Technica, isolate their outbound links by putting the link intensive comments section for each article on a separate ‘discussion’ page. Notice that how at the bottom of this article at Ars, there is only one single ‘discuss’ link followed by internal links to other Ars Technica articles. Most other technology news sites would have a live section of comments in this area, bur Ars avoids this PR drain gracefully with their ‘discuss’ link. The effect is that whenever an Ars Technica news entry gets a higher PageRank because of people linking to it, almost all of that rank stays within the site. (Perhaps this is unintentional; Ars has a forum which they promote by putting their discussions there.)

All of these methods have two things in common. They’re obvious at a glance, and they do at the end of the day pass at least some PageRank on to other sites.

The ‘nofollow’ method is much less obvious. Nofollow is a ‘tag’ you can add to a link so that search engines won’t take note of it. It is invisible to the user and does not affect their experience in any way. The nofollow tag was brought to life by Google, who back in 2005 announced that they would disregard nofollow links. The announcement can still be found on their official blog. The reason Google introduced this policy was to give webmasters a tool to discourage spamming with. If all user entered links had the nofollow tag added to them, the links would be less useful to spammers. Even if a spammer put hundreds of links to their site in some blog’s comments, the site wouldn’t become any better ranked. All of the links would get the nofollow tag and the search engines would disregard them.

Because of their invisible property, nofollow is the ultimate page rank retaining technique. If a site went ahead and put nofollow on every single external link on the site, it would become a site from which no PageRank would ever ‘leave’. Every incoming link would add rank to that site, and the site itself would never add rank to any other site. Previously the only way to achieve this effect would be to simply not have any outgoing links, and the site would suffer from it. With nofollow you get the best of two worlds. You can link like there’s no tomorrow, and make your users happy, while at the same time you can tell search engines that you couldn’t care less about the sites you link to.

And this is exactly what Wikipedia has done. As most people using search engines are aware, Wikipedia is often at the top of the search results for almost any relevant query. People like to link to Wikipedia. We have done so ourselves here at Playing With Wire from time to time. And now that Wikipedia has gone into nofollow mode, it will never ever let go of the rank you give it by linking to it.

As other bloggers have pointed out, some quite angrily, this will have widespread repercussions. Wikipedia becomes a black hole of PageRank. Search engines are affected negatively. If a majority of the sites on the internet started to use nofollow, then what would the search engines have to work with when determining the most popular site?

Wikipedia claims they made the change to reduce spam, and I believe them: this might have been their intention. But at the end of the day Wikipedia has greatly increased its own PageRank at the cost of the rest of the internet. And in doing so, Wikipedia has shown the dark side of nofollow. Even as I write this I am sure there are greedy site owners combing their whole sites and adding nofollow tags to every external link, following Wikipedia’s example. Wikipedia has effectively demonstrated the ultimate PageRank retaining technique. Indeed, Wikipedia has perhaps taken the first step towards a future internet where no-one links to anyone in a search engine compatible way, just in order to hoard the precious currency of the internet: PageRank.

What do you think? Is Wikipedia’s new policy an honest spam reduction effort or a masterful Seach Engine Optimization move? Will every site on the internet soon be using nofollow? Will Google have to retract their nofollow policy to save their search system from breaking down?

             

A little announcement just dropped into my mailbox:
Apple’s cute iPod shuffle now comes in multiple colors. Five different colors in fact: standard grey, pink, green, blue and orange.

You have to hand it to them: Apple knows how to do business. There is probably little doubt that they could have made these colorful little things from the start. The new Shuffle has been on sale for a while, but as opposed to the Nano, it only came in brushed metal grey. Then Apple happily sold that for a while, especially during the Christmas rush.

Then as soon as the novelty of the positively tiny shuffle was past, they made a new announcement. Brilliant colors!

Here’s the page on Apple’s site.

         

You may have seen our farewell letter to blogger. And yes, it is true. We have switched from a Blogger system to a WordPress system.

The transition was made during the late hours of Saturday night. We hope we have caused as few disturbances as possible – we have even relinked every old article to its new permanent address (by hand none the less!). Still, let us know if there is anything that is broken or doesn’t seem to be working like it should. Your feedback is greatly appreciated.

So why did we make the switch? Well, as the first post hinted, we have had trouble with Blogger’s stability. But this was not the main reason we switched. The main reason was that even that we were hosting our own published version of our Blogger blog, there were a few links that were part of every page that went back to Blogger.com material. And despite Google’s legendary connection speeds, Blogger.com did not seem to get any of that. Time and time again we saw situations where the front page was not loading for several seconds as the web browser was waiting for a Blogger.com file. And whatever they were doing, they didn’t even seem to have cache control enabled. So these files would be fetched over and over again, possibly with a multiple second delay.

We don’t think WordPress is faster than Blogger. In fact, I’m very certain that WordPress is slower by an order of magnitude. Blogger generated static pages for all content. Every time a comment was posted, the relevant post’s static page file was updated. Every time a new post was made, old pages were regenerated with the relevant links to the new page. This of course is optimal. You can hardly make a web server any faster than it is when serving static pages, especially not with the right Apache configuration.

So in theory Blogger was extremely fast, and we will be taking a performance hit by switching to dynamic pages with WordPress. But in practice, Blogger was often very slow due to those few non cacheable header links. It shouldn’t take seconds to load a single page, especially not if all images are already cached.

So we didn’t have a choice. Blogger was pushing us to upgrade to the new version of Blogger, and if we did that we wouldn’t be able to use WordPress’s built in Blogger import feature, which we ultimately used to get all old posts and comments over to the new system.

That said, the switch wasn’t entirely because of Blogger’s drawbacks. There are some very nice things with WordPress. For example, you can create pages like our About page. This is a great touch of CMS functionality that saves us from the trouble of theming random pages by hand.

Next we will look into generating a new sitemap and page caching. We hope that your Playing With Wire experience is faster already though.

Please let us know if you find anything that doesn’t seem to work as expected.

Update 1: We did try to contact Blogger about the performance issues two months ago, but we never heard back from them.

Over the last year I have run a server using the Linux flavor Gentoo. While most of the servers I deal with these days run FreeBSD (WireLoad’s servers included), I was curious about what speed I could get out of old hardware with Gentoo. Gentoo is a system built to make it easy to compile every little piece of software in the system with the biggest and baddest gcc flags imaginable for your particular hardware. In theory this should lead to a faster system.

The experience has been a bit of a mixed bag. There are things I really like about Gentoo: the package management, USE flags and the sophisticated dependencies system. But unfortunately the drawbacks are severe for a server setting.

The Good

The system is better than most Linux systems I have seen when it comes to general package management and installation. The emerge command is excellent. It makes it easy to update and install software together with the necessary dependencies. A well thought out system called USE flags complement this system well. Rather than having to tinker around with every program you compile, you can set some global USE flags. Most packages will take note and your system will be made into one harmonious whole of software agreeing with each other about what should be used and what should not.

The easiest way to describe the benefits of this is by comparison to a normal FreeBSD server and the installation process. Assume that you don’t have X11 and that you don’t want it. Now, you are installing PHP and you want to have support for graphical operations (image conversion, CAPTCHA generation, etc). So you find the make flags to do this and you add them to your make.conf. You hit ‘make install’. During the build process the ports system goes out to build the dependencies of your graphics library. The graphics library requires fonts. The fonts want a font manager. The font manager is by default configured for X11. Wait, X11? Suddenly, X11 becomes a dependency and you find yourself hitting Ctrl-C rapidly!

In Gentoo on the other hand you would have -x11 in your USE flags and everything would be cool. It’s a slick system.

Colorful Gentoo.

Gentoo also has an active community. As we have mentioned in this blog before, there is an excellent forum for Gentoo users. There are also little touches like how Gentoo uses colors by default, to improve clarity and to go easier on the eyes.

Disadvantages Expressed in Time

All of these things make it a pleasure to use Gentoo. I would love to use it for a desktop some day, should my Apple OS X machine fail me. But I really don’t see myself using Gentoo in a server setup again. Here’s why:

1. Gentoo is Time Consuming to Install
At least when I installed Gentoo there wasn’t really any installer. The documentation is excellent and describes exactly what you need to do, but it takes a while. In fact, it took me several hours to set up my first Gentoo system. And that was just the beginning.

2. Gentoo is Even More Time Consuming to Install
The strength of Gentoo is the compile everything mentality – at least that seems to be the main selling point. Unfortunately on my low-end test server it took about three days to compile even the base system with Apache, MySQL, Python and some other important software. My machine was working non stop compiling things during this time.

I understand that the latest recommendation is to not perform a so called ‘stage-1′ installation anymore. I would recommend following this suggestion. But then what happens to the compile everything advantage?

3. Gentoo’s Stability Strategy: Update Everything
Since it takes a long time to compile a program, you usually don’t want to have to do it too often. Unfortunately Gentoo encourages you to update software on a frequent basis, just for the sake of updating.

There is no ‘stable’ version of Gentoo. Gentoo is rather a moving target where emerge will forever cause your system to approach the cutting edge. From the Gentoo handbook:

From the beginning, Gentoo was designed around the concept of fast, incremental updates.

If all you’re concerned with is keeping your web server up, what you usually want to do is to set up a stable system and then forget about it. You install security updates as needed but that’s it. With Gentoo, this isn’t really feasible because there is no ‘stable’ Gentoo release.

What’s worse, there will on occasion be a sort of ‘system update’. This is called a new ‘profile’. The Gentoo documentation and the handbook will at this time encourage you to update to this new profile. A profile update will try to replace your basic system. If you are a system administrator, rather than a desktop user, this should be enough to scare the living daylights out of you!

A profile update will touch a very large number of configuration files, and it may even alter your startup process. Obviously this is not something you want to do to any server. It would be very difficult to verify that everything works as it used to afterwards, and you’d be fairly likely to end up with broken configuration files that may stop working the next time you reboot. This is in fact exactly what happened to me, despite a substantial time spent updating /etc files. The end result: the machine had to be resuscitated on-site with associated downtime.

For a more sensitive server than my test system, you’ll want to simply retire the system whenever a new profile comes out. Just start over fresh with a new Gentoo installation on an alternate machine and go through the setup process. This way you can be fairly sure it’ll work even after a reboot. Once you’ve verified that everything works, switch to the new system.

4. Gentoo’s Security Strategy: Update Everything
As you might be aware, FreeBSD has a nice little program called portaudit. This utility will alert you if you have any software installed with known security holes. Then you can go ahead and update that software with a simple ‘portupgrade‘ command. There’s rarely any problem with this process.

Now, Gentoo also has something like portupgrade. What it doesn’t have is portaudit.

In all fairness, Gentoo has an experimental command called ‘glsa-check’. This command automatically examines whether your system is affected by vulnerabilities described in Gentoo issued security advisories. It also knows what steps need to be taken to fix a given security issue. I really like this development, but I understand that this command is not considered production ready. The Gentoo manual page about it is filled with warnings that this is a tool under development.

In the meantime, Gentoo rather encourages you to update the whole system. And of course a system wide update tends to cause just the amount of havoc you would expect from it.

Gentoo too Time Consuming and too Risky for Servers

I firmly believe in updating server software only when you need to. If you don’t need new features, and things are working, why change anything? If you update anything you will doubtlessly need to update configuration files. You will need to fix things that break in the upgrade process. This is exactly what happened to me with Gentoo during its test year. I had nearly no idea of what I was updating as I ran the dreaded but most needed “emerge world” update. And once I was done I still no idea. I spent a long time working my way through updates in the /etc folder, using the built in ‘etc-update’ command. I tried to read the enormous emerge log file and take appropriate actions. And still things broke.

The best way to keep a system stable is to get it working and then not changing anything. This is hard with Gentoo. Gentoo wants you to change a lot of stuff. It wants to be bleeding edge.

And hence my conclusion. Gentoo is fun to play with, but oh is it time consuming. I guess that’s the cost of living with a hardcore compile everything attitude – you’ll be on the bleeding edge and you better make sure you can balance on such a thin edge. For a desktop system, Gentoo seems fabulous. Fun to work with, colorful, a beautiful ports like system for software. USE flags.

But for a server, especially a production server, Gentoo just isn’t time effective. It’s both the time it takes to put in security updates, and the time possible reinstalls will take. I believe there were three profile updates for Gentoo in 2006, and very little support for older profiles. If you’re like me you’d probably much rather not reinstall your servers three times a year!

In closing I want to quote something Gentoo told me recently:

* An update to portage is available. It is _highly_ recommended
* that you update portage now, before any other packages are updated.
* Please run ‘emerge portage’ and then update ALL of your
* configuration files.

Call me stressed out but I really can’t fit too many ‘update ALL of your configuration files’ into my schedule. :)