The Danish Joomla spider

Mission: To examine how many joomla sites there are in Denmark

Many times i asked myself – how many sites actually runs joomla in e.g. Denmark and i’ve could’t find the answer anywhere. Even extensive search on Google, posting in forums etc.

So i decided to find out myself.

Currently lacking danish doimains in the DB
Please drop a comment, if you know how to get a 
complete list of domains in the danish TLD.

The Plan: Build a spider

My idea was to build a spider, that could parse HTML files for evidence of Joomla, and then increment in a DB.

What did I do

The Spider loads the index page of a domain (with the DK TLD), searches for “joomla” in the “Generator” meta tag. (Simple, is’t it 🙂 )

Next I wanted the domain database to grow, so a added a domain grapper, thats takes all the links on the index page, checks for duplicates in the DB, and if new – adds to the spiders TO-DO list.

Conclusions

For now, I can conclude that under 1 percent is running Joomla, but I am only approx.  25-30% through the 1 million danish registered domains.

I’ve added a small “widget” here under, that refreshes on every load the spiders progress.