A brief history of search engines......

For those of you that live under rocks!- search engines are tools used to negotiate the Internet in the form of displaying relevant web pages for given search queries. The search engine is a tool that allows this to happen through a public interface. Whilst we almost take the use of a search engine such as Google or Yahoo for granted these days, the history and the development of Search Engines is fascinating, particularly the speed at which the technology for searching has advanced.

The very first search engine was developed in 1990 by a Canadian student called Alan Emtage, who created ARCHIE, a program that could search for computer files within the FTP sites across the world. However, it was GOPHER, designed by American Mark McCahill which was the first program to search for plain text documents, allowing a user to find information stored anywhere in the world on public databases of documents. This allowed the first file sharing across the world and was most prevalent amongst university and scientific communities.

As people began to see the potential in storing information on the Internet for others to access, the level of investment from governments and the private sector dramatically increased.

In 1993, students at Stanford University created a search engine called ‘Excite’ that used the relationships between words to search for items in available databases. Excite was launched in 1995 as the first commercial search engine. However it was ‘Lycos’ that was the first commercially successful search engine, although it was bought by AOL (America On Line) as their main search capability. In 1994, Yahoo was created (also by Stanford students) as a simple website that listed some of its creators favourite places to visit on the Internet. This directory model was a popular way for people to access sites of interest remained popular and Yahoo continues to use it today, although their search engine capacity has greatly increased.

Whilst directories are still available, the creation of Google, now a vast commercial venture began the rise of the powerful, relevancy crawler based search engines.(Google interestingly was developed by… yes, you guessed it; Stanford students, do you see a pattern going on here?) This was an engine that ranked its results in order of relevancy and popularity to the search carried out. The development of Googles’ PageRank system which measured the popularity of websites by the quantity and quality of inbound links to a site made their search results much more relevant. Instead of simply bringing up all pages related to the word or the set of word, it would filter and test the search against its analysis of the content and the sites linking to it. This ranking of information in order of supposed relevance lead to the commercial industry- SEO - Search Engine Optimisation, techniques that work to promote the placement of a website within the SERP’s. The idea is that the higher up the rankings, the greater amount of traffic that site will receive.

The future of search engines is likely to include engines that will make their searches using your own personal preferences. The user will give over some personal information to the engine but then it will begin to intuit your needs when it makes it search. This might take into account your age, your online searching habits, your income, your profession and other personal references that will act as markers for the engine to make intuitive ‘guesses’ about what you wish to see. At present, surfer’s habits are used often accessed via their web history. The more personal information the engine can collect, the more relevant the information will be.

Article written by Jon Dunn