Lets Talk Topical Clusters
May 5, 2024How Google Crawls, Indexes & Ranks Websites – Simply
How Google crawls, indexes, and ranks websites using a library analogy
I have tried to explain this so many times, but always found it difficult to put it in layman's terms, so put some time into making it as simple as I possibly can. As I'm a big reader, I went down the library route.
Imagine Google is a massive library with billions of books (websites).Google sends out little robots called "crawlers" to find new books, either in the library (existing websites) or outside of it (new websites).
Crawling is like these funny little robots going around the library, looking at the books on the shelves, and making a list of all the books they find. Books tables of contents and chapters, websites have links that the crawlers follow to discover new pages. Same principle.
Once the crawlers bring back this big list of books, the library (Google) needs to index them. Indexing is like creating a super-massive catalogue of all the books in the library. This catalogue lists every single word, in every single book and which books every word appears in. It's like having a hyper detailed table of contents for the entire library!
Now, when someone comes to the library and asks for a specific topic, like "fresh water fishing," the librarians (Google's ranking system) can quickly look through the index to find all the books that mention fresh water fishing. But there might be thousands of books on fresh water fishing, so how do they decide which ones to recommend first?
This is where ranking comes in. The librarians look at various factors to determine which books are likely to be the best match for what the person is looking for. They consider things like:
- How many times the word "fresh water fishing" is mentioned in the book (this is the relevance)
- How popular and well-regarded the book is (this is the authority and quality)
- How up-to-date and well-maintained the book is (this is the freshness of content)
- How easy the book is to read and understand (this is the user experience)
Based on these factors (and a few others), the librarian ranks the books from most relevant and useful, all the way down to least relevant. Then, they present the top-ranked books to the person who asked about fresh water fishing.
Google's crawlers find new websites like librarians find new books, the index catalogues all the content like a library's database, and the ranking system orders the most relevant and useful websites to show for each search, just like a librarian recommending the best books on a topic.
And breathe.
By the way, I don't fish.
