Crawl or No Crawl Report June 7, 2024- Day 1000
Milestone in Indexation Research History
Welcome to Crawl or no Crawl Report - its been 1000 days since its inception. Someone had to chronicle the history of Alexandria (what Google calls the indexation system as per the leaked Google doc last month.)
A lot of information gleaned from this research.
All our sites in search console in the Page Indexing Reports update at the same time on a general 3-day schedule. Sometimes it can take longer and it usually corresponds with some announcement of a feature within search console that might be getting an overhaul or update.
When the chrome builds in the Googlebots update, it can create SERP volatility. These mini-spikes aren’t updates in the normal sense related to sort order.
(FYI - They seem to be more a side effect of what appears to be testing the new chrome build googlebot against what was the order was with the previous chrome build and as we sometimes see various search results rollback to a previous version and then after something happens then things go back to where they were.)
How did this start?
In 2021, the javaScript rendering function was not working for almost 3 full month in those 12. But how can we know for sure? What specific research documented that time?
Surely, there had to be something to prove that portion of the indexation system was broken so that SEOs weren’t blamed (and fired) for something that wasn’t anything to do with them.
Surely, there was some 3rd party keeping things honest.
Nothing.
In August 2021, I decided, how hard could it be to test a page once a week?
If I could publish new content and then take measurements as to how long it would take to go through the system, it could help our industry and provide written record, no anecdotes, all data.
It started out once a week but quickly moved to daily.
What is being tested?
Every test page published is to test two things:
is the simple html crawl system working
is the javaScript rendering pass machine working
The answer comes from whether or not a test keyword can be found when searched.
Then rinse and repeat for another 999 days!
Thanks for coming to the party.
There are two levels - general, free account and premium accounts with $18/month support level. The premium accounts receive details of the state of indexation fresh outta search console.
This week the testing is focused on content that in search console says
Discovered - currently not indexed
The reason listed is Google Systems and while that can cover a wide ranges of reasons why a page isn’t indexed, in order to get Google back to recrawl them, they have to be resubmitted. You can tell that when you look at the URLs in this category and it says “N/A” for last crawled.
If you don’t work on them, they will stick there until something happens. But leaving them there Google isn’t going to make the first move.
June 2024 - resubmitting content via GSC
For each test since June 1st, I’m looking at the pages and making specific changes and taking notes on what I changed.
Of the six tests resubmitted this month, 2 of them made it all the way through both the html crawl and the rendering crawl.
Which ones will be shared with the premium members over this week.
Hope you’ll celebrate with me. It hasn’t been easy to keep this up but then nothing worthwhile is supposed to be.
Thank you for your support.
Carolyn Holzman