Indexation and the Terrible, Horrible, No Good, Very Bad Week - Part 1
Indexation and the Terrible, Horrible, No Good, Very Bad Week - Part 1
October 29 - November 5th 2023
November 10, 2023
This spreadsheet is embedded currently on this URL - Google Index Detector Page - you can find the tab by the date - Oct 29 - Nov 5 and follow along as I break it down by day so you can understand why that week was so horrible in terms of content ranking then not ranking, content served and then content NOT served. If your head was spinning I propose this is the reason why and not your SEO skills.
I’m going to split the data and its implication into two posts - this is part 1 and part 2 will be published tomorrow.
I didn’t go into this too much on the video. Verbally trying to describe this is challenging let’s go into it here.
The sheer number of updates and changes in the crawler agents and general environment in the SERPS during the week of October 29 - November 5th 2023 was unprecedented in the past 27 months of this project.
Sunday - saw all the InspectionTool crawlers that have a chrome build were updated to the same chrome build (118.0.5993.88)
Monday - A new Mobile Crawler chrome build launched - (118.0.5993.117)
Tuesday - The Search Console Page Report data was updated in the middle of the day from October 27th to October 30th - one day later than expected (4 days since last report update) and only contained 3 days of data.
Google announces - Mobile First Indexing is Complete
Wednesday - New InspectionTool Crawlers (the non-rendering ones) updated to latest chrome build (118.0.5993.117)
Thursday - New mobile chrome crawler launched (119.0.6045.105) * note this is unusual to release a mobile before closing out the previous build with a desktop new crawler - I think its just that I saw the new one before I saw the desktop close out.
Google announces - November Core Update
Friday - “quiet” day
Saturday - New Desktop Crawler build launches (118.0.5993.117) AND the rendering InspectionTool Crawlers - there are two of them - they also updated to the same build (118.0.5993.117) -
[Significance] - since I started to note when the chrome builds changed, I started to notice that Google “tests” the NEW chrome build against OLD chrome builds.
New chrome builds only seem to do html crawls of new/revised content
Old chrome builds that previously only did the html crawls are now used for rendering crawls.
The more crawler chrome updates that occur the more reports of content disappearing from SERPS only to reappear within 24/48 hours without any changes to the on page.
It almost feels like they roll back the serps for some keyword sets to test with a new crawler and assuming there are no issues, the serps get rolled back to their original state.
Indexation Rates
Search Console Indexation Requests
Three tests -
66% Simple Keyword (html crawl)
66% Javascript Keyword (rendering crawl)
content submitted on Oct 30th (Monday) still not crawled
Google Indexing API Requests
25% Simple Keyword (html crawl)
0% Javascript Keyword (rendering crawl)
Why No Javascript Success via Indexing API
I went back through the logs and found that the only an html crawl of the page occurred. No rendering bots followed the html crawl (which was previously normal for the API requests)
Simple answer. All things being equal, the rendering crawlers aren’t being sent for requests made through the API.
Check back for tomorrow’s PART 2 - where I share the data’s take on mobile first indexing.