Google has additionally up to date the crawl stats report assist documentation final week. Google made a whole lot of assist documentation modifications prior to now week together with product wealthy outcomes, Googlebot doc, job posting assist doc, transferring the HTTP standing codes doc and likewise this crawl stats report assist doc.
- Observe about robots.txt 404ing (can crawl any URL)
- Utilizing a 12hr – 30 day timeframe for fetching
- Particulars about homepage utilization
He posted the doc earlier than and after modifications as nicely:
Brodie wrote, “Heads-up: Google simply added some fascinating clarifications to their Crawl Stats Report doc.” Good catch Brodie!
Discussion board dialogue at Twitter.