Once crawling These recently located URLs, they assess Each individual URL's HTTP reaction code. If the URL returns 404, bots don't include the specified URL to their index and in some cases clear away that URL from their index if it absolutely was now indexed before. The sitemaps Show mo