Resource Hub

How to Find Out if Your Website Can Be Crawled

This type of sitemap helps search engines know what’s on your website.

Does your website have a sitemap?

Follow these steps to find out:

  1. Open up any website browser
  2. Enter this link: https://www.websitedomain.com/robots.txt (Remember to change this to your own website domain!)
  3. Copy and paste the sitemap link into the same browser window.
  4. You should see a list of links on a page titled, “XML Sitemap”.
  5. If you don’t, something might be wrong!

More about sitemaps and two other key phases of the website design process.