Community Information
-
robots.txt, home pages and canonical URLs
My CMS uses a non-standard default page filename: index.dot and I think that is why SEO checkers see this as two pages with the same content and they suggest I determine which page is the canonical URL (times hundreds of folders), but they're literally the same page: /foldername/index.dot and /foldername/ So, can/should I add something like this to my robots.txt: Disallow: /\*index.dot so the index.dot URLs are ignored but the /foldername/ urls are crawled/indexed? Is that the right syntax for robots.txt to disallow all index.dot files? Is there another way to tell search engines index.dot is the default page not a duplicate?2
© 2025 Indiareply.com. All rights reserved.