Kotlin-Posts with sitemap and robot.txt
Kotlin expert Antonio Leiva had written many posts, so interesting as difficult to find them, no easy index at all. First thought: look into sitemap.xml file, but ouch! it was missing. Then I remembered that sometimes the robots.txt file show some cool data.
And there it was the sitemap.xml with a little tweak in its name:
https://antonioleiva.com/sitemap_index.xml
Inside, bingo! all posts ordered by date.
https://antonioleiva.com/post-sitemap.xml
"The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned."
https://en.wikipedia.org/wiki/Robots_exclusion_standard
"The Sitemaps protocol allows a webmaster to inform search engines about URLs on a website that are available for crawling. A Sitemap is an XML file that lists the URLs for a site."
https://en.wikipedia.org/wiki/Sitemaps
Inside, bingo! all posts ordered by date.
https://antonioleiva.com/post-sitemap.xml
"The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned."
https://en.wikipedia.org/wiki/Robots_exclusion_standard
"The Sitemaps protocol allows a webmaster to inform search engines about URLs on a website that are available for crawling. A Sitemap is an XML file that lists the URLs for a site."
https://en.wikipedia.org/wiki/Sitemaps
