PlugPress
Howdy, visitor Get PlugPress One

Google Is Indexing Your .md Pages: Why It Happens and How to Fix It

What to do when Google Search Console shows your pages indexed twice — once as the normal page, once as its .md Markdown version.

Google indexes both /my-post/ and /my-post.md

Symptom. Search Console’s Pages report (or a site:example.com search) shows the same content indexed at two URLs: the normal page and a plain-text .md variant. Sometimes the .md version even appears in search results, showing raw Markdown to visitors.

Why this happens. Waggle publishes a Markdown version of every page (append .md to any URL) so AI assistants can read your content cleanly. In Waggle versions up to 1.0.0, those .md URLs were also listed in your XML sitemap, and the .md response relied on a Link: rel="canonical" header alone to tell search engines “the real page is the HTML one.” Google treats a canonical header as a hint, not a rule — and treats a sitemap listing as a vote for indexing. With the two versions’ content differing (Markdown vs. HTML), Google sided with the sitemap and indexed both.

How to confirm.

  1. In Search Console, open URL Inspection and inspect any .md URL — if it says “URL is on Google,” you’re affected.
  2. Or search Google for site:example.com inurl:.md and look for plain-text results.
  3. Check what your site currently serves: 
    curl -sI https://example.com/some-post.md | grep -i 'x-robots-tag|link'

    On a fixed site you’ll see both X-Robots-Tag: noindex and the Link: <…>; rel="canonical" header. If X-Robots-Tag is missing, you’re on the affected version.

Fix.

  1. Update Waggle past 1.0.0. The update does two things at once: .md URLs leave your sitemap (it goes back to normal pages only), and every .md response sends X-Robots-Tag: noindex. AI assistants still find and read the Markdown — it stays listed in your llms.txt, and every page keeps its <link rel="alternate"> pointer — but search engines are now explicitly told to drop it.
  2. Let Google clean itself up. As Google recrawls each .md URL it sees the noindex and removes it from the index. This takes days to a few weeks depending on crawl rate. In Search Console the Pages report’s “Excluded by ‘noindex’ tag” count will grow as the duplicates drain out — that’s the fix working.
  3. For anything embarrassing right now, use Search Console → Removals → New request with the .md URL (or the .md prefix) to hide it from results immediately while the recrawl catches up.
  4. Do not block .md in robots.txt. It feels like the stronger move, but a blocked URL can stay in the index (“indexed, though blocked by robots.txt”) because Google can no longer see the noindex. Let the noindex do its job — and blocking would also cut off the AI crawlers the Markdown exists for.