Raw vs Encoded Bracket URLs: How Google Crawls Both

Square brackets look harmless in a URL, yet they create a quiet technical problem. A crawler can encounter the raw form, such as /products?filter[color]=red, and the percent-encoded form, /products?filter%5Bcolor%5D=red, and record them as separate addresses. If your site exposes both, you may be inviting duplicate crawling without knowing it. This guide explains why it happens and what to do about it.

Why brackets cause trouble

Under the URL standards, square brackets are reserved characters with limited legitimate uses (mainly IPv6 hosts). In paths and query strings they are normally expected to be percent-encoded: [ becomes %5B and ] becomes %5D. In practice browsers, frameworks and CMS plugins are forgiving, so both forms work and return the same page. A server that answers 200 to both has, from a crawler's viewpoint, two URLs with identical content.

Search engines do try to normalize URLs, but normalization is not guaranteed for every character in every position. The safe assumption is that the two forms can be discovered, fetched and evaluated independently.

Where the two versions come from

The split usually starts inside your own stack rather than from outside.

• Framework conventions: PHP, Rails and many JavaScript frameworks use param[key] syntax for arrays, so faceted navigation emits brackets natively.
• Mixed link generation: templates output raw brackets while a sitemap generator or API outputs encoded ones.
• Third-party links: social platforms, email tools and chat apps often encode characters when users paste a link.
• Redirect rules and canonicals: a canonical tag written with one form while the page is served on the other.
• Analytics and tag tools: campaign links copied from dashboards frequently arrive pre-encoded.

What the duplication costs you

For a small brochure site, probably little. For ecommerce, marketplaces or listing sites with filters, the effect multiplies: every filter combination can exist twice. The costs are practical.

• Wasted crawl activity on near-identical parameter pages instead of new products or fresh content.
• Split signals when internal and external links point to different forms.
• Messy reporting, with impressions and clicks divided across variants in Search Console.
• Index bloat from low-value filtered pages that were never meant to rank.

How to diagnose it

Start with evidence, not assumption.

Server logs. Filter requests from Googlebot and search for both %5B and a literal [. Count how many distinct URLs exist in each form and how often each is hit. Logs show real behaviour, which is more reliable than any crawler simulation.

Search Console. Use the Pages report and URL Inspection to see which variant Google selected as canonical. Look at the Performance report filtered with a regex for brackets or the encoded form.

Your own crawl. Run a crawler on the site and export all URLs containing either form. Compare the sitemap, internal links and canonical tags against that list. Any mismatch is a source of the split.

Spot-check responses. Request both versions and compare status codes and headers. If both return 200 with no canonical pointing to a single form, you have confirmed duplication.

How to fix it

1. Choose one canonical format

Choose the encoded form for anything that must remain bracketed, since it follows the standard and survives copying best. Better still, remove brackets entirely (see below).

2. Redirect the other

Add a server-level 301 from the non-preferred form to the preferred one. Test carefully: some servers decode before matching rules, which can cause redirect loops. Verify with a command-line request that the redirect resolves in a single hop.

3. Align every signal

• Internal links and navigation output the chosen form only.
• XML sitemaps list only the chosen form.
• Canonical tags use exactly the chosen form, character for character.
• Hreflang and structured data URLs match too.

4. Control parameter sprawl

Decide which filtered pages deserve to be indexed. Keep valuable combinations as clean, static paths, and keep the rest from being crawled with robots.txt rules or noindex where appropriate. Remember robots.txt rules match on the literal string, so write patterns for both forms if both may be requested.

5. Prefer simpler URLs going forward

Replace filter[color]=red with filter-color=red or, better, /products/red/. Fewer special characters means fewer ways for the same page to appear twice.

A quick example

Imagine a shoe store with 400 filter combinations. Logs show Googlebot requesting 400 raw-bracket URLs and 380 encoded ones, with the canonical tag using the raw form. After switching templates to one encoded format, 301-redirecting the raw version and cleaning the sitemap, the duplicate requests fall away over several weeks and Search Console shows a single variant per page. Exact timing varies, so monitor rather than expect an overnight change.

Conclusion

Bracket handling is a small detail that exposes a bigger principle: every distinct string is a distinct URL until you tell search engines otherwise. Audit logs, standardize one format, redirect the rest and keep new URLs simple. If you want a technical audit that finds this kind of hidden duplication, Growth Asteroid can map your crawl behaviour and build a fix your developers can ship quickly.

Frequently asked questions