When I compared crawl data for a large ecommerce site, one matched group of pages stood out. Its 708,000 clean URLs, with no parameters, received about 1.1 million Googlebot requests over 30 days. Their parameter copies received about 14.5 million.
The copies received roughly 13 times as many requests as the clean URLs.
Those copies already had canonical tags pointing to the clean versions. That setting seemed to tell the search engine, "This is the same content." Yet requests continued to land on the different parameter URLs. Why?
I'd start by tracing those URLs back to whatever was generating them. A change to a crawl rule might bring the request count down quickly. But if links keep producing new versions every day, part of the problem remains.
A canonical tag identifies the preferred version
The same product can have a regular link as well as links carrying an advertising source, a campaign marker or affiliate tracking information. If those URLs show the same product content, a canonical tag tells Google which URL the site would prefer to represent them.
It doesn't immediately turn the other URLs into addresses Google must stop visiting. Google needs access to read a page's canonical tag, and it makes its own choice of canonical URL. The presence of the tag isn't enough for me to conclude that Google has adopted it. It isn't a crawl restriction, either. Google's canonicalization guidance describes the tag as a strong signal, rather than a guaranteed outcome.
So the 13:1 ratio alone wouldn't convince me that the canonical setup had failed. It establishes something narrower: these copies were receiving many requests despite the canonical signals. To explain why, I'd need to check whether the links kept appearing, whether the pages were actually equivalent, and which version Google actually selected.
Another number makes a quick verdict tempting. Over the same 30-day period, about 5,000 search clicks were attributed to parameter URLs, or 0.16% of roughly 3.1 million clicks. Put that beside the crawl activity and it is easy to read it as "all those requests for almost no return."
I wouldn't make that calculation. Under Search Console's attribution rules, clicks and impressions for different versions are generally assigned to Google's selected canonical URL, with some exceptions. A small click count against the parameter URLs could partly reflect that attribution. It also tells me nothing about visits or orders from advertising and affiliate links.
There is still a reason to investigate repeated crawling. I'd want to know whether the same content keeps acquiring new addresses and whether updates to important products are being missed. Dividing figures measured on different bases wouldn't establish that the links have no value.
I'd follow the URLs back to their source
One source stood out when I compared parameter groups across the site: Shopping-related links accounted for about 62.7% of Googlebot requests. The denominator here is all requests across the site, not the matched group above or just requests to parameter URLs.
That share would lead me to inspect the link templates in the product data source first. Some of the related links contained tracking placeholders that hadn't been replaced with actual values. A separate group of affiliate links used different tracking markers to create many versions of the homepage URL. Cleaning up addresses one by one would be hard to keep up with. I'd want to inspect the rules producing them.
I'd follow one product from the data source to its landing page: the URL submitted in the product data, anything a campaign link adds, the destination after redirects, and whether the page itself keeps emitting parameter links. That would help me distinguish a change to the product from a change to its tracking information.
The distinction affects the fix. If a parameter selects a model, a color or a market-specific price, removing it could show the visitor a different product state. If it only records the source, I'd still need to check that removing or redirecting it preserves ad attribution and affiliate payment tracking. I want to reduce unnecessary URL variation without losing the function the link was meant to perform.
I'd check robots.txt too, but the dates matter. The configuration checked on October 8 included specific allowances for some tracking parameters alongside a broader rule disallowing parameter URLs. Under Google's rule-matching behavior, a more specific matching rule can apply. Seeing a Disallow directive doesn't establish that all parameter URLs are blocked.
The crawl data, though, covers August and September. The configuration was checked in October. Before claiming that these rules caused the earlier requests, I'd need the version in effect at the time. The October configuration gives me something to investigate now; it cannot explain the past on its own.
I'd test the change on one group of links first
If I were organizing this repair, I'd select a group of Shopping-related URLs confirmed to change only tracking information, not product content. I'd follow one product through the full visit and attribution process before extending the change to similar templates.
New links need to become stable first. Unchanged product content shouldn't keep acquiring new crawlable addresses because of template errors, unfilled placeholders or unnecessary random markers. Where tracking information is required, I'd work with the people responsible for advertising, affiliates and product data to decide where it belongs. SEO shouldn't simply clear the parameters. Merchant Center's landing-page and link-field requirements would need checking too.
For existing copies, the next question is whether they still need to remain accessible. If an address is being retired and the content is equivalent, I'd consider a direct redirect to the corresponding clean URL. For copies that need to remain accessible, I'd check that canonical tags, internal links and sitemaps point consistently to the intended version. If a copy is indexed and Google has selected the wrong canonical, Google needs an opportunity to read the consolidation signals. Blocking access before that happens could prevent it from reading the canonical tag I want it to process.
Once those relationships are clear, I'd decide which URLs Googlebot has no ongoing reason to crawl and whether robots.txt restrictions are appropriate. Google explicitly cautions that robots.txt is not a canonicalization tool. A blocked URL can still be indexed without its page content.
Before extending the change to the next group, I'd verify that visitors reach the correct product, price and availability, that attribution is retained, and that the necessary crawlers can still access the pages. Access needed for ad and product checks would also need verification.
I'd look for important product updates being fetched promptly
Once parameter requests fall, a downward-sloping chart can make the work look finished. I'd keep looking at the clean pages.
During the same log window, about 298,000 clean pages had no observed Googlebot requests, even though their parameter copies were requested. That contrast deserves investigation. But no request to a clean URL in 30 days doesn't mean the page was unindexed or its product information was out of date. Less frequent crawling may be perfectly reasonable for a product that hasn't changed.
I'd select a fixed group of products still on sale whose prices or availability had changed. I'd record when their content changed, then check when Googlebot fetched the updated content and whether Google's selected canonical was correct. Where appropriate, I'd examine the clean URL alongside copies confirmed to be equivalent, so that content fetched through another address wasn't overlooked.
That separates two questions: have repeated requests declined, and is the content that needs updating being fetched sooner? Improvement in the first doesn't establish improvement in the second.
Google's crawl budget guidance also cautions that restricting a group of URLs doesn't guarantee that the freed capacity will go to other pages. Whether the site is already hitting its crawl capacity limit affects that transfer. I wouldn't treat the roughly 14.5 million requests to copies as a fixed allowance I could hand directly to clean product URLs.
If link generation had stabilized and parameter requests had declined, but important product updates still weren't being fetched promptly, I'd investigate how those pages are discovered, how the server responds, and how changes to those pages are signaled to Google. Extending the blocks further might not address the problem.
After the repair, I'd want to be able to explain which duplicate addresses were no longer being created, how existing copies had been handled, and whether important product updates were being fetched in time. Any subsequent changes in impressions, clicks and orders would need to be observed separately. A smaller request count is only one result of the work.