by thaterrormessage » Sun Nov 28, 2021 1:04 pm
Some pages with GET parameters in the URL duplicate the contents of other pages (without GET parameters). For example,
https://example.com/tovary?from=mainpage duplicates
https://example.com/tovary. Because both pages are crawled, it might take longer for information about important pages to be added to the search database. This may affect the site's search status.
See examples. If the search has duplicates due to GET parameters, we recommend using the Clean-param directive in robots.txt so that the robot ignores insignificant GET parameters and combines signals from identical pages on the main page. When the robot learns about the changes made, pages with insignificant GET parameters will disappear from the search.
[quote]Some pages with GET parameters in the URL duplicate the contents of other pages (without GET parameters). For example, https://example.com/tovary?from=mainpage duplicates https://example.com/tovary. Because both pages are crawled, it might take longer for information about important pages to be added to the search database. This may affect the site's search status.
See examples. If the search has duplicates due to GET parameters, we recommend using the Clean-param directive in robots.txt so that the robot ignores insignificant GET parameters and combines signals from identical pages on the main page. When the robot learns about the changes made, pages with insignificant GET parameters will disappear from the search.[/quote]