Click here to Skip to main content
15,884,353 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
Hello all,

Introduction and code sample:
A friend of mine told me that I should update the HTACCESS of our company site to avoid giving duplicate contents to Google, his recommendation would be to update the .htAccess file adding these lines:
RewriteCond %{HTTP_HOST} ^codeproject.com [NC]
RewriteRule ^/?(.*) http://www.codeproject.com/$1 [R=permanent,L]

RewriteCond %{HTTP_HOST} ^codeproject.es [NC]
RewriteRule ^/?(.*) http://www.codeproject.com/$1 [R=permanent,L]


The idea here would be to tell the indexing spiders not to index all the domains we have as if they would have different contents redirecting the spiders to the main domain.

Of course I've tried to change the htAccess to see which would be the result, but nothing seem to be updated so I've went back to the original one...

I'm worried on the real impact of having multiple domains that seem to be pointing to the same content, if he is right, then we have a problem.

I can understand the main idea and I can even understand the code lines that I've put into the htAccess file, but how can I be sure that the hosting company has not done anything to avoid this wrong situation by themselves at the moment of buying the domains?

Note:
In one recent comment, It has been recommended me to use "robots.txt" to avoid that. As far as I know, this is not what I need as "robots.txt" is used to tell the spiders what files to crawl and what files to avoid crawling. In my case, I have multiple domains that point to the same web pages, therefore, it is like I would have different web pages with the same content and that would mean to be punished by Google as it can be understood as a black hat SEO technique. This is the reason why my friend recommended me to use that 301 redirect.

Questions:
Is there any way to know if we are being punished by Google?
Any hint?
A good explanatory link?

If I want to avoid Google punishment and keep my different domains pointing to my main domain, which would be the correct way of doing it?

As always thank you in advance. :thumbsup:
Posted
Updated 11-Mar-17 22:04pm
v4
Comments
Peter_in_2780 23-Jul-13 19:44pm    
Hi Joan,
Rewrite is almost certainly not what you need for this. All "good" spiders respect robots.txt, which is basically a map of where spiders shoud and should not index. Google "robots.txt" for more info.

Cheers,
Peter
Joan M 24-Jul-13 2:45am    
I do know about robots.txt, but I don't mean to avoid robots to crawl certain files in the domain but to avoid Google to think I'm duplicating pages as I do have multiple domains that are pointing to the same web. I'm updating a little bit my question to clarify this.
Thank you for your comment!

1 solution

That's how it worked for me at the end:

RewriteCond %{HTTP_HOST} ^codeproject.com$
RewriteRule ^/?$ "http\:\/\/www\.codeproject\.com\/" [R=301,L]


:thumbsup:
 
Share this answer
 

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900