Welcome to the Q&A Forum

Sometimes I get alerts from Google Search Console that it has detected soft 404s on different websites, and since I take great care to never have true soft 404s, they are always false positives.

Today I got one on a website that has pages promoting some events. The language on the page for one event that has sold out says that "tickets are no longer available" which seems to have tripped up Google into thinking the page is a soft 404.

It's kind of incredible to me that in the current era we're in, with things like chatGPT that Google doesn't seem to understand natural language. But that has me thinking, are there some strategies or best practices we can use in how we write copy on the page so Google doesn't flag it as soft 404? It seems like anything that could tell a user that an item isn't available could trip it up into thinking it is a 404. In the case of my page, it's actually important information we need to tell the public that an event has sold out, but to use their interest in that event to promote other events. so I don't want the page deindexed or not to rank well!

Hi here's some more background info on this situation that makes it even stranger. I can perform some pretty specific searches on Google where these indexed search result pages show up. And I can look in Google Search Console under the performance section and see that those pages receive impressions and clicks. However, if I inspect the URL, Search Console says it is not included in Google's index, and the reason it gives under indexing is because it says it is honoring the canonical URL. So search console is saying it isn't indexed because of the canonical, but I can do searches and find that exact URL in the index. Any ideas what this could be from?

Hi Gaston,

Thanks for the response. I can confirm that the example, /search and /search?q=foo are pretty much identical. However that may not always be the case, only when a user searches for something that would return no results. So, a website that sells widgets, /search and /search?q=widgets would not be identical, and in that case it would make sense that Google would not honor the canonical link. What's really strange is if I search google for the site: operator of the domain, the top pages are not user queries for things that make sense. The top indexed pages are random, non-relevant user searches.

I do not have a way with this system to control noindex tags on these search result pages. The only thing I could do is take the nuclear option and just block it all with robots.txt using wildcards. But that means no search result pages would get indexed, relevant or not.

I have a strange situation on a website, when I do a Google query of site:example.com all the top indexed results appear to be queries that users can perform on the website. So any random term the user searches for on the website for some reason is causing the search result page to get indexed - like example.com/search/query/random-keywords

However, the search results page has a canonical tag on it that points to example.com/search, but that doesn't seem to be doing anything. Any thoughts or ideas why this could be happening?

What is the current advice for displaying content based on a user's geolocation? On the one hand, I know the rule of thumb is that you are not supposed to treat googlebot any different than any other user to your site and shouldn't show different content than what you would show a regular user, however on the other hand, if we personalize the content based on the geography, it means that the content that is indexed would be specific to Mt. View, CA in Google's index, correct? I know I heard years ago that the best practice was to use javascript to personalize the content client side, and block the js with robots.txt so that google indexes a default page and not a geo-specific page. Any insights or advice appreciated.

Hello,

I recently upgraded Firefox and the Mozbar SERP overlay stopped working. I rely on that data so need to get it working. Are others experiencing this issue?

Hi,

Thanks for your responses. There are no issues with robots or canonical tags that are apparent. The 301 redirects are accessible by Googlebot, I checked in Webmaster Tools. And the page that the 301 redirects to on the other domain has a canonical tag set to the proper URL (itself).

A client website was moved about six months ago to a new domain. At the time of the move, 301 redirects were setup from the pages on the old domain to point to the same page on the new domain. New pages were setup on the old domain for a different purpose. Now almost six months later when I do a query in google on the old domain like site:example.com 80% of the pages returned are 301 redirects to the new domain. I would have expected this to go away by now. I tried removing these URLs in webmaster tools but the removal requests expire and the URLs come back. Is this something we should be concerned with?

Here is the other aspect of this that is disturbing to me, Google's implication with this is that other people could claim the same trademarked brand name? So I guess I can go make a G+ page and call it +TacoBellCom and they wouldn't stop me from trying to pass off as the official brand.

Hi, yes you are correct that I'm mainly looking at brands that are not specific to a geographic region. Thanks for your suggestion on branding the URL/domain as an option.

I'd be interested if anyone has any insight as to what drives this. I can find many examples of small companies and brands that were able to claim their brand name (+BrandName) and I have even found examples of companies that claimed keywords instead of their brand names? There doesn't seem to be any connection with company size, followers or online ad spend. The only thing I can figure is the rules used to be different and at some point in time they changed and some pages got in during the window of time when it was different.

I have several brands which I am managing G+ profile pages for. These range from established brands with large followings to just starting out. When I try to claim a custom URL for these on Google+, it says to add some extra characters after the brand names to make them unique. I can't find any example of established big brands who have G+ URLS like "+toyotausa24" "+tacobell3" or anything like that. This does not seem to be well documented anywhere. Can someone tell me what the deal is with this feature?

Also what is the best practice for large brands when claiming this? +BrandName1?

Yes actually you are correct. After I read this answer I tested it on my personal site by adding the tags around some nonsense words. Not only did Google index the pages with the nonsense words making it into their cache of the pages, but my site ranks for those nonsense words. So while it would be awesome if Googlebot honored those tags, they only work for the Google search appliance!

Maybe in GA, just under where you got that chart from, click "Source" and then plot out the different search engines to validate where that extra traffic is coming from. My assumption if you are seeing all the keywords rise and not sudden traffic from one or a few new keywords is that maybe you just started ranking in one engine across the board? Like maybe all the steady traffic you were seeing before was all from Bing but suddenly Google started blessing the site? I know that doesn't answer your question but at least will eliminate some of the possibilities.

Also, the sources you mention for looking for new backlinks take a while to update (like OSE). So new links wouldn't be showing up in there yet. You might want to look in your traffic sources > referrers report in GA to see if there are some new links in there that you didn't know about before. Or if the total number of unique referrers has increase comparing two time frames? Might be a needle in a haystack but just one more place to look.

Good luck!

Have you tried looking in any of the other link graph tools like ahrefs.com or majestic SEO?

Actually I see a very similar breakdown of Google as a referrer and direct traffic for safari as a segment in my Google Analytics as I see for my total traffic so I do believe GA is reporting the referrer correctly for Safari. I have no idea if Safari specifically has more of an issue with "not provided" but I do know that Chrome has been particularly bad all year since it started treating any search made in the address bar as a secure search by default. Maybe Safari does too, I don't know. But the traffic is at least attributed back to Google organic as a referrer, not direct, in my analytics.

Just curious what analytics solution you use? I've been a long time Omniture user and just recently started using GA (not by choice). I was definitely seeing issues with the way Adobe was classifying and under reporting not provided searches from Chrome due to this issue!

lJOkDWdi

Looking at Google Webmaster Tools and I'm noticing a few things, most sites I look at the number of indexed pages in the sitemaps report is usually less than 100% (i.e. something like 122 indexed out of 134 submitted or something) and the number of indexed pages in the indexed status report is usually higher. So for example, one site says over 1000 pages indexed in the indexed status report but the sitemap says something like 122 indexed.

My question: Is the sitemap report always a subset of the URLs submitted in the sitemap? Will the number of pages indexed there always be lower than or equal to the URLs referenced in the sitemap?

Also, if there is a big disparity between the sitemap submitted URLs and the indexed URLs (like 10x) is that concerning to anyone else?

I do have some experience in this area. If you operate a highly measurable marketing program, some web marketing agencies will agree to a "pay per performance" model of compensation, but you will have to work with them for it to be clearly defined, and they will still want a flat rate compensation for their hours spent. At the end of the day, agencies want to get paid period. And they should be. You may end up paying more for their services going this route, so if saving money is your concern I wouldn't recommend it. If ensuring that your agency can deliver and that they have some "skin in the game" to keep them honest, then this could be a great direction.

A typical setup I've seen is the agency will give you their hours at "cost" or a very low rate as a baseline to cover their expenses and time, then if you have very good past historical performance reporting setup, and they are comfortable that they can do what they say they can, you can define a payout based on "results" such as website conversions from organic search sources. So comparing year-over-year, say you got 100 conversions in October 2012 from organic search, you could say for every conversion we get in October 2013 above 100 you get 25% of the revenue, or something like that.

Also keep in mind, the industry is somewhat in free fall right now in my opinion due to the increase of "not provided" keyword data. In the past, you would do a contract like I outlined above specifying that you would not count branded keywords. The last thing you want is to run a magazine ad which increases searches for your brand 2000% and have to pay the agency for the influx of organic search conversions that you would have gotten anyway! With all the organic search data lumped into one bucket now, I don't see how that will work anymore personally.