Welcome to the Q&A Forum

Very hard to prove these things before they're done - good luck with getting buy-in for what you need to do and in undoing the worst of the damage.

Good luck!

You're not the only person reporting odd indexation happenings here on Q&A (see for example this question). And, just like I found for that question, your site appears to have more pages indexed in Bing than in Google - which at least seems to point to us not having missed something obvious like meta noindex or similar.

I did also read Google saying that they had issues with the site: command (link) but I don't think that can have anything to do with your situation as they say they have now fixed that issue, and I couldn't find any other pages on your site even with non-site: searches (i.e. it does genuinely appear as though those pages are missing from the index).

While I am loathe to point just at links these days, I do wonder if in this case it is just a case of needing some more authority for the whole site before it is seen as big enough and important enough to justify more pages in the index.

It sounds as though you should be OK in that case - if they are all site.com/post, then it shouldn't matter how many categories they are in.

In theory you can have Topics and Categories - it all depends on how the site is set up, but I would probably say it's best to focus your efforts on one if I had to guess without knowing the site and all the considerations inside out.

Good luck.

It seems like Google only has a handful of distinct pages indexed at the moment - whereas bing has about 10x as many. So that seems to indicate something wrong specifically for Google.

I'd start by checking your search console - are there any errors? If you use the URL inspection tool and visit the URL in question (studyplaces.com/15-best-minors-for-business-majors/) does it tell you why it has been canonicalised? What happens if you view the page as Google saw it? Is there any chance that you are blocking googlebot / cloaking in any way? Have you had any website outages or downtime?

As others noted, your about page is now missing - did you do that deliberately to see if it resolved this issue?

The answer Google would like you to believe is that it's possible for the word ordering to imply different intents. In reality though, I think it's mainly an artefact of them not fully understanding meaning, or not being able to classify all pages and keywords perfectly.

My colleague Sam Nemzer wrote a post with research on this topic that you might find interesting.

This sounds tricky - it appears to be indexed for me at the moment.

Some things to check:

Server monitoring - is the website up and available at all times (including e.g. robots.txt)?
Server logs - are you serving any status code other than 200 to googlebot at any point?
Search Console - are there any errors recorded, or if you go to the URL inspector during an outage, has it canonicalised the homepage to any other page somehow?
I noticed a heavy reliance on JavaScript - and some of the text that appears on the homepage doesn't appear to be returning the homepage on a search for it in quotes (e.g. ["Not just any old snack box, SnackMagic lets everyone customize"] so I'd check for JS rendering issues for googlebot as well

Hope something there helps.

Hi there,

The answer to this depends a little bit on how your CMS / website treats content that appears in multiple categories. The "correct" way of it working is that those pages are still accessible at only one URL (i.e. not at both /category1/slug and /category2/slug ). This is normally achieved either by having a primary category that appears in the URL (and then having that content also appear on the category2 page but not with category2 in the URL) or by having the content available at /slug and appearing on both category1 and category2 pages.

Assuming this is the case, then it's perfectly safe to have content appear in more than one category.

If not, you could investigate whether you can add a rel=canonical link from secondary categories to the primary category. This would be OK but you might want to limit it only to times when content really needs to be in both categories otherwise you may waste crawl budget depending on the scale of your site.

I actually wrote a post about the difference between URL structure and site architecture that you might find useful.

Hi Jeroen,

Many websites have category or listings pages that contain substantially different lists of links each time Google crawls them. This can be because they are rotating the top listings (like you describe) or simply because the velocity of content creation (and in some cases archiving / removal) is high enough that it appears to change dramatically (think e.g. the reddit "new" page).

As such, I don't think you need to do anything particularly special here - it should "just work" for the page in question - depending on the details, you might want to make sure that there is enough other content on the page that it is substantial enough in its own right.

The other thing I'd consider is whether you want to have more static crawl-paths available to make sure that googlebot always has a way of discovering and crawling all listings - whether you do this via categories, tags, or via some other means.

In addition to the advice and tips you have already received here (in general: be super careful with .htaccess / httpd.conf files, and revert to previous versions if you see unexpected behaviour) one additional tip is to consider turning on logging while you debug the problem.

@MarathonRunner - you are correct in your inline responses - it's totally valid to serve an image (or other filetype) without an extension, with its type identified by the Content-Type. Sorry that you've had a less-than-helpful experience here so far.

To answer your original questions:

From an SEO perspective, there is no need that I know of for your images to have a file extension - the content type should be fine
However - I have no reason to think that a filename in the Content-Disposition header will be recognised as a ranking signal - what you are describing is a rare use-case and I haven't seen any evidence that it would be recognised by the search engines as being the "real" filename

If you can't always refer to the image by its keyword-rich filename, then could you:

Serve it as you propose (though without the Content-Disposition filename)
Serve a rel="canonical" link to a keyword-rich filename (https://example.com/images/golden-retriever in your example)
Also serve the image on that URL

This only helps if you are able to serve the image on the /images/golden-retriever path, but need to have it available at /images/123456 for inclusion in your own HTML templates.

I hope that helps.

Hi there. Unfortunately, I would err on the side of saying that not only are those links most likely not helping, but there is a significant risk of them **hurting **your performance.

In general, any links that you can just get in this fashion by purchasing them off a public forum with nothing related to any differentiation of your business are very much going to be links that Google is going to want to discount (at the very least - if not penalise).

The mention of "guarantee" in the second paragraph is a warning sign for me too - I'd advise you to dig into the actual tactics and know for sure what is being done in your name and for your website.

Having said all of that, as a general rule of thumb, because of the power-law structure of the web, a few very powerful links can be more effective than many weaker ones.

Thanks for checking this out and not just taking it at face value guys. I have to admit I didn't test it in detail. It seemed plausible because GA is generally last touch, but I'm glad to see someone testing it. I have updated the post.

Unfortunately most of the largest examples are not shared publicly. You could go looking for external data (e.g. SEMrush / searchmetrics / similarweb) around the time of big / well-known mergers. There have been a few over the years - but I don't know off-hand what historical data there is available. The biggest I remember being interested in was when Adobe bought Macromedia back in the day (but there won't be historical data going back that far, and anyway everything has changed since then).

Aside from "outside looking in" approaches, the biggest public case study I could find is this one.

Hope something there helps.

Hi Simon,

It's not looking great. I don't have any inside information from Bing / Microsoft, but there seem to be SNI pages that are still not being indexed correctly. See, for example, this page: https://www.mnot.net/blog/2014/05/09/if_you_can_read_this_youre_sniing which is nowhere to be found in Bing's index.

A site: search on bing finds two posts from the mnot site (out of hundreds). So it appears likely to still be flaky but I haven't gone down the rabbit hole of looking for other reasons that site may not be indexed, nor checked that it actually is using SNI. I also don't have an SNI site to test things on, so that's the best I have for you, I'm afraid.

The only other link I found that may be interesting is this page claiming that SNI is supported by bing: https://www.ssllabs.com/ssltest/viewClient.html?name=BingPreview&version=Jan%202015

Sorry that's not more useful.

You could reach out to the author of that SEL post (Ben Goodsell) and see if he's seen any changes since he wrote it.

There's been a lot of controversy back and forth on this. I have definitely seen sites get hurt by link profiles that competitors could have built (sadly, often it was the company itself). However - I have only seen this for pretty weak sites (as others have said). I have not yet seen a case where a strong site with a natural link profile got hurt by competitors. I wouldn't spend an awful lot of time worrying about it.

In theory, there should be no difference - the canonical header should mean that Google treats the inclusion of /images/123456 as exactly the same as including /images/golden-retriever.

It is slightly messier so I think that if it was easy, I'd go down the route of only ever using the /golden-retriever version - but if that's difficult, this is theoretically the same so should be fine.

Hi Sam. A correctly setup firewall or CDN should not have implications for SEO. If you can share the URL / domain then I can take a look at the specifics, but in principle, you should be fine.

Things to watch out for:

Site speed impacts - in theory a CDN should generally be good for site speed, but if you didn't select one primarily for this reason, it's something to check
HTTPS - ensure that you keep this consistent through the change
Avoid changing URLs in the process (if you have to, make sure you have redirects set up and treat it like a migration with associated risks)

Searchcap is what you're looking for - feed version gives me a daily update of stuff to read: http://searchengineland.com/feeds

I'm amazed that the acronym has spread as far as it has. I see it actually being used all the time. Quite amazing really.

As Hannah mentioned, we have a bunch of internal memes at Distilled which are similar to a bunch of the principles behind TAGFEE. If you're interested in how we embed this stuff in our culture, you might like to read this post Tom wrote about how changing culture with language: http://onstartups.com/tabid/3339/bid/40478/Startup-Culture-Memes-Do-You-Have-A-Duck-Of-Awesomeness.aspx

Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

willcritchlow

@willcritchlow

Latest posts made by willcritchlow

Best posts made by willcritchlow

Blog Posts