How can I make a list of all URLs indexed by Google?
-
I have a large site with over 6000 pages indexed but only 600 actual pages and need to clean up with 301 redirects. Haven't had this need since Google stopped displaying the url's in the results.
-
@aplusnetsolutions create a seperate sitemap for not indexed pages and upload to google web master tools. i had a website had same issues. Also remember it will take some time also
-
@aplusnetsolutions
Make sure all these things are selected in screaming frog and -
Going to Google search console it will tell you how many inches are inside Google search index there are a few other ways to do this if you want
#1
By far the easiest is to going to search console and select coverage and all known pages
number 2
For the quick and dirty method, simply perform a simple site search in your Google search bar with “site:yourdomain.com”
#3
You can crawl your site using screaming frog (it will 500 pages for free but every SEO should have it) use it to index your site and connect it to Google search console API as well as Google analytics and you will have a easy-to-read CSV export of your index and non-index pages
more on how this is done
video https://youtu.be/iYeXSdUt_hg
#4
or indexer site with any thing that can export a CSV download the CSV uploaded here and it will tell you which pages are in Google's index
https://www.rankwatch.com/free-tools/google-index-checkerhttps://searchengineland.com/check-urls-indexed-without-upsetting-google-follow-267472
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
SEO title tag, url
Hi I am launching a website that will sell initially 1 product with different colours, sayings and slogans. Is it okay to use the same keyword for the title tag and url. eg Love dog bow tie, sassy dog bow tie, I am the boss dog bow tie.
SEO Tactics | | SuseO
Thanks for any help.
Susan0 -
Redirecting Homepage to Subdomain Bad or Good Idea??
I have a very old forum that still gets a lot of traffic, but when migrating over to another software that is cloud based we cannot redirect using same domain, SO the only option would to be to change the cname on a subdomain and then REDIRECT all the traffic from the ROOT domain permanently - would this be a bad move as the root domain wouldnt be used anymore as now its just setup to be redirected in order to use the software we need to use? Domain is 17 years old.
Technical SEO | | vbsk0 -
Is 301 redirects a deal breaker for Migrating content or moving to new software?
I have this forum with about 2 million posts for 16 years on root of the domain. I am looking to switch softwares but the top ones won’t help setup 301 redirects. But I can still migrate all my members and all my content (threads/posts), would Google still reindex all our content or if we don’t setup redirects would it really kill our entire traffic for a long time or maybe just a month or so? I really want to migrate to software that isn’t forum based but rather something that offers courses, chat, live video streaming, subscription based etc. and this is the only way to do so OR to set it up on an entirely new domain OR subdomain but to me that is like starting all over from scratch? I could archive the forum to read only and set it up on subdomain or another root domain - then on the archived forum setup banners and a pop up linking to the new site or new subdomain? . This is such a hard decision for us as the current forum we have had for so many years has lost members posting from 1k a day to just a handful a day, our fb group though gets 1k a day so I’m trying to revive a site into something more modern and has all the training features we can offer as well.
Technical SEO | | vbsk0 -
Slug redirect to existing subdomain?
Hi everyone, I'm Levi, new here and new in the world of SEO so please don't judge if my question is silly. Back on the days when my website was built we tought of having subdomains to help with SEO. We have a couple subdomains set up https://stansted.tonorwich.uk, https://heathrow.tonorwich.uk, https://gatwick.tonorwich.uk, https://luton.tonorwich.uk etc .
Link Building | | Leviiii
We do have some content on these, but from seo perspective creating a slug and redirecting it to the subdomains would be beneficial? Would it make any difference at all, good or bad? Ex. Creating
https://tonorwich.uk/taxi-minibus-tofrom-stansted and redirecting it to https://stansted.tonorwich.uk
Will this help or we better create different content on the slugs? Waiting for your opinions.
Thanks in advance.0 -
Multiple URLs from One Domain on Page 1
Is it possible (and if it is possible, how difficult is it) to get multiple URLs from the same domain name to be indexed on the first page of Google for the same keyword or phrase? Assuming each URL has significantly different content of course... Context: A large real estate brand has 6 franchise offices in the same city. Each office has its own listing/landing page on the parent brand website. Each franchise owner wants their page on the website to rank for the term 'Christchurch real estate'. The homepage of the parent brand website currently ranks on the first page of Google for this search term, but none of the franchisee pages do. So my question is: with the right unique content on each franchisee page, supported by quality backlinks to each of the different offices, is it possible to get multiple franchisees listed on the first page of Google given that ultimately the 6 URLs are all attached to the same domain name? (And, if so, do you have any hot tips you can share to assist me on this uphill battle?)
SEO Tactics | | BeKonstructive0 -
What is the difference between "document" and "object" moved redirect errors?
What is the difference between "document" and "object" moved redirect errors? I'm used to see "object moved" as a redirect chain issue that needs to be fixed, but this week my report contained a "document moved" redirect chain issue. And it's on our homepage. Looks like it might be a HTTP versus an HTTPS issue.
Reporting & Analytics | | Kate_Nadeau0 -
Plagiarized Site Effecting Google Rankings
Can someone provides insights on a de-indexing example? I have gone through the depths of Google lack of support and requesting duplicate content flags, so no avail. Here's the scenario: Client had a competing SEO provider try to earn his business. In doing so, he copied word for word our blog that we have been producing content on over the last 5 years. He also integrated Google reviews in the structured data on this new URL. Well, fast forward 1-2 months later, our rankings started to drop. We found this 100% plagiarized site is taking away from our keyword rankings on GMB, and is no and Google search, and our site GMB is now only displaying on a branded name search as well as our search traffic has dropped. I have identified the plagiarized, duplicated content, being tied to our GMB as well, as the source of the problem. Well, I finally obtain ed control of the plagarized domain and shut down the hosted, and forwarded the URL to our URL. Well, Google still has the HTTS version of the site indexed. And it is in my professional opinion, that since the site is still indexed and is associated with the physician GMB that was ranking for our target keyword and no longer does, that this is the barrier to ranking again. Since its the HTTPS version, it is not forwarded to our domain. Its a 504 error but is still ranking in the google index. The hosting and SSL was canceled circa December 10th. I have been waiting for Google to de-index this site, therefore allowing our primary site to climb the rankings and GMB rankings once again. But it has been 6 weeks and Google is still indexing this spam site. I am incredibly frustrated with google support (as a google partner) and disappointed that this spam site is still indexed. Again, my conclusion that when this SPAM site is de-indexed, we will return back to #1. But when? and at this point, ever? Highlighted below is the spam site. Any suggestions? Capture.PNG
SEO Tactics | | WebMarkets0 -
Spam on Google SEO
Do you know any good tips to reduce spam and if spams have an on google ranking?
SEO Tactics | | easyjobber0