If you want to achieve an SEO success, your site has to be indexed. That’s the underlying assumption. If the search engines don’t know about your URLs, they can not show them in the search results. It sometimes happens that search engines for some reason do not index some pages or specific segments of your web.
You can find URLs that are not indexed using the Miner Fulltext Index Checker, which will also help you to discover why your site is not indexed.
Search engines do not allow bulk indexing. Tools such as Google Search Console show only basic statistics on the number of indexed URLs but do not provide specific URL data. And operator site: unfortunately doesn’t offer 100% reliable data.
You can use Fulltext Index Checker:
- If you just started to work on a new project and you are working on the initial SEO audit. One of the main steps is to find out if your URLs are correctly indexed.
- You noticed a drop or no traffic from an organic search to a specific page, and you need to find out if the site happened to fall out of the search index.
- If you have any doubts about the status of your site indexation.
- If there is a significant fluctuation in the indexing curve, for example, in Google Search Console. The sooner you find out which URLs have been deindexed, the easier it will be to detect the cause of it so you can fix the problem as soon as possible.
Get started by clicking on Create report button and specifying URLs at the input. Then write or copy the dataset of the URLs for which you want to find out if they are indexed.
For a report to be easily identifiable, it is good to name it. Simply click into Your Report Name field.
Then click on the flag to select the country for which you want to get the data and finish by clicking on Next Step.
Selection of the Miner and data collection
In the Miner selection section, click on Miner Fulltext Index Checker. This Miner inquires to search engines about given URLs by search operator info:. This way the Miner checks if given URL is indexed and if a search engine returns the same URL as the one that was presented in the input.
Update: Google doesn’t support operator info: . So we developed our own way how to check if URL is indexed or not!
Nothing else needs to be set. Click on Get Data to start processing your inputs. Once the report is complete, the processed data will be sent to you by email.
- Input: URLs, of which indexing was checked.
- Indexed by Google: Detection, whether URL is indexed by the search engine. It returns either yes(indexed) or canonicalized or not indexed(because Google doesn’t support info: operator anymore, we can’t be sure if the URL is not indexed or just canonicalized to another URL).
- URL in results: Information about what URL, entered by the operator info: was returned by the search engine.
- Same as input: Comparison, whether URL at the output is the same as the one at the input. It can be useful for canonicalization functionality identification as well.
Check of non-indexed pages
Regarding output, you should be primarily interested in the column Indexed by Google that indicates if given URL is indexed in given search engine (TRUE/FALSE indication). The correct procedure is filtering out the list of non-indexed pages and using these to find out why they aren’t shown in the search engine index and how to fix the situation.
In individual cases, there could be a different URL at the output, by using operator link:, then the one at the input. This is a sign of the fact that the search engines know about given URL, but they use its canonized URL in search results. To detect these URLs, see the column Same as input, which returns either TRUE, in the case given URL at the output is the same as the one at an input, or FALSE, if it is not.