A couple of days ago, Google announced the release of a new Google Search Console API called URL Inspection API.
The URL Inspection API are a good way to access URL-level data outside of the Google Search Console interface. It provides different information like:
- date of the last crawl
- indexing status
- canonical tag
- mobile-friendliness
- structured data
- …
If you want to know more about the Inspection API I recommend you to read the official documentation.
In this post, I’m going to show you how to access the URL Inspection API using R. This tutorial is quite similar to the one I wrote about Google Indexing API in R.
How to generate Google URL Inspection API credentials
The code that I’m going to show you only work if you activate the Google Search Console API and generate a client id and client secret.
Open the Google API Console and search “Google Search Console API” in the APIs library and then you have to enable the API.
Then go to the Credentials section where you’ll be able to generate your credentials. Click on Create Credentials and choose the OAuth client ID option. 
Select Desktop App and choose the name you want.
Now, you should have a popup on your screen showing your client id and client secret credentials.
The R code for the URL Inspection API
My R code uses as input a .csv file like the one you see below. I decided to use it because I know that many SEOs are not familiar with R and they prefer to work with Excel / CSV files.
But if you know how R works you can directly use a vector containing all the URLs you want to analyze.
Let’s not forget that the URL Inspection API has a limit of 2,000 queries per day!
Once I ran for the first time the R code, I realized that in the response of the API there are 3 different types of information:
- indexStatusResult – which contains the information about the last crawl, canonical tag, indexing status, sitemap,…
- mobileUsabilityResult – which contains the result of the mobile usability test
- richResultsResult – which returns all the structured data implemented on that url
In my case, I decided that I wanted as output only the first “sub-report” indexStatusResult.
The code below accepts in input a .csv file of URLs, if you want to use it you have to replace the 4 placeholders you find in the code:
- client id
- client secret
- siteURL – the name of your Google Search Console Property e.g. https://www.mydomain.com/
- languageCode – in which language you want the output
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 |
# LAST UPDATE: 2022-02-01 # Install & Load the packages install.packages("googleAuthR") install.packages("tidyverse") install.packages("readr") library("googleAuthR") library("tidyverse") library("readr") # Set credentials and scope clientId <- "PASTE HERE YOUR CLIENT ID" clientSecret <- "PASTE HERE YOUR CLIENT SECRET" scope <- "https://www.googleapis.com/auth/webmasters" options("googleAuthR.client_id" = clientId, "googleAuthR.client_secret" = clientSecret, "googleAuthR.scopes.selected" = scope, "googleAuthR.verbose" = 0 # Not mandatory - I just use it to debug the script ) # Google API OAuth gar_auth() # List of URLs - Daily Limit of 2,000 queries per day urls <- read_csv("~/Desktop/Your-file-name.csv") urls <- urls[ ,1] # get_inspection_api function - you can use the function to send requestes to the inspection API using urls vector as an input # It also GET the response from the API and stores it in a data frame get_inspection_api <- function(page) { body <- list(inspectionUrl = page, siteUrl = "PASTE HERE YOUR SEARCH CONSOLE PROPERTY NAME", #e.g. https://www.mydomain.com/ languageCode = "it-IT" # CHANGE IT WITH YOUR LANGUAGE CODE ) f <- gar_api_generator("https://searchconsole.googleapis.com/v1/urlInspection/index:inspect", "POST") result <- f(the_body = body) result <- data.frame(result[[6]][[1]][[2]]) return(result) } # The API responses are stored in a data frame api_response <- map_dfr(.x = urls, .f = get_inspection_api ) # You can download the API response as .csv file write.csv(api_response, "Your-file-name.csv") |
And this is an example of the output you can get from the URL Inspection API with some of the columns.
Hi Ruben, you made a useful guide.
On some URL I’m receiving this error on RStudio:
Error in (function (…, row.names = NULL, check.rows = FALSE, check.names = TRUE, :
arguments imply differing number of rows: 1, 3, 2
Within my investigation, I checked the differences of output of this specific URL on Google Search Console and it seems the error is caused by the presence of more than 1 result (currently 2) in “referringUrls”. All the other URLs with only 1 result in “referringUrls” aren’t affected by this error.
Could it be a bug in your code?
Yes, it could be. I have to run some tests and I’ll get back to you.