How to Automatically Submit a Sitemap to IndexNow#
If you use GitHub Actions to build and deploy your website, you can automate the process of submitting your sitemap to the IndexNow API. This is particularly useful if you have a large number of pages on your site that you want to submit all at once.
Find a simplified version of IndexNow for Python on GitHub Actions and begin automating your workflows:
Workflow Automation with GitHub Actions#
Regardless of whether your codebase is highly or less active, it is recommended that you do not submit your sitemap to IndexNow each time your site is deployed to production. This could overwhelm IndexNow and reduce the website's ranking.
Instead, you should automate sitemap submissions to IndexNow using GitHub Actions at regular intervals, allowing the changes to accumulate over time and allowing the search engines crawlers time to index the latest content.
Example workflow with GitHub Actions:
.github/workflows/submit_sitemap_to_index_now.yml | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
|
Checklist
Before running the workflow, make sure you have done the following:
- Added the API key
INDEX_NOW_API_KEY
as a secret to your repository. - Uploaded the API key to the location specified in the
api_key_location
parameter. - Updated the URL of the sitemap in the
sitemap_location
parameter. - Adjusted the
host
,endpoint
, and other parameters to suit your needs.
Scheduled Runs#
The example above can be adjusted to run at different intervals. Simply adjust the cron
job definition to your needs.
Warning
Too many submissions to any of the IndexNow API endpoints could result in your site being ranked lower by search engines, maybe even blacklisted. It's highly recommended to only submit sitemaps to the IndexNow API once a day or less, ideally only the latest updated URLs.
Monthly Schedule#
Run the workflow at midnight UTC on the first day of each month:
.github/workflows/submit_sitemap_to_index_now.yml | |
---|---|
5 |
|
Weekly Schedule#
Run the workflow at midnight UTC on the first day of each week:
.github/workflows/submit_sitemap_to_index_now.yml | |
---|---|
5 |
|
Daily Schedule#
Run the workflow at midnight UTC on a daily basis:
.github/workflows/submit_sitemap_to_index_now.yml | |
---|---|
5 |
|
Only Submit Latest Changes#
Rather than submitting all the URLs in the sitemap as one large payload to IndexNow, you can also submit only the latest changes by targeting the latest modification date of each URL in the sitemap using the <lastmod>
tag. This is particularly useful if you have a large number of pages on your site that you want to submit all at once when deploying your site.
Simply adjust the sitemap_days_ago
parameter to the desired number of days, as highlighted below:
.github/workflows/submit_sitemap_to_index_now.yml | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
|
Event-Triggered Workflows#
Imagine the sitemap is submitted before the site has been fully deployed. This is something we want to avoid, as otherwise we won't be using the most up-to-date sitemap.
There are several ways to trigger workflows in GitHub Actions. The most common options in this context are:
- Use
needs
when having several jobs in the same workflow file. - Use
workflow_run
when you want to trigger jobs in different workflow files.
Tip
If you want to update the previous workflow example, simple update the trigger to workflow_run
instead of schedule
and adapt the workflow names to your needs:
.github/workflows/submit_sitemap_to_index_now.yml | |
---|---|
3 4 5 6 7 |
|
GitHub Pages#
For users of GitHub Pages, you can still use the automated index-now-submit-sitemap-urls-action
workflow. This will also work if you're using MkDocs to build your site with GitHub Pages, and it will probably work in many other cases as well.
All you need to do is adjust the on
condition and adapt the example below to your needs:
.github/workflows/submit_sitemap_to_index_now.yml | |
---|---|
3 4 5 6 |
|
Info
The workflow_run
event is used to trigger the workflow after the GitHub Pages build and deployment is completed. This ensures that the sitemap is submitted only after the latest changes are live.
Custom GitHub Actions Workflow#
You can customise your workflow even more, although the starting point is the same: Put a YAML workflow file in the .github/workflows
directory of your repository.
Imagine that we want the workflow to be triggered every time you push to the master
branch or create a pull request to the master
branch, and we will then execute the submit_sitemap_to_index_now()
method.
Example of a workflow file:
.github/workflows/custom_workflow.yml | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
|