Inoreader Fetcher

Inoreader Fetcher is the engine that collects RSS feeds on behalf of users who choose to subscribe to them in the Inoreader app. The Fetcher periodically connects to RSS feeds to check for new content. You might see those requests in your web server logs.

How does Inoreader Fetcher identify itself?

Inoreader Fetcher identifies itself with the following user agent:

Inoreader/1.0 (+http://www.inoreader.com/feed-fetcher; 3 subscribers; )

The “subscribers” number tells you how many Inoreader users are subscribed to your feed. However many subscribers there are, Inoreader fetches your feed only once and distributes the content to all of them. This is an excellent benefit of cloud-based RSS readers, unlike stand-alone desktop applications, where every user connects separately to your website to retrieve the feed.

A handful of websites that reject non-browser clients are fetched with a standard browser user agent. When a request fails, Inoreader may retry it from another network address or through a browser.

How do I request that Inoreader not retrieve some or all of my site’s feeds?

Inoreader allows users to add any RSS feed to their account if they have the public URL. If your website provides a public RSS feed, Inoreader will not stop users from subscribing to this feed. Inoreader Fetcher doesn’t read your website’s robots.txt, so you can’t block it using rules in this file.

To keep Inoreader out of some or all of your feeds, answer requests whose user agent contains Inoreader/1.0 with 404 Not Found or 410 Gone. Inoreader never retries those responses from other addresses. Other refusals, such as 403 or 429, may be retried. If you want Inoreader to stop fetching your whole domain for good, contact us and we will block it on our side.

How often will Inoreader retrieve my feeds?

Inoreader adapts to how often each feed publishes, to give users timely updates without overloading your website:

  • An active feed is usually checked every 30 minutes to a few hours. A feed with only a few subscribers is checked at most once an hour.
  • A feed that a subscriber has boosted is checked every 10 minutes.
  • A feed with Pro or Team subscribers is checked about once an hour, even when it publishes rarely.
  • Otherwise, a feed without new content for a month is checked about once a day, and after a year, about once a week.
  • Feeds that no active user reads are not fetched at all.

Why is Inoreader trying to fetch incorrect URLs from my website?

Inoreader fetches feeds for as long as users are subscribed to them, even if they return errors. Some websites break occasionally, and their feeds return errors until the website owners fix them. Inoreader cannot know if or when a feed will be restored, so it keeps retrying: at the usual pace at first, then about once a day after a month without success, and about once a week after a year. If a feed is gone for good, answer with 404 or 410, or redirect it to its new address.

Why isn’t Inoreader obeying my robots.txt file?

Inoreader is not a typical crawler that downloads an index of your website and crawls your links recursively. Instead, Inoreader only fetches direct RSS feed URLs provided by our users. There is no need to disallow Inoreader from certain pages of your website, as it will not try on its own to fetch content that is not an RSS feed.

What IP addresses does Inoreader Fetcher use?

The Inoreader backend uses a few different pools of IP addresses. An up-to-date list of those pools can be found here: inoreader.com/.well-known/ip_list.txt. Cloudflare recognizes requests from these addresses as a verified bot in the “Feed Fetcher” category.

My site is behind Cloudflare and Inoreader gets blocked or challenged.

Cloudflare knows the Inoreader Fetcher as a verified bot in the “Feed Fetcher” category, but whether verified bots may pass is decided by your zone’s settings. If Inoreader users report that your feed doesn’t update, check the following in your Cloudflare dashboard:

  1. Security → Security rules → Create rule → Custom rules: under When incoming requests match, select Edit expression and enter (cf.verified_bot_category eq "Feed Fetcher"), or (cf.client.bot) to allow all verified bots. Under Then take action, choose Skip, and select All remaining custom rules, All rate limiting rules and All Super Bot Fight Mode rules. Under Place at, choose First.
  2. Super Bot Fight Mode (Pro plans and above): in Security → Settings, filter by Bot traffic, open Super Bot Fight Mode and set Verified bots to Allow.
  3. Bot Fight Mode (Free plan): in Security → Settings, filter by Bot traffic, and turn Bot Fight Mode off. Rules cannot skip it, and it challenges verified bots too.
  4. Under Attack mode challenges every request, including verified bots. You’ll find it in Security → Settings under DDoS attacks; turn it off once the attack is over.

Other firewalls and CDNs: allow the IP ranges from ip_list.txt, or requests whose user agent contains Inoreader/1.0.

Do you support push technology?

Yes. Inoreader fully supports WebSub (formerly PubSubHubbub), and all website owners are advised to support it. Add a hub link to your feed, <atom:link rel="hub" href="…"/> in RSS or <link rel="hub" href="…"/> in Atom. When Inoreader detects it, it subscribes to your hub, receives new items as soon as you publish them, and polls your feed only every few hours as a fallback. If you use WordPress, there is a free plugin that can automatically enable WebSub support for your blog.

Do you support conditional HTTP GET?

Yes, Inoreader Fetcher supports conditional HTTP requests. Inoreader sends If-None-Match and If-Modified-Since from your previous response, and when your server answers 304 Not Modified, the feed is not downloaded again. Compressed responses (gzip, brotli) are accepted as well.

I need to redirect my feeds to a new website or domain.

Inoreader Fetcher follows HTTP redirects. Temporary (302) redirects are cached for 24 hours, after which the original URL is retried. Permanent redirects (301) are remembered indefinitely, unless the new address stops working. Use 301 redirects only if you move your feeds for good.

My question isn’t answered here. Where can I get more help?

If you’re still having trouble, you can contact us at any time.