Skip to main content

Disabling HTTP WebSub for HTTPS Feeds

FeedMail supports WebSub for real-time feed updates. WebSub is a great technology that allows faster and more efficient content distribution. However WebSub hubs have the ability to inject arbitrary content into feeds. This is normally not an issue because feed owners will use hubs that they trust. The decentralized nature of WebSub even allows feed owners to run their own hub and trust no one.

However if a WebSub hub is hosted over unencrypted http:// anyone who can see traffic between the subscriber (in this case FeedMail) and the hub can inject arbitrary content into the feed. We consider this too risky for too little benefit and will be removing support for this configuration.

When a user sees an https:// feed they expect that it is private and secure. In order to maintain this expectation FeedMail already takes a number of precautions.

  • https:// to http:// redirects for feeds are not followed.
  • Feed URLs are never updated to http:// URLs. (Generally, FeedMail follows feed URL changes.)

This change should ensure that users can trust the content of any https:// feeds.

What's Changing

Starting May 24th FeedMail will no longer use WebSub for https:// feeds if the WebSub hub is not https://. Affected feeds will continue to be polled like other non-WebSub feeds. If a feed specifies both http:// and https:// hubs the https:// ones will be used.

Update May 24th: This change is now active. No new insecure subscriptions will be made on insecure hubs. 

Updates from existing http:// hub subscriptions will continue to be accepted until the end of their lease (up to 7 days) at which point they will be ignored.

Example

If https://example/feed.atom contains a link to the hub http://example/hub that hub will be ignored. In order to use WebSub for https:// feeds the hub must be https:// as well.

Exception

pubsubhubbub.appspot.com is the most popular hub and it supports HTTPS. However many feeds reference it using http://. FeedMail will use the https:// endpoint for these feeds as if that is what they specified. This is not expected to cause any problems and will simply improve security.

This exception may be removed at any time, we highly recommend that feed owners update their feeds to reference the https:// endpoint. It should require no other changes on your part.

Effect

This is expected to have no significant impact on users. Despite FeedMail subscribing to hundreds of feeds via WebSub, only 8% of those feeds currently have this issue.

Of those, 75% will be covered by the above exception.

The remaining 25% feeds are all using http://medium.superfeedr.com/ as a hub. This hub is intended for use by Medium which has a broken WebSub implementation that never sends updates. This means that ignoring this hub will actually improve update speed as FeedMail will poll more often.

So in conclusion, 8% feeds will become more secure and 2% of those will also update faster. No current feeds will become worse in any way.

For Feed Owners

If you are a feed owner please inspect your feeds for links that start with http://. These links hurt your reader's privacy and risk the integrity of your content. We recommend using https:// for all links even if they are covered by the above exception.

If you are still unsure of how this change will affect your feed feel free to reach out to FeedMail support.

Labels:

Comments

Post a Comment

Popular posts from this blog

Delivery Delays to Gmail

In the past 48 hours Google has started delaying the delivery of some FeedMail notifications. This is currently affecting about 10% of messages to Gmail users. These notifications will be resent with a delay. We also speculate that some notifications will be marked as spam.   Update : As of 2023-05-09 this appears to be resolved. If You Are Affected If you use Gmail you may be affected by this. Notifications may be delayed or marked as spam. If your notifications are marked as spam you can create a filter to avoid this. Use "from:*@feedmail.org" as the rule and select " Never send it to Spam". If your notifications are delayed we are unaware of any action that you can take. However marking notifications that ended up in your spam folder as "Not Spam" may help avoid future delays.  It does appear that these emails are eventually being accepted but we are unsure if that means that they are actually ending up in users' mailboxes (or even their spam folder
Post a Comment
Read more

Updates to HTML Processing

Since its inception FeedMail has done processing on HTML content in feeds to ensure that it renders as expected in email form. At first this was fairly simple things like rewriting URLs to point to the correct location (many feeds use non-absolute URLs that won't work in email) but over time more complex transformations were added such as adding fallback content to media embeds without any. The full-text scraping feature requires even more complex processing as it requires stripping away most of the page and handling content that was designed for full-featured browsers. What changed? Recently FeedMail has migrated all HTML rewriting to use new infrastructure. This provides more flexibility and enabled new features (such as showing controls on all media embeds) and made our processing much more reliable. What does this mean to me? As a user you shouldn't see much difference. Overall the emails you receive should be better formatted but the difference will be subtle. Full-text sc
Post a Comment
Read more

No body notification option.

Previously FeedMail provided two options for notification bodies: Content from the feed. Attempt to scrape content from the linked website. A third option is now available which includes no content in notifications. This can be useful to reduce email size or if you prefer reading articles in your browser anyways. This option is especially useful for digests. By setting subscriptions in a digest to "No Content" you can get a headlines-only digest. Or you can keep content for some short content like micro blogging but just get headlines for news sources with longer articles. Simply go to your subscription management page to change this setting. You can find a link at the bottom of each email notification.
Post a Comment
Read more