Nobody sees the trouble I’ve known. Maybe not nobody, exactly, but not many. There are two primary RSS feeds published by this site. One is for this blog. The other is for the trip journal. Among the many ways of subscribing to these feeds is a service called Feedly. It is well done and popular. It is the RSS reader I use. If I didn’t, I wouldn’t have seen the trouble myself.
People subscribe to feeds through Feedly and Feedly periodically fetches current copies and makes them available in a convenient and personalized manner. Feedly users can read and manage dozens or hundreds of uniformly presented feeds without dealing directly with the individual providers. It’s a fairly common producer to broker to consumer arrangement.
Back in March, I noticed that some of my posts were failing to show up in Feedly even after several days. A fairly convoluted email exchange between me, Feedly, and Arvixe, my hosting provider, followed. It was eventually discovered that Arvixe was blocking Feedly and the block was removed. Because of the asynchronous nature of Feedly’s polling and of RSS in general, things were not instantly fixed but caches, buffers, clouds, and other nebulous cyber-things eventually worked through their stockpile of bad-stuff and started working properly again.
Unless you were paying close attention to dates and time stamps, you might not have realized there was any sort of problem at all. The screen capture at the top of the article is from an attempt to subscribe to a blocked feed through the Feedly Google Chrome plugin. Existing subscribers would have seen nothing; No new posts and no error messages. For those unfamiliar with RSS, think of the problem as somewhat similar to a friend trying to call you with a broken phone or service. Your phone doesn’t ring but you don’t know that it should so see no problem. Even the friend may not see a problem if the failure shuttles them to a voice mail system. You only become aware of the issue when the friend confronts you in a bar about never returning their calls.
The problem resurfaced during the first half of June and this time, after about a week of that convoluted email three-way, I learned of a fourth player in the game. Arvixe uses a security product called BitNinja. Something about Feedly’s access tripped BitNinja’s defenses and a block was activated. Unknowing Arvixe technicians repeatedly removed the block only to have it reappear a few hours later. Once in awhile, the short-lived removal and Feedly’s polling lined up so that posts would slip through. They might appear in a clump and they would have the current rather than publication date but they could be seen by subscribers.
When I first learned of BitNinja’s role, I urged Arvixe to configure their installation to have the security service treat Feedly as a good guy. They, for valid reasons, declined. I also pressed them for a description of what specific aspect of Feedly’s access raised BitNinja’s hackles. Near the beginning of the episode, Feedly had suggested they might be able to alter their behavior if specifics were available. Not being privy to BitNinja’s inner workings, Arvixe could not supply those specifics. Although it was Arvixe who first mentioned it, I suspect all of us thought of it about the same time. Feedly needed to deal with BitNinja directly. The problem might not be limited to my website or even to all Arvixe hosted websites. The risk of Feedly being blocked could exist everywhere BitNinja was being used.
Feedly did contact BitNinja directly and, while I don’t know the details of the exchange, I do know that it resulted in BitNinja removing “a too strict log analyzing rule” about a day later. That was just over a week ago and since then my feeds seem to be flowing through Feedly as they should.
Maybe we should have dug a little deeper in March and it would have been nice if we had not thrashed about for a week in June. There were times when I thought Arvixe could have been more cooperative and Feedly more responsive. I don’t think anyone involved is a candidate for Trouble Shooter of the Year but neither do I think anyone screwed up horribly. I can’t be certain that the same problem won’t pop up again in a couple of months though I’m positive that some problem will pop up someday. This episode has increased my confidence that, when it does, these vendors — Feedly, Arvixe, & BitNinja — will get it sorted eventually. It would be nice is the next problem is as invisible to readers as this one was but I’m not going to count on it.
Incidentally, this post’s title comes from a Wheels song that, in case you don’t remember it, is here.
You should send Arvixe, BitNinja, and Feedly a consulting bill.
I suppose you’re right but I don’t want to financially burden Feedly in any way. 🙂 I want so much for them to succeed but it’s really hard for anyone to make a living from RSS. The simplicity that makes it so wonderful also makes it hard to monetize.
Hi Denny,
I’m Bogi, member of the BitNinja team. It was interesting to read about this problem from the user side, and I’m really happy this inconvenience is solved by now.
In case of good bots, we maintain an always up-to-date list about them, based on reverse dns, GeoIP, feedbacks, user-defined whitelists and the behavior of our honeypots. Sometimes all that’s not enough, so we try to keep continous contact with RSS providers and highly appreciate every feedback about these kind of situations. So if you experience any odd phenomenon in the future, please contact our team, we are always happy to help.
Thanks for the comment. Although I’ve had no direct contact with BitNinja, you folks seemed to respond promptly to Feedly’s query. Thanks for that, too. I was somewhat surprised that Feedly was not already identified as a “good bot” but it is a big web out there.
We try to cooperate with all the popular bots (like Google or Feedly), but sometimes it’s hard to keep up with their IP set updates. That is what causes complications sometimes. 🙂
Feel free to contact us on any situations in the future.
Oh my goodness! I wish I had read this post 2 months ago – going through the same thing now for my client’s blog and Feedly and Hostinger that uses BitNinja. Seems 6 years later it’s still an issue and I’m currently being passed between the three of them. Definitely not being paid enough for this.
I have been busy and not paying close attention lately so your post prompted me to immediately check my own feeds. For the record, I encountered a different problem with Arvixe and left them some time back. For what it’s worth, I still use Feedly and both of my feeds appear to be functioning well in recent history. I doubt I can be of any help but I sure do wish you luck.
I realize that is an old post but I am having the same issue. How exactly did you go about having this resolved? Did you contact Feedly and asked them to talk to BitNinja? I’ve been given the run around between my blog host (Hostinger), their security firm (BitNinja) and Feedly without a permanent resolution of the issue. It’s been so frustrating.
Yes, it basically took Feedly and BitNinja communicating directly to get it sorted. Unfortunately, it was some time ago and I’ve lost any emails that might offer clues as to what I did to get that to happen and my own memory is no more useful. I think the key is to get BitNinja’s cooperation since only they can really determine what is being triggered. That does not necessarily mean that the solution lies with BitNinja.
Thank you so much for your reply, Denny. I have contacted Feedly (to ask if they can directly contact BitNinja). l
BitNinja told me that according to their security protocol, ” greylist IPs (like the ones from Feedly) will be challenged by a CAPTCHA to inform the user regarding the block to the domain/website and give them the opportunity to remove themselves.”
Since Feedly pollers are basically “bots”, they can’t resolve the CAPTCHA to get in.
I have yet to find out why the Feedly IPs are greylisted/blocked in the first place. It’s a legit RSS Reader Service.
Anyway, sorry for the blabber, and thanks again for your response. It’s 9 years later and there are still issues. Go figure.
Getting the right people on both ends talking is the first step. Hopefully things are on that path. If BitNinja can determine what is putting Feedly on a greylist then maybe a decision can be made that either Feedly quits doing that or BitNinja quits being so picky.
Although the symptoms may be similar or even identical, it’s unlikely that the issue is the same as what I saw. Neither Feedly nor BitNinja (nor your installation nor mine) have been static for nine years. One guy’s improved security can be another guy’s loss of a feature.
After way too much back and forth my client ended up moving to a new host (wordpress.com) and the issue was resolved. I also had several back and forth conversations with feedly, hostinger, and bitninja and they all blamed the other services.
Oh man, Amanda. That’s discouraging to hear. I just moved to Hostinger last year and am overall so happy with their hosting service, but the bit with BitNinja is so frustrating and I also feel they’re all blaming each other.
It’s really a deal-breaker if it can’t be resolved. Thanks for chiming in.