It was more than four years ago that Stephen Downes whipped up a custom version of his edu_rss application for the Merlot conference being held here in Vancouver. It scanned the posts of what was then pretty much the entire education-technology blogosphere (granted, it was smaller then, but it had a ton of feeds), and anytime anyone mentioned “Merlot” in a post it was republished onto a custom page that Stephen had pirated repurposed from the official Merlot site. It worked beautifully, it was dead easy for the users, and I remember thinking it would only be a short while before this sort of functionality was available to everyone.
I’m still waiting. What I want is the ability to take any number of feeds, filter and re-organize them with minimal stress on user behavior, and republish them where I wish.
There’s no shortage of tools that promise this functionality. Most of them break down if you add more than a half dozen feeds. Few of them seem to like OPML. Other approaches require ordinary users to employ arcane techniques to facilitate the process (and they may not work anyway, hello Technorati tags), or an editor to manage even simple sorting and filtering. We never could get Stephen’s edu_rss to run on our local server. We developed our own system via a guerrilla in-house project that got tantalizingly close to fulfilling the EduGlu dream, but it fell victim to a lack of ongoing funding and student programmers’ inexplicable desire to graduate and get on with their lives.
The past year has seen the introduction of mash-up editors from major players like Yahoo!, Google, Microsoft, IBM among others, and the processing and filtering of feeds should be trivial for them, but all that my experiments with these tools have yielded to me is literally days of my life poured irretrievably down the sinkhole of wasted time.
And yet I blog merrily along, talking about the power of syndication to anyone who will listen… and I’m still trying to achieve what seemed to be just around the corner four years ago.
There has been some promising progress on the ‘RSS portal’ approach, and I had begun to recommend this method to most people as the best available technique. A librarian here wanted to create a simple portal of relevant journal feeds for a course, but has found the technology buggy at best (tabs of feeds disappearing, funky rendering in some browsers) and is uncomfortable with depending on it for a course. Oh yes, it also doesn’t seem possible to export OPML from Pageflakes, so if things go wrong it’s laborious to reconstruct. I have had some success with sharing Netvibes tabs (like this — 
), but that system requires viewers to set up a Netvibes account to do more than preview content. It’s gotten to the point where our working plan is to handcode HTML tables and paste in Feed2JS javascript for each source… which just strikes me as an insanely laborious way to provide simple RSS rendering with a stable, public URL.
Add this into the mix — I’ve had remarkably good luck the past few years with grant applications and conference proposals. But so far, each submission that’s focused on RSS or syndication in any way has been rejected — come to think of it, these have been my only failures. I recently suggested a paper to some peers proposing to articulate “The State of Syndication” and that idea seemed to strike absolutely nobody as worth doing. It’s as if I’m raving about professional wrestling as the future of online learning or something. I honestly wonder if my enthusiasm and interest in syndication is grievously misplaced.
Brian, I know you’re not a regular on the legal blog circuit, but check out my latest … an RSS mashed site of new publications from Cdn publishers – legalpubs.ca.
My next stage is to scrape a number of university law school library websites, and integrate a yahoo pipes feed mix in for newly added books.
Your vision is similar to what I’ve been trying to find the time for. Just takes a little longer when you have a day job. Love the graphic, btw. All the best, Steve
My new version of Edu_RSS (which will be renamed) is just a shirt way away from bing released as open source (just awaiting NRC permissions). No, it will not do everything, but it should at least run. And if it doesn’t, I’ll help you out.
> There’s no shortage of tools that promise this functionality. Most of them break down if you add more than a half dozen feeds.
Oy. yes. Hardest problem of the bunch. Harvesting and parsing is very bandwidth- and processor-intensive. No problem if you have a separate machine to do this, but I have only one machine, it runs my website and my harvester, and it has to stay working without big lags.
The *really* depressing part of this is that you are clearly right… but if you can’t get the word out how can any of us mere mortals hope to?
This is a phenomenon of complexity: high density connectors are rare in networks, so their complex needs are generally only partially met (at best) by social software tools because the perceived returns is small. And of course the benefit is aimed towards growing the massive scale of casual users, which works even further against sophisticated development.
I’ve gone round and round on this kind of issue with Joshua and other del.icio.us staff regarding the never-fulfilled promise of a more complete boolean language for tags. Such an development would enable people like me to do so much more with del.icio.us resources in terms of republication and other re-use, but since most people never even use simple intersections…
To some degree we can compensate with custom programming, but that’s no substitute for robust tools (in addition to being a general brain drain from time better spent elsewhere). I’m also tired of theoretical “it can be done” statements that are used to excuse the actual implementation of solutions. I have a similar gripe about portability of data between apps– in many cases the gulf between “it can be done” and “here’s how to do it in a way regular users can replicate” is quite large.
Perhaps Stephen’s new tool will provide some assistance… any forward progress would be welcome.
Dude, re: the funding proposal rejections, clearly what is missing is acronyms. RSS is so passé, what you need is SOA – Syndication Oriented Architecture! cf. http://blog.jonudell.net/2007/09/11/a-conversation-with-rohit-khare-about-syndication-oriented-architecture/
What is stopping the agrssive code from being released as Open Source? I realize that in itself it may not have the bones to carry this off, but in the absence of some other solution (which I still believe must exist) that fulfills your need, open sourcing that may be a useful gambit for attracting some attention and effort to the cause.
@Steve – sounds cool! Will check it out for sure.
@Stephen – I look forward to seeing Edu_RSS Mark II. I appreciate your offer. And yes, I’m aware of the server strain involved with parsing — it’s a huge reason why putting in filtering was such a huge burden.
@Chris – great points all round. Now I’m even more depressed.
@Scott – dude, I used SOA and referenced that post in two proposals. I’m telling you, this stuff is death. We did release the source code for the first (working) aggRSSive, and got no takers… so it’s tough to motivate the boys to make the steps to open source the non-working version: https://eduforge.org/projects/aggrssive/
I’m sure there was more we could have done, but the peril of unsupported rogue projects is that there is never enough time.
Don’t jump in the life boats yet, just keep shuffling the deck chairs around. If there is not a stopper for that drain, just stuff a washcloth over it.
There are 2 strands here- one is the fight to keep promoting the concept of thinking about information in a networked, distributed mindset, something on a far horizon for many folks. The second, what it might take to “sell” it for a conference/paper proposal… hmm, like Scott suggests, sexier buzz words?
My experience with this over the years is that they will come around to it just about the time you give up in despair… which sounds like now?
Who knows, maybe this “blogging” thing might be big someday.
So I thought I was a pretty close reader of Abject Learning, but the URL above is the first I remember seeing of the Aggrssive source code. Maybe I missed the announcement, but I couldn’t find it by searching on ‘aggrssive’ here. Anyways… this is absolutely an important missing piece. These kind of lightweight registry/aggregation/filtering services are exactly the kind of ‘middleware’ that is needed to make the small pieces/loosely-coupled approach a reality on campus where the entrenched CMS (for better or worse) set the bar and the expectations around ‘integration.’ I know you know this. I know a lot of us know this. But we need to get it working or else windows will start to close. Let me work on it…who else is up for it?
This is also the first I recall seeing of aggRSSive. I would have at least tried it out. PHP isn’t really my bag, but I’m sure I would have learned from it.
These days, it seems, if you;re going to write PHP, you have to be writing WordPress or Drupal extensions.
Brian,
I love it when you post these “end of the world” posts, they only motivate me more! Now, I think one of the issues is that we want one think to be able to parse through everything, which in many ways is at odds with the very strategy (namely SPLJ) that we need this tool to implement.
So why not think on a more microcosmic level, and I’ll riff off something Stephen says above, why not further trick out applications that already do most of the work for us, like Drupal or WordPress. Why write an entire program from scratch and try and make everything work, chances are it won’t.
Here is an example, I can take one of two wp-plugins (WP-Autoblog)and captures categories, tags, post content, and images from RSS 2, atom, and rss .92. That covers a lot of ground, I can also choose from WP-O-Matic or BDP RSS to slice and dice feeds, make a custom OPML, and manage the various feeds more granularly. There are certainly things it can’t do yet, but with the advent of tags in WP 1.3 we can manage our own ecosystem of sitewide tags and feed classes from posts tags rather than catgeories, which we are doing now. So, using spamming plugins I have the very rudimentary beginnings of a very limited eduglu.
Point is, it ain’t eduglu, but it is an approximation that kinda works, doesn’t cause the pain and suffering of these tools that always dissapoint, and the content is always portable if you want to get it out.
I have contemplated feeding blogs like Projections by Jon Beasly-Murray and Half an Hour by Downes into UMW Blogs as class blogs that students can follow and study as a part of this environment, they would feed in fine, and we can incorporate tags (or even add tags automatically) so that they would be searchable in the blog archive. But how is that different from google? Well, it isn’t necessarily (just far less efficient) but it is a space that might bring students into contact with these thinkers in an academic micro-network that tends towards the macro, but can never fully approximate it.
The tools are there, and I think we should be writing PHP for WordPress or Drupal, and making these applications do this job for us–at least for the moment. Sure something may come along soon that solves all of this, but as your post suggests- who knows if and when that will ever happen, and by no means is anything lost. These tools do work, and approximating eduglu in a learning network is worthy and noble cause to pursue together.
No more gloom and doom, baby, we own this sucker.
I believe!
Jon,
Are you mocking me? If not, and you truly believe, I have a shiny new bumper sticker with your name on it 🙂
http://store.theonion.com/images/get/161
By the way, can I feed your Projections blog into UMW Blogs? It rules!
@Scott and Stephen – I blogged it, but not as aggRSSively as might be expected, for internal political reasons. I’ll tell you about it over beer someday if you’re interested. I’ll see if the existing code can be packaged for others to take a look.
@Jim – Maybe I’ll change my tagline to “Social learning, open education, and the occasional end of the world doom and gloom.” I have to say, the functionality of say Planet Northern Voice or UMWBlogs is as close to success as I’ve seen (though that’s in part due to the broad mandate). I’m certainly willing to look a modifying the immediate goal if we can move things ahead.
@Jon – given how much your thinking has pushed me, and how patient you’ve been, I’ve never questioned your faith.
Thanks all of you. I honestly didn’t intend this post as more than some venting (and kind of hoping somebody would point out some awesome tool I didn’t know about), but this has been an encouraging thread.
“It’s gotten to the point where our working plan is to handcode HTML tables and paste in Feed2JS javascript for each source… which just strikes me as an insanely laborious way to provide simple RSS rendering with a stable, public URL.”
That’s sort of what I did with http://ouseful.open.ac.uk/opmldashboard/ using js to handle the layout; I’m sure a bit of tweaking with YUI stuff could be used to tidy up the display a bit more.
The easiest way I’ve found of creating the OPML files is to use the drag’n’drop Grazr interface loaded into a firefox/flock sidebar (just drag feeds into it; the layout in the sidebar is a bit too wide is the major issue).
tony
Thanks Tony, I was thinking a bit of your recent experiments (your OPML dashboard is slicker than what I envisioned), and was wondering if Grazr might be useful…
Jim, no mockery at all. Thanks for the bumper sticker! here’s one for you.
And you’re welcome to do what you want with the Projections feed. Thanks for the kind words!
I managed to wheedle a small amount of funding for development, and you can see the first results at:
http://blogs.cetis.ac.uk/feedforward/
intro here:
http://www.cetis.ac.uk/members/scott/blogview?entry=20071130222727
(Its closer to RSSWritr than to EduRSS)
I get around the loading and parsing load issue using lots and lots of threads, schedulers and caching.
Scott — I’ve been following FeedForward with interest since I saw the reference on your blog. I’m excited!