Sunday, April 27, 2008

Feedparser does not work with Google App Engine

After my laughable mistake of trying to do a import feedparser.py, I sat down yesterday and spent half an hour writing my rss aggregator for Google App Enginer. Critical, of course, was use of the excellent feedparser project. It was easy to get everything working, and while not styled it looked good. Everything except for using Feedparser to parse the incoming rss and atom feeds.

Alas, Feedparser tries to use a few modules that the enterprising folks at Google restricted. I haven't done any research yet, but I wonder if it is in the arena of fetching data from URLs, since app engine has its own library. I'll poke at it tomorrow.

In any case, I was very pleased with Google App Engine. Lets go over why:
  • Database is not a RDBMS. Some people might scream at this issue, but the benefits we get are wonderful. Expando seems really fun to use.
  • Built-in ORM. Sure, its not SQLAlchemy or the Django system, but its not that far different in approach and implementation.
  • Django Templates! If I'm not doing TAL and I'm doing XHTML/XML, then my choice is Django. Templates.
  • Cleanly documented. Clear and simple sentences with good examples that are working code, not doc or CLI tests.
  • Easy and intuitive. This part is critical. The framework is not in the way.
Update: Apparently Feedparser works with GAPE. Either something changed about GAPE (feedparser hasn't been updated in a while) or maybe I had a bug. Thanks to Alex UK and crchemist for pointing this out.
at

7 comments:

  1. reedobrienMay 1, 2008 at 3:12 PM

    "Cleanly documented. Clear and simple sentences with good examples that are working code, not doc or CLI tests."

    I suppose that is another jab at zope.formlib and their doc tests on pypi.

    I will note again that if you want understand what is in a zope library you should read the interfaces. That is what they are for.


    And as far as the quoted text above doctests are by definition working code.

    ReplyDelete
  2. Silas SewellMay 4, 2008 at 4:38 PM

    Universal Feed Parser isn't supported because urllib and urllib2 aren't supported (see: http://code.google.com/p/googleappengine/issues/detail?id=61).

    ReplyDelete
  3. LiliAugust 28, 2008 at 8:29 AM

    what's the alternative to feedparser.py that would work with google appengine? anybody know?

    ReplyDelete
  4. pydannyAugust 28, 2008 at 9:16 AM

    @Lili,

    My hope is that the urllib and urllib2 will be fully supported in a future GAPE release.

    That or feedparser will be enhanced to work in GAPE via urlfetch. This might be a fun task.

    ReplyDelete
  5. Alex UKAugust 31, 2008 at 9:49 AM

    It may be too late but look at this article
    http://www.ibm.com/developerworks/opensource/library/os-eclipse-mashup-google-pt1/
    they use feedparser with GAE and it works (at least at my comp, I hope it will work on GAE server as well.

    ReplyDelete
  6. AnonymousSeptember 5, 2008 at 2:55 PM

    FeedParser works fine at appengine. I use it as
    [code]
    content = fetch(url).content
    d = feedparser.parse(StringIO(content))
    [/code]

    ReplyDelete
  7. crchemistSeptember 6, 2008 at 3:08 AM

    I currently use feedparser at http://tagsfetcher.appspot.com . For fetching tags from different sites for exmaple http://tagsfetcher.appspot.com/plone/python or http://tagsfetcher.appspot.com/zope%202

    ReplyDelete

Note: Only a member of this blog may post a comment.