Playing with Trac, SVN, Apache, and LDAP

For work I have a task of creating a script utility that lets system administrators create Projects in a mix of Trac, SVN, Apache, and LDAP. Its a bunch of work, and lets me lay with pure Python yet again. Much as I like Plone/Zope, sometimes its nice to just do this sort of thing.

Anyway, Trac already comes with a command line interface. I extend it and use it quite a bit. The problem, alas, is that the Trac CLI mixes logic with presentation. So you can't extend all of it. Also, I have lots more stuff that my system is doing.

So today I'm creating views for my script. The views are the individual methods called by the user of the script, and they exist as functions within a single class. No business logic. It means at some point we can put a website around the core functionality of the script.

Anyway, I needed to write documentation for this script. I figured I would stick it in the doc strings of the individual view methods and call it with the __doc__ method. Well, I did so, and it works like a charm. So again, in Python, the code documents itself.

Again, more reason to love Python.

:)

My human rights are violated!

The coffee machine at work is broken. No caffeine. No hot water for tea even!

I'll have to either use the soda machine or run out for coffee.

Totally unfair.

import antigravity

Looks like we have a new Python statement of reality thanks to xkcd:

http://xkcd.com/353/

Capturing shell output in Python

I need to capture the terminal text returned after I run some shell commands to create SVN repos. Unfortunately, os.system() doesn't capture the text, just the numeric value. Fortunately, Python has the command library. So now I can do this:

>>> import commands
>>> print(commands.getstatusoutput('ls -l'))
(0, 'total 0\ndrwxr-xr-x    9 dgreenfe  dgreenfe  306 Nov 28 19:02 bar\ndrwxr-xr-x   11 dgreenfe  dgreenfe  374 Nov 28 19:02 baz\ndrwxr-xr-x    9 dgreenfe  dgreenfe  306 Nov 28 19:00 foo\ndrwxr-xr-x   25 dgreenfe  dgreenfe  850 Nov 27 12:33 svn_bo11282007\ndrwxr-xr-x    9 dgreenfe  dgreenfe  306 Nov 29 10:37 tsvn1\ndrwxr-xr-x    9 dgreenfe  dgreenfe  306 Nov 29 10:46 tsvn2')

How fast can I convert a Zope 3 app to Grok?

Figured it would be quick and easy to make a conversion. I'm still new with both frameworks, but since Grok is built doing DRY and KISS and various Ruby/Django-isms, this should be pretty easy.

The application in question is a Zope 3 Captcha server that I wrote. It served as my learning box for doing Zope 3 and got me past most of the Zope 3 hump.

Complexity added to Plone 3

Mikko Ohtamaa raises some good points about the problems with adding viewlets in Plone 3. It is too complex. Design patterns are good, but they can capture your focus and make you do lots more work than you need, and not just necessarily up front.

Playing with Django

I've been playing with Django over the past few days. I'm not going to compare it to Plone or Zope. But I will happily compare it to Pylons. Ready?

Django is easier than Pylons.

There. That sums it up.

Easy, intuitive, and fun.

Happy Thanksgiving!

I'm thankful for:

• My wife and son
• My students
• Everyone who helped out when my bike got stolen
• Only two minor injuries this year
• My friends
• My family
• My crazy coworkers
• The trip to Italy
• My NASA customer
• Python
• The Plone and Zope communities
• Good training partners
• Good teachers

RIA arguments

I don't like to work in RIAs. Why reinvent the web when we have AJAX?

That said, I have a bit of respect for what Adobe has done with Flex and little respect for OpenLazlo, Silverlight, or JavaFx.

Anyway, someone I know is a huge OpenLazlo fan. I think they are mistaken to be entranced by OpenLazlo. Flex does all that OpenLazlo does, does it better, and does it for the same cost (free). Furthermore, the chart here shows just what the job market is for OpenLazlo, Flex, and other RIAs:

http://blogs.msdn.com/rohant/archive/2007/11/19/so-how-safe-is-your-bet-on-silverlight-wpf-and-net-3-0.aspx

Need to get into the habit of using Python sets

Periodically I have use cases where I want lists with unique values. And I so often write logic to do it on my own. You know, things like if 'hello' not in mylist: do x(). Well, I need to stop doing that and remember to use sets.

Adding a new user at the root of Plone/Zope 2 programmatically

Sometimes you just need to do this sort of thing.

>>> app.portal._addRole('MyRole')

What do you do when zope.Public refuses to be Public?

I'm serving out image and audio files from Zope 3 for my Captcha application. Since the images and audio files are for public consumption, I marked their components in the zcml as having the permission of zope.Public. This works fine for the image, but not for the audio component. Is there something I'm missing? For reference, this is part of my ZCML:

browser:page
name="captcha.wav"
for="captchad.interfaces.ICaptchaContainer"
class="captchad.browser.folder.CaptchaAudio"
permission="zope.Public"


browser:page
name="captcha.png"
for="captchad.interfaces.ICaptchaContainer"
class="captchad.browser.folder.CaptchaImage"
permission="zope.Public"

Update 2007/11/14: Looks like the object those things reside in need to have zope.Public declared for it as well. I'm not sure I like that approach, and I'm wondering if I'm just not 'getting' something about Zope 3 security.

How do I deploy development code with buildout?

I've got a project where I want people to run a buildout of Plone 3 that adds in the development trunk into a site, and also creates the admin account and starts up Plone. I know this can be done because p4a does it. However, I want a much simpler version of things.

If I can get this done, then my KSS tutorial is done.

Zope 3 Security

Everyone is right, its not hard to figure out. I would have to say that the docs for it are a bit wordy. So here is my brief summary of things:

• Permissions are associated with components of your applications. Zope has some built in ones but you can define new ones via ZCML and link them to components via ZCML.
• Roles are what something is supposed to be generally doing. Such as be a 'user' or 'manager'. They are defined in ZCML and are associated with Permissions and Principals via grants.
• Principals are anything that accesses something. You could call them 'users' or 'members'. They are called principals because nearly anything can serve as one. Hence why users and members is not used. Principals are associated to Roles via grants. Principals are defined in ZCML but can also be called from special components called 'Password Authentication Utilities'.
• Grants are how things are related to each other in this via grants.

And since a picture says a thousand words...

New blog format

I hate super narrow columns of text in blogs. So I changed my layout.

I'm not happy with blogger. But until I launch some sort of professional site again, I'll just stay here.

Plone lets you import/export workflows!

This is for an application that I'm hoping we'll be able to replace after this month. When I put it together it was my place to learn new stuff and make some critical mistakes. In the future workflows will be defined via code, but for now, I'm going to do it in the ZMI.

The problem was doing it in production. We are simply not allowed to touch production boxes or the ZMI there without getting into trouble. And our sys admins know nothing about Plone and Zope (and arguably *nix). So I have to automate the process.

My first thought is Selenium since hopefully the sys admins know how to use that fun tool. But using Selenium for configuration is problematic at best. The ZMI often uses dynamic form field ids per page load, or per Zope instances. This has bit me before.

I looked up doing it programmatically. I could write a script, but instructing a sys admin to run the script might be painful too. I've run into problems with this too.

Then I remembered that much of the ZMI lets you do imports/exports. A quick check and yes, we can do it with workflows.

Now the downside is that it is a slippery slope. Once we start doing things through the ZMI you can't go back. But again, this project will probably be migrated to something much better next year.

Pylons vs Django

I like the interior workings of Pylons, I really do. But if I want to do SQL based crud apps then Django is simply stronger. That strength is what drew me to TurboGears for a bit before parts of TurboGears annoyed me. Pylons just lacks a lot of the things that I want, like built-in auth . Sure, I can roll my own, but I'm lazy. Of course, Django is really meant for the CRUD sort of thing, and I wonder how good it would be beyond that avenue.

Note - I'll be expanding on this over time.

Update: My problematic comparison of Pylons vs Django.

ZPUG-DC Report 2007/11/07

Got a good presentation on repose.zope2 by Chris McConnough, which lets you deploy easily in virtualenv with just a few commands. Paste is used a lot, of course. This repose thing lets you use WSGI, which means you can use middleware of choice. Chris shared evalerrer with us, which promises to be a great middleware for any developer.

I like this sort of development because the easier it is to install independent versions of things with variations quickly and without a lot of fuss. So you can try new things, discover you don't like them, and try something else. And you don't need to know a lot to make this work.

Sure, knowing the stuff about how it works is important, but this serves as a nice tool for understanding why. It lowers the barrier for entry, which in the Zope (and Plone) world is important.

Afterwards had great discussions with ZPUG at some bar in DC. Talked to Paul Everitt, Matt Bowen, Chris McConnough, Kapil, and some others. I was very sleepy so I was not as sharp as I can be, but I learned some interesting things.

Kapil agreed with a growing suspicion I have about Grok. I'll post about that in the future. Its mostly positive but it does have a couple critiques.

Kapil reinforced my feelings about Pylons vs Django. Another item for a future post.

Head meets desk

I need to add a DateTime field to my primary content object. Unfortunately, in Zope 3, they named it Datetime in zope.schema. Now, before you say I should notice the coding standards, please note that the single textline object is zope.schema.TextLine.

Figuring out this one took me way too long for comfort.

This is the first time I've seen this sort of mistake in Zope 3. Until now I've been delighted with Zope 3's consistency and elegance.

I'm hoping it will be a while before I find the same sort of thing again.

I grok Zope 3!

I've struggled with Zope 3 for a bit. I read books, perused documentation, and took a class in Italy. And yet, it felt confusing. Well, this week I had to finish up a Zope 3 solution in record time. So I really hit the books and squinted my eyes real hard. It was frustrating because I got the individual bits, but putting them together was painful and a matter of trial and error rather than an understanding of how to meld components together.

The long and the short is that I used a mix of using Grok and writing Zope 3 tests to figure out problems I was having in my efforts. And then after about two days of struggling and banging my head against the wall, something clicked.

I get Zope 3.

I made something not working work in minutes.

I'm no expert. I've got a long way to go. I've got to work more with Adapters. But tying pieces together is no longer something scary. It makes sense now.

So, here are my lessons learned for anyone who wants to do Zope 3 and is a newbie..

1. Start with Grok. It removes a lot of the overhead and tells you the core bits of how Zope 3 components and interfaces work.

2. Write Zope 3 unit (doc) tests. I can't understate this issue. Don't worry about functional tests. The nice thing is that they insure that before you go to trying to put pieces together, that your component level bugs are mostly gone, so you don't think that zcml is a monster when really its a bug you didn't spot.

What is a real language?

I remember back in my Java days I thought it was cool I was doing a 'real language'. That meant it used semi-colons to close statements, had static typing, was compiled, and could do pretty much anything. Unfortunately, I was also unhappy with hunting down semi-colons, found static typing to be annoying, and found the compile phase to be dull.

I drifted back into scripting languages then, and was ashamed for a bit. I mean, I can't tell you how many times I heard this phrase: 'Language x is a scripting language' as 'Language x is just a scripting language'.

Of course, I could throw back that I was much more productive in a scripting language, or that my code was smaller, meaner, and easier to read. But deep inside I felt guilty that the scripting languages I was using weren't 'real languages', but rather just scripting languages. It was a classic demonstration of how emotion can triumph reason.

It was frustrating because there were things I am easily doing in Python that are really hard in Java. Ever put together a quickie script to read a text file and then use that data to generate critical responses? Or write a hundred views in a month? Doing either in Java is painful, but in Python its a snap. So my gut and instinct told me that I was right to back my intellect, but sometimes I felt like I had stepped backwards.

It look me a long time to get over that self-inflicted stigmata, but I think its gone now. Why?

Because I suppose time heals all wounds.

Handy code snippet

[div content="structure provider:plone.leftcolumn"]

KSS or Plone4Artists sprint choice

I'm interested in both. I think I might do better at the KSS sprint because they really need a good set of tutorials and documentation. But Plone4Artists would be good for me to do because I would get stronger on Interfaces and Subtypes, however, my ability to do them some good is limited by the fact that the HTML/CSS trick I want to do I haven't figured out how to do yet.

Hrm...

After some thought, I'm going to do the KSS sprint until I get to a point, then switch over. I'll be able to take the skills from one sprint and bring it into the other group.

Technorati Fun

Technorati Profile

Top 5 Design Tips for Skinning Plone

=================
Tip 5 - Centered Design
=================
- Fixed width
#visual-portal-wrapper {
width: 980px;
margin-left:auto;
margin-right:auto;
}

- Liquid Design
#visual-portal-wrapper {
width: 980px;
margin-left:20%;
margin-right:20%;
}

=================
Tip 4 - integration of IE
=================
- put all stuff in IEFixes.css
- Write styles for IE7 first
- Then hack your styles for other IE version
- portal top is top part of Plone
- For IE6 and lower hack:
// Below is IE7 , Firefox, and Safari
#portal-top {
background: blue;

}
// below is IE6 version
html #portal-top {
background:red;
}
- easy to use
- almost no chance of breakage
=================
Tip 3 - Styles alterations
=================
- different sections styling
- site root
- news: blue
- products:orange
- events: yellow;
- about:green
how:
body.section-news {
background-color: blue;
}
body.section-products {
background-color: orange;
}
// This is auto-written and any_custom_view could be working_group.pt
body.template-any_custom_view {
background-color: spotted duck;
}
// example
body.template-working_group_view.pt {
background-color: spotted duck;
}
=================
Tip 2 - Drop down menus
=================
- Suckerfish for Plone 2.5.x
- Accessible
- valid CSS
- Obvious and clean XHTML
- Plone Dropdowns is for Plone 3.x
- webcouturier.dropdownmenu
- Build it out in folder items and use a heirarchy
- Autobuilds via Plone 3 into portal tabs
- Uses INavtreeStrategy
- Uses SitemapQueryBuilder()
- Can change the depth of the navtree via sitemap properties

=================
Tip 1 - Rounded Corners
=================
- Cornerstone of designer's minds
- pure CSS solution
- Initial nifty corners
- Too ugly XHTML
- No hooks in Plone
- People don't like dealing with CSS if they have to change images
- Images based solutions
- Sliding doors - often used for rounding corners
- Adam Kalsey technique
- Plone has XHTML hooks in portlets for this
- pretty simple css
- Most of the cases use nested HTML elements
- Fixed set of images for the corners
- JS + CSS solution
- The most flexible
- Doesn't require nested elements in HTML
- Does not require additional CSS
- Potential Solutions
- Nifty Corners Cube (Javascript Library
- First doesn not work with borders and background images
- JQuery corners
- Requires jquery and does not work with Safari
- CurvyCorners library (recommended)
- Supports most modern browsers
- Works with borders
- Works with background images
- Supports antialiased corners
- Cons:
- Some problems when background images are used and box has different colors
- Does not work well when used with multiple boxes
- collective.roundedcorners
- On presenter's laptop
- Normal Plone Package/Product
- Uses a mix of Javascript + CSS
- Raw, and will be released hopefully soon

Required methods to make a class iterable

Really useful:

#required iterable elements
class MyIterator(object):

    def __iter__(self):
        return self.data.__iter__()

    def __len__(self):
        return len(self.data)

    def __contains__(self, v):
        return v in self.data

    def __getitem__(self, v):
        return self.data[v]

Travel Blog!

Here it is for those who want it:

http://dannygreenfeld.blogspot.com/

Case sensitive search in Zope 2.9.7

I'm not 100% happy with this function, and I'm wondering if I'm doing too much work. Especially waking up the object when the word object is not found in the object.Description attribute. Is there a better way?

def getWords(word):
   pc = app.msrd.portal_catalog
   results = []
   for brain in pc(SearchableText=word):
       if word in brain.Description:
           results.append(brain.getPath())
           continue
       try:
           content = brain.getObject()
           for field in content.schema.fields():
               name = field.__dict__['__name__']
               if 'body' in name or 'Body' in name:
                   accessor = field.__dict__['accessor']
                   text = content[accessor]()
                   if word in text:
                       results.append(content.absolute_url())
                       continue
       except:
           continue
   return results

Interesting...

http://www.gamearchitect.net/Articles/SoftwareIsHard.html

Choice quote that worries me about counting lines of code:

"Speaking of lines of code, my friend Rebecca, who's a Pentagon reporter, has sat through countless briefings about military software projects. She's bemused at the briefer's fondness for citing the number of lines of code in the systems they're developing."

Lean, mean code is better than big, bloated code. And using lines of code as a metric is something I thought went out of vogue a long time ago.

Here is another interesting bit:

Let's look back at the F-22. Why did it take dozens of software engineers working on the F-22 avionics code four times as long to produce 750,000 lines of code as it's taken the Fracture team? Is the average programmer at Day 1 Studios a hundred times as productive as the average programmer at Boeing? Of course not. The F-22's avionics suite is life-critical software. It has strong interdependencies on changing hardware.

Life critical software is hard. Toss in the extra work developers have to do in a CMM-5 shop and things start to bulk up. Everything has to be explained and checked. On the other hand, often your code ends up looking pretty good since you get really smart people giving you peer reviews.

The closer thought is this line:

The hardware it runs on is nearly twenty years old now, which increases iteration times (the F-22 runs on a VAX/VMS system like the college mainframe when I started at Davidson, back in 1990)

I'm not sure I'm happy or not at this statement. The F-22 costs us about $450 million dollars a plane so I'm happy that they've got super mature systems in place. On the other hand, wouldn't it be nice to see five hardened Linux pizza boxes in the place of the VAX mainframe?

Better yet, lets ditch this series of fighter and just go to UAVs. Cheaper, better performance, and no pilot risk.

I'm so glad I don't work for DoD anymore.

Can I be a bit snarky here?

So I've worked professionally with a number of different languages now. Let me see: Foxpro, Perl, PHP, ColdFusion, Java, T-SQL, PL-SQL, Javascript, and lastly Python

And which language is getting me sent to Italy and California?

Python.

Sure, I can be all geeky about how Python's dynamic typing, multi-paradigm, scaling, and low/high level capabilities makes it uber-powerful.

But who cares.

I'm going to be eating Pizza in Naples.

Python wins hands down.

Supersizing YouTube

I hate the format of the site but the information is critically useful.

http://www.scribd.com/doc/244443/Supersising-YouTube-with-Python

Python Tips ahoy!

On the eve of Talk Like a Pirate Day, I am announcing a new personal project. A little site I'm going to call pytips, Python Tips, Yellow Monkey Land, or maybe something else. Its going to serve up recipes and tips for Python, and eventually other languages as well.

So why am I doing this?

Four reasons:

1. I think the Python community could be well served by such a site.
2. I want to practice in another framework besides Zope and Plone.
3. I need to keep my relational database skills from getting rusty.
4. Sounds like fun!

I'm going to do this in Pylons. I thought about Django, and maybe I'll make a port of it at some point. Shouldn't be too hard, right?

Lambdas no more

When I was first learning Python I loved it all. It was a breath of fresh air. So easy, so powerful, and so uplifting. Python solved so many problems on a project I was doing that we switched from Java to Python because work I anticipated would take me 45 days in Java I managed to do in 90 minutes of Python. And this was my first real Python effort!

So everything was great, right?

Wrong.

Lambdas just plain escaped me. Experience in Java, Perl, PHP, Foxpro and ColdFusion left me unprepared for them. Took me ages to grok it. Finally after several weeks of tooling around with them, it clicked one marvelous day.

For a while I had happy fun with Lambdas. They were so cute. I could minimize my code a bit and could hang with awesome Pythonistas and share quips with Lisp programmers. Not that I knew any Lisp programmers, but if I did know them then I could talk about Lambdas too.

However, I quickly found myself resorting back to the 'standard' Python function/method way of doing things. In a nutshell, I found it much easier to read my code. Comments were much more clear. And the time it took to write out a couple extra lines of code wasn't all that bad.

Look, I'm really lazy, and I hate writing extra lines of code. Yet I think in the long run avoiding lambdas is worth it. Sometimes concise != good.

TurboGears merges with Pylons

I'm not an expert on either. In 2006 I spent about 4 or 5 weeks working with Pylons. A couple months later, I spent a couple weeks playing with TurboGears. Both were very interesting. I really tried to like Pylons but hated the Myghty which was what you had to play with via views. TurboGears back end wasn't as much fun as Pylons, but the front end used Kid as the template language for views.

That said, both frameworks really embodied things I like. They employed NIH and DRY all over the place with a vengeance. Sure, Django is a bit 'hip', but nix the scaffolding and all you have is Yet-Another-*SP-Template-Language combined with non-Pythonisms in a not-quite-RoR framework.

Well, once the Pylons and TurboGears marriage happens, Python's choice of quality frameworks will be down by one, but in a good way. Pylons will do the backend and you can load in TurboGears to be the front end. And with use of Genshi as the template attribute language. Yeah, Genshi isn't TAL, but it isn't that far off.

Links of interest:
http://compoundthinking.com/blog/index.php/2007/06/27/turbogears-11-and-beyond/
http://www.blueskyonmars.com/2007/06/27/turbogears-2-a-reinvention-and-back-to-its-roots/

Psuedo Code to Real Code

We always say it about Python, that our hand scratch code often becomes real code. That what other people sue to map out their thought processes is what we code in.

Well, over last weekend I was coding away on a personal project and typed in 150 lines of code. When I started to debug, I found I was done.

Not bad.

Playing with Catalog brains

So after a month of doing a lot of work with Plone content type modules, references, relations, catalog searches and teaching graphic designers basic TAL concepts, I've picked up some interesting habits.

First off, brains are better than tuples. And unlike in my previous post
, I now now that to get a full object off a brain you can just do something like this: [div repeat="item itemBrain/getObject"][/div].

I also ran into problems with objects that are related to objects with are related to objects which are related to objects. How do you find them and make sure you don't have duplicates and all of them are published? Well, I wrote a method to handle that. You run it on the context and give a list of the necessary default accessors. Maybe the Relation product has something for it already, and people will make fun of me. Or maybe not...

Anyway, Plone isn't just fun, its productive. I can't wait for our project to hit the real world.

A thought on methods in Plone

Note: Because of the limitations of blogger, I am replacing the '<' and '>' in XHTML statements with '[' and ']'.

We just did a sprint on our big project at work. My first step was to create a bunch of methods in the content types that would traverse between items via the reference catalog (yes Virginia, we are using the Relations product along with Association Classes get ArchGenXML).

Our first thought was to return a tuple of the title, description, and url of the referred item. Well, that turned out to be annoying because:

[div content="python:item[1]"]

is not as much fun as:

[div content="item/description"]

Also, for the graphic designers, having them start doing 'python:' everywhere just seemed like fighting a battle we didn't need to fight.

Then we decided to return a dictionary containing title, url, and description. And in some cases, acronym, image, and more. Now we have:

[div content="item/description"]

Hooray! Except that there are about 10+ and growing examples now of times when we missed something that needed to be returned.

So it hit me, why not just return the darn catalog brain? Sure, reference_catalog doesn't return a usable URL (you have to strip off the /at_reference/md5_hash), but I've already written an often reused python script that does this called cleanUrl. If we went this method, we get:

[div content="itemBrain/description"]

And if we hit another snag where we need to also return field x, y, and z, we have them do this:

[div define="item python:itemBrain.getObject()" content="item/funnyQuote"]

Plone Introspection

So you have a Plone content type with some fields. What are the properties of that content type? Well, you can look it up by finding in your file system the product that content type comes from, then examining the schema and finding out your information. Of course, if the product extends another product, then you have to look up that other product.

Bleah.

Introspection is what we really want. Python has it in spades. On pretty much any object in Python I can wrap it in dir() or help(), or check the __dict__ attribute. But you can't do this easily in Plone. Cooking up a universal instrospection view is something that ought to be done for my CMS of choice. I know there are lots of various barely documented utility methods I can find. I can wrap it up in a product and just use it by having a special introspection view that lays out some pretty HTML.

Sigh.

My coworker Reed pointed out that Zope 3, and the Five stuff in Zope 2 uses Interfaces that give some neat tricks you can do for introspection. I think I'll give that method a try. I think thats the right way to go because:

1. The Zope 3 specification handles introspection much more gracefully than Zope 2. And the methods for doing it seem better documented.
2. The technology is going that way anyhow.
   

Image of the Day 'extension' of FeedFeeder experience

Introduction

For a huge work project I was asked to extend FeedFeeder so we could have an image of the day type. I like FeedFeeder and this would give me the chance to speed up on Five and Zope 3. I was so excited! Five turned out to be lots of fun and not that hard to learn. Zope 3 Products looked like tons of fun, sort of like Plone products on steroids or J2EE done right.

And yet the effort turned into an exercise of frustration. Why?

The thing I like about Plone (and now Five / Zope 3 (Z3)) is that once you grok something, it is really easy to make new content types or extend new ones. You can do that via UML using ArchGenXML, or simply code it out yourself. I really enjoy this part of plone, being able to handle new content types so easily.

Things go downhill

Alas, FeedFeeder is a weird amalgam of ArchGenXML output and Five technology. ArchGenXML generated a lot of the boilerplate, but then the authors went and did things in all sorts of weird places that violate what you are supposed to do with ArchGenXML output. It is very obvious that FeedFeeder is the designer's method of learning and Five and component based design. Which is nice, but what you get is something that is very hard to extend and control. By combining ArchGenXML and Zope 3 and Five the way they did, it is actually more work to make it do what we want.


Normally adding a new content type in Plone / Five / Zope 3 / ArchGenXML, you just extend/implement an existing content type and modify either a ZCML (Z3, Five) or Install.py (Plone) or AppInstall (ArchGenXML) and maybe stick in an adapter (Z3, Five). Well, not so much in FeedFeeder, where to add an extended/new content type you have to modify the following: configure.zcml, install.py, and pretty much rebuild the content type from scratch. Then add in views and skins. Then pray it works. Just getting a simple extension without any modification to work was looking to be way, way to much work.

After working on it for hours yesterday at CC, and an hour last night, and during the wee hours between 3 and 5 am when I thought I had an epiphany, I was getting really frustrated. This wasn't like association classes where I knew that if I could just find the right bits in the lackluster docs it would work, I really felt like I was walking through someone's spaghetti code.

The Recovery

So I start thinking about building my own product from scratch to do the work. This annoyed me. I hate reinventing the wheel. Oh well, time to take notes on what it was doing. And I discovered right away that Feedfeeder curiously had something called enclosures. I researched it out, and you know what? Enclosures where a way to include content in Atom and RSS feeds.

I tested it out. I created a set of sample Atom and RSS feeds with enclosures with images. Then I created a FeedFeeder folder called 'iotd feed' to grab them. Then I check, and FeedFeeder grabbed the images and stored them as enclosure objects!

POW!!! Done!!! Feedfeeder does it already! All we need to do now is create a smart folder or view that looks for the 'iotd feed' folder to supply image of the day views!

Notes & Lessons learned

• I'm still not happy with FeedFeeder's internal architecture. But since out of the box it does everything we need and probably more, do we need to care?
• When I hit a brick wall like this I need to do more research. Especially when it comes to something that is using a standard. For example, I need to get a 100% understanding of something before I start trying to invent something new.
• Time Summary:
• Time spent learning Five: 1 hour
• Time spent playing with Zope 3: 2 hours
• Time spent trying to extend FeedFeeder gracefully: 4 hours
• Time spent extending FeedFeeder via gruesome hacking: 2 Hours
• Time spent examining FeedFeeder for things needed in a new product: 15 minutes
• Time spent researching Atom/RSS for how they handle enclosures: 5 minutes
• Time spent testing out how FeedFeeder handled enclosures: 15 minutes
• Total Time: 9 hours and 35 minutes
• Lesson Learned: Do your @#$%ing research before you commence work!

Interfaces in Zope 3 and Five

We use Plone a lot on the job. And Plone is leaning towards Zope 3 these days, which means Interfaces (thanks to Zope 2's inclusion of Five)! Until now I've not had a reason to really poke at Interfaces, because Python, unlike Java, doesn't really need them. Also, the Plone work I've done so far has been via UML, external methods, views (ZPT), and fancy install scripts.

However, now I'm working with feedfeeder, a Plone ATOM/RSS handler and we need to extend it to include an image-of-the-day content type. And feedfeeder is built with a lot of Interfaces and Five technology.

Alas, I don't have a Zope 3 book handy here at work.

Until my copy comes in, I'm using the Zope 3 tutorial created by the Zope 3 book author. Good stuff indeed. I'll post my thoughts when I'm done.

Beautiful Soup is Beautiful

I have a bunch of content stored on an old instance of pmwiki. I've never liked pmwiki, since it seems to only have a half-hacked state method, and just in general feels insecure. Also, I've found that wikis can be useful, but if you have short content on each page, often a FAQ style treatment will do better than a regular wiki.

So I decided to convert the pmwiki pages into a pbwiki toc construct. It would put all the content onto one page, and use the tag to provide a top level table of contents. That meant I would have to:

1. Scrape the pmwiki content index for all the meaningful links.
2. Scrape out the title and urls of each link.
3. Grab the content from each link.
4. Reformat it all to work in the pbwiki format.

I've done screen scraping before, but not in Python, and not in this scope of effort. Well, Python seems to do everything well so I opened up htmllib and started to play, thinking I would be done by brunch-time.

Immediately I'm unhappy with htmllib. The docs suck. And it just seems awkward to use once I figure it out. Doesn't feel Pythonic, although I'm sure I'm wrong in that respect somehow. Its just for me, my Python pseudo code often ends up being close to the end effort. And this was not the case.

Then a work buddy told me about Beautiful Soup. Its an HTML/XML parser that is real easy to use and can work with badly formed HTML, like the sort that pmwiki sometimes generates. Its not optimized for speed, but for usability. Thats fine with me, because this is a one-time operation on maybe 150-200 entries.

The final effort worked real nice. Not super fast, but real easy to code. Beautiful Soup meant what I thought would be a quick and simple task remained so.

The Dark Side begins its seduction...

...but I don't think many will convert.

I'm talking about the Python community embracing the Microsoft Silverlight Dynamic Language Runtime (DLR). This lets us write Iron Python for the browser, in the same way that Action Script is used in Flash, or Java in Applets.

It just seems odd. Plus, Iron Python, for all its virtues, has to do some Microsoftisms in order to work. This means that unlike moving my code wrote using cpython on Windows to Mac OS X to FreeBSD to UNIX to Linux, I have to worry about when I move from cpython to Iron Python. So this means we will have a big code set (Iron Python) in DLR that won't work anywhere else.

Bleah.

Here is to hoping that Silverlight does not take off well.

JSON and Python

I hate XML.
I love JSON.

Now that I've made that clear, lets go into how I want Python to handle my JSON.

1. I want a built-in function to JSONify.
2. I want a built-in function to deJSONify.

Now in the Python world there are multiple JSON libraries. cJSON, simplejson, demjson and more. I've used a few of these and they all seem good. However, most have extraneous methods that I don't care about, or name their json handling methods funny. For example, simplejson has you do loads() and dumps() for loading and dumping of objects/strings, and load() and dump() for loading and dumping of files. Kind of nice, but all I really want is a JSONify and a deJSONify function from my JSON handler. I can do the rest!

So what I've thought about doing is writing a Python JSON package that would do the following:

1. Load a JSON handling package. It would have a list to select from so that if simplejson was not already on your machine, it might check for cjson, demjson, etc and grab the alternate instead.
2. Have a JSONify function that would convert Python objects to JSON.
3. Have a deJSONify function to convert JSON into Python objects.

This looks like a fun and handy little project.