NOVA-DUG April 29th summary

For the second meeting there were four attendees besides myself. Again it rained, three people from last week were out of town, and my co-leader was at home tending to an injured spouse, so that knocked down our numbers. Yet I'm fine with that, as I realize it can take time for these groups to grow. Attendees (first name only for now - I will update if attendees contact me and request their names/jobs):

• Andrew
  
• Cory
  
• Matt
  
• Ross
• Danny (me!)

We went over the reasons behind founding the group and had everyone meet each other. Then we discussed the DC Branch of the Euro DjangoCon Sprint. I've summarized it here at my DC branch of the Euro DjangoCon Sprint Presentation. One thing I tried to make clear about our upcoming local area sprint is that we want it to be very inclusive. If you go, we'll try to give you as much guidance as you need. :)

After that I presented my first draft of my testing in Django presentation. Yeah, I've gone a bit mad with Google Presentation recently. Some day I wouldn't mind making this into an actual conference presentation, but it needs more work and experience from yours truly. The guys asked some good questions, and made some excellent observations that will go into future changes of this effort.

Then we went over in detail various bits that individuals are working on. I think Cory might be a good resource for Django scaling issues, and is also into security. Matt is working on a custom CMS tool for his job that sounds pretty interesting. Ross is the guy behind DC Tech Events. I got asked questions about Pinax.

There was more. Lots more. I probably need to talk less and let things go of their own accord. :P

NOVA-DUG meetup for Wed, April 29th

We are planning to get together on Wednesday, April 29th, 2009, at 7:30pm EST most likely at the Ballston Commons mall in Arlington, VA. We know this is the day right before Django District, but since Euro DjangoCon is the next week, and we want 'host' a DC area sprint during the Euro DjangoCon sprints on May 7th and May 8th, we felt it was important to get another meeting off the ground.

Location

Ballston Commons Mall Panera (ignore google maps and just go into the front entrance of the Mall)

Agenda

• Meet and greet everyone
• Present the rules of NOVA-DUG meetings
• Discuss DC Area branch of the Euro DjangoCon sprints (May 7th -8th)
• Testing in Django presentation

Sweet spots of Plone

Yes, I am posting this with Django and Pinax tags because I think my Django and Pinax friends can learn from the lessons of Plone.

I'm admittedly more interested in doing Django these days yet I keep myself firmly aware of the things for which I think Plone (and its components and relatives) is unsurpassed in the Python framework world. Arguing these things to Djangonauts is interesting, because so many times they just reject things out of hand (just like certain Zope zealots I know). And I think that is to their loss. When you hit Plone's sweet spot, things get interesting.

Standards
I've ranted about this before. In essence Plone follows a subset of the Dublin core. Any database object in Plone has certain fields you can rely on in searches, views, and business logic. You don't have to introspect the objects to find these stock fields. Until I went back to entirely developer defined models with Django, I forgot just how much I had taken this for granted.

Workflow
Years ago some madman integrated DCWorkFlow into Plone. You can build custom workflows either in script or via the UI. The scripts and interface to use it has been unpleasant until recently, but then name a workflow engine capable of handling complicated workflows that is fun/easy to use? Thanks to Martin Aspelli his product called collective.wtf, Plone workflow management has become much, much easier. As far as I know, nothing in Django (or TurboGears or Rails) compares except maybe GoFlow, and I don't speak French.

Object Oriented Database
I like the Django ORM. I like SqlAlchemy. Until I run into the edges of the fact that they are modeling table records as objects. Object Relational Impedance Mismatch anyone? Ouch the pain! I do recognize that relational databases are very useful and powerful, but sometimes I wish things were different. One day, when I'm indepedantly wealthy and own my own private tropical island I plan to get the Django ORM running with the ZODB.

ACLs
Groups and Users permissions in Plone are implemented really well. Object oriented databases handle hierarchies and references very well, so the whole parent-child ownership thing is done easily. This is critical in CMS work - ownership and control. Other frameworks have this built-in, but it is not automatically attached to each content object that you create. You get very used to it being magically done for you in Plone.

Finger method of judging graphic design

Katie Cunningham mentioned this in her blog post today. Since I was asked what that was meant by several people I decided to write it up so you can see one of my rants.

The method is simple:

• Put your design on a screen.
• Your hand goes on the screen with fingers horizontal. Ignore the thumb.
  
• Count how many fingers it takes from the top of the screen to content. If you run out of fingers on one hand that means your wonderful design is going to force users to scroll to content. Maybe not on your huge desktop monitor, but on the sort of desktop monitor or laptop that has become ubiquitous, absolutely!
• I have tiny hands for a guy. So for you people with big hands, try it with two or three fingers.
  

Now lets take a look at how the Python community matches up to my finger method.

• http://python.org is just one of my fingers! Python.org FTW!
• http://djangoproject.org is two of my fingers! Good job Django!
• http://plone.org has a new and improved site that is two to three of my fingers, depending on the location. Plone does good!
• http://www.turbogears.org jumps between three and four fingers. Acceptable.
• http://code.google.com is two and a half fingers to content on project pages. Definately workable!
• https://github.com gets it with three fingers.
• http://pinaxproject.com is over four fingers to get to content? Boo!
  
• http://code.pinaxproject.com makes up for the parent site with two fingers to get to content.

So the Python community does pretty well on average. So does Facebook, Twitter, and the other good social networking sites. Now think about the sites that never went anywhere and how my finger method applies.

How about some some abject failures from outside the python community?

• CSS Zen Garden is lovely. And yet the designs are often unworkable to anyone outside of a group of graphic designers looking to hand each other design awards. You get designs that take ten (10) fingers before you get to content. WTF?
• http://www.apple.com/mac fails at over four fingers.
  
• http://www.microsoft.com/windows/help-how-to.aspx goes over four fingers and includes my thumb.
• http://www.adobe.com is as bad as the stuff on CSS Zen Garden.
  

Is this scientific? Heck no. Yet it is a quick and cheap gauge as to why you rarely get repeat visits to your site or why they only visit your download pages.

What I learned at Pycon 2009

Why write a repetition about all the things that everyone else has documented the greatness of Pycon 2009 when I can write about the awesome stuff I learned?

• How to use Python via XLRD and XLWT to handle truly brobdingnagian Excel files (8 million records anyone) without anything Microsoft.
• New Internet scraping tools like html5lib and mechanize to reinforce lxml and the fading but still lovable BeautifulSoup.
• Many awesome things about the Sphinx documentation experience.
  
• That spending a day in test focused sessions is an intense experience.
• argparse makes writing command line python interactions so much easier.
• Good rules of thumb for breaking apart applications in not just Django, but for python modules in general.
  
• Git! The Pinax community embraced Git just as I was started to work on it. I owe a lot to Brian Rosner's patient coaching, and Jannis Leidel's patience.
• That I am addicted to checking on the Github graphs to see how I compare.
• 15 minutes of sitting next to James Tauber gave me the grounding I wanted in JQuery event handling.
• That people who hate XML based template languages HATE them. HATE HATE HATE.
• Mashed potatoes and bacon pizza is yummy. And spinach and anchovy pizza rocks. Yes, I ate anchovies and oddly enough liked them.
• I really want to teach python, Django, and Pinax. I may not be the best coder by far, but I think this could be the largest contribution I ever make to this community.
  

Some of what I need to follow up on

• GeoDjango
• sql-python
• content-mirror

Did I miss anything? Let me know!

NOVA-DUG April 15th summary

Forour first meeting there was seven people and several cancellations. Not bad for short notice and rain. Attendees (first name only for now - I will update if attendees contact me and request their names/jobs):

• Eric (possibly providing meeting space!)
  
• Sal
  
• Jack
• James
• Cory
• Katie
  
• Danny (me!)

There was talk of Django projects still under development, with one public effort exposed by Sal:

• https://www.concentrateme.com Django powered search analytics tool.

We explained Pinax to Sal. In return South was mentioned by Sal as a functional Django migration tool (thanks to everyone who pointed me at the right place). Corey talked about the limitations of the Django ORM compared to SQL Alchemy. I think we agreed to disagree about templates. Everyone griped about the quality of the Panera coffee shop's wireless network.

Upcoming planned events:

• EuroDjangoCon US-DC area sprint (May 7-8)
• Django Dash get-together (May 30-31)

We chatted about a bi-weekly schedule. We may have to do some juggling to avoid falling on the same weeks as ZPUG-DC or Django District.

Future format ideas include user presentations, voting on the open source django presentation of the week, and planned attacks on user problems.

Show me your open source Django blog application

Want your blog engine to be used by NASA?

Unlike everyone else in the Django world, I have not written a blog application.

Instead I want to use your blog application. Definitely for my upcoming blog transfer to my own personal site (Blogger's limitations annoy me), and possibly for use in NASA Science Mission Directorate Spacebook project. So what am I looking for in your blog?

In no particular order these are the must-haves:

• Elegant user interface.
  
• Follows Django/Python best practices.
• Easy to integrate into another application (which should be the case if you followed the above point).
  
• Code highlighting via pygments.
• Relies on JQuery for JavaScript, and degrades properly.
• Publishes legal RSS feeds.
• Allows for use of several input formats (Restructured Text, Markdown, etc)
  
• Hooks for integrating WYSIWYG editor
• Allows for multiple users each with their own blog.
• Renders humanely in FF, Safari, and IE 6, 7, and 8.
• Any sort of decent documentation.

In no particular order these are the nice-to-haves:

• Publishes ATOM feeds.
• Allows for multiple users on a particular blog.
• Already has a WYSIWYG editor.
• Handy import/export functions that follow whatever standards Blogger might have.

Candidate killers:

• I have my own server space. Plus, NASA has its own servers. So Google App Engine compliant blog systems need to also support the standard Django ORM.
• I am doing this in Django/Pinax/Python/PostGreSql on Linux. Systems that do not play well there need not apply.
  

What do you get out of this if I pick your blog engine?

Well, as much as I am a fan of Pinax, the default blog application doesn't do everything we want it to do for Spacebook. So your application might become the blog engine used by us. And when we launch, we'll be sharing credit with anyone who contributed from the open source community to our efforts.

Edit on August 26th, 2010: I solved how to do this research by co-authoring Django Packages which gives us this handy reference. Also, at this point in time, as part of larger systems, I've written several blog systems for clients.

NOVA-DUG meeting on Wednesday, April 15th, at 7:30 pm

The Northern Virginia Django Users Group NOVA-DUG plans to get together for the first time on Wednesday, April 15th, 2009, at 7:30 PM somewhere in Arlington, Virginia. Everyone is invited, although my bet is that only people from the Washington, DC metropolitan area will show up.

Location?

We are working on that. If we can't find a decent sized meeting room to host us, we will likely converge at Panera Bread in Clarendon. It is close to the Clarendon orange line metro stop, has eatable food for dinner, and has free wireless that works with every operating system AFAIK. If a lot of people show up, we may not be able to all sit together, but we'll make sure that everyone gets to meet everyone else.

http://maps.google.com/maps?oe=utf-8&client=firefox-a&ie=UTF8&q=panera&near=Arlington,+VA&fb=1&split=1&gl=us&view=text&ei=v8vcSZH7NIjaNveM8OEK&cd=1&hl=en&sll=38.879623,-77.110897&sspn=0.006295,0.016628&latlng=38879623,-77110897,15003625430679650664&sig2=echk_5e81Ly5FHLc2OaNhA&dtab=0&oi=&sa=X

Why the rush?

Because the inertia is there.

Because we (myself, Katie Cunningham) have waited across languages for years for a user group to meet on a night and location that works for us.

See you there!

The end of my Feedfeeder story

Another post about Plone... but this time about me and not about Plone.

For about 18 months I have wrestled with consuming broken RSS feeds to pick up image of the day fields stipulated by customers. These are feeds so broken that no RSS parser, including the masterful Feedparser, can handle them (for example, one image of the day feed usually puts the image in the RSS header and changes that each day - no history is maintained). They aren't actually RSS, they just possess a file name that ends with '.rss'. Plus, periodically the way they are written changes so custom logic fails.

I have forked Reinout van Rees FeedFeeder project, and even proposed complicated logical revisions to handle broken these broken feeds and their shifting implementation. I called it Feedfeeder v2. Reinout always seemed hesitant, and I watched as other people extended on his work and despaired. I knew something was wrong but couldn't put my finger on it. I hesitated to work on it, even though funding for it was readily available.

Then between Spacebook, Pinax, and other efforts I shelved this effort for months, hiding my head in the virtual sand. And yet I knew it needs to be addressed. How could I handle something that broke the otherwise wonderful Feedparser?

During Pycon 2009 I came up with the answer. I took an excellent tutorial on html scraping and learned lots of little tricks to reinforce my skills with BeautifulSoup. You see, screen scraping is a secret pleasure I have. Scraping out a bit of data from a page is like a little puzzle. When I talked about this to someone, in the middle of my discussion with them the answer became clear as day.

The answer was to turn the problem from a RSS interpretation problem to a simple web page scraping puzzle.

1. Fetch via urllib the XML file that pretends to be RSS.
2. Parse it using BeautifulSoup or html5lib.
3. Get all the images listed.
4. Discard all but the largest image.
5. Guess out the meta-data from the XML file and store that for the image.

Problem solved.

Now I just need to make a Plone 3 package to do this for me and my angst is finished.

My apologies Reinout for the time spent on trying to cook a solution via Feedfeeder. Thank you for your insights and your extreme patience. I think you tried to tell me to take a different path.

What I would change in Plone: Generic Setup

At the Washington, DC 2008 Plone Conference I took Joel Burton's awesome two day tutorial on Plone Theming. I don't do design myself, but I have implemented them from time to time. My CSS skills are weak, but Plone gives us lots out of the box so I figured this would be a great way to ramp up on skinning. Also, from a work perspective it made sense, because many Plonistas seem to care more about the engineering/database side of things rather than on the front end.

Joel's class, to put it succinctly, was magnificent. If you want to learn Plone, take one of his classes. It is well worth the money.

We learned about reorganizing viewlets on the fly, making css changes, tab controls, content well manipulation and lots, lots more. Two days of knowledge that didn't just go in one ear and out the other, but really stuck.

I was one of the tech-heavy guys in the room, and the rest were graphic designers. At first the ZMI terrified them, and the two different types of views did not go over so easily. But thanks to Joel, by the end of the two days the designers were really enjoying what they could do with Plone. Everyone felt confident and ready to move forward. The positive energy in the room was energizing!

Then it came time to 'save' the site design we had been working on. In other words, export it so it could be reused. Or sold even! We fired off Generic Setup to export things, and then...

The positive energy diminished

I'm capable of searching through Generic Setup XML files to figure out what is needed in order to capture a Plone skin. I'm sure anyone bothering to read this blog entry is the same way. Yet graphic designer skills tend to end at Photoshop, sometimes at CSS and JavaScript, rarely at PHP. Asking them to go through a few thousand lines of XML is not the way to garner good will.

There was grumbling, and mumbling. Some that were happy until that moment left the class unhappy. Not because of Joel, but because of the export tool that Plone gives us.

When I brought this up with Joel after the class, he said he was aware of it but it was beyond his control. When I brought it up with others, I got a mix of apathy or rejection of the issue. I was told by several that the designer could hand it off to a developer if this was beyond them. Or that it would be too challenging to address. I mentioned it to Tres Seaver and I believe he said he would look into it.

In retrospect, I should have made a bigger stink about it.

In my opinion, Plone should be as developer friendly as possible. Generic Setup should have an 'Export Skin' function, available via the Plone Control Panel. Not the ZMI.

And I think that this should be part of core Plone. As soon as possible. Preferably Plone 3.3.

Why?

At least half the customers you will ever have care very little about the back end. They just want a pretty and functional site. Which is why graphic designers are often so important to them. For example, how many customers do you have who care more about the comps of the graphic designers than the engineering that publishes it?

By making Plone more accessible and friendly to graphic designers, it means they will stay excited about it. Which will attract more of the graphic designer friendly customers.

It gets better.

This export utility would empower graphic designers. They will be able to create Plone skins and publish them. They can make money doing this, advertising their craft. In fact, I predict dedicated on-line stores for this sort of thing will become popular, making the cottage industry for plone skins much more viable.

Which is great for us developers who lack decent design skills. If exporting skins is easy, then I can find (and purchase) them and modify to suit customer needs. And if I need one from scratch, I can rely on a larger crowd of graphic designers.

Northern Virginia Django Users Group lives!

Our acronym is NOVA-DUG.

We have a google group here: http://groups.google.com/group/NOVA-DUG

Me and my co-worker Katie Cunningham, are the co-owners of the group. We insist on ownership because we know exactly what we want out of the group.

• It has to meet on a night we can easily attend and that night is invariably Wednesday.
• It has to meet at a location we both can get to easily.

See you around.

Django Dash worst concept contest

The very scary Tony Hauber mentioned Django Dash on twitter. I was doubting if I would try to do it, because coming up with original ideas is hard and my design skills suck.

Then, via twitter, the wonderful Barbara Shaurette suggested that a clever implementation of a stupid idea + bad design is the only way to go in such a contest. That made me laugh. Then she suggested a little side competition for 'worst concept'.

Oh yeah! The contest is so on!

After a bit of trash talking with each other, it was exposed that she thinks that by living around start-up central in San Francisco she has more stupid ideas to work with than me. I am certain, however, that being in the Washington, DC area, the hundreds of elected representatives and senators around gives me the advantage of more stupidity.

I'm going to be decisive and suggest some rules for our little side 'worst concept' contest:

• Judging of the worst is by any third party we mutually agree on. I suggest a BDFL or framework tyrant, but will go with anyone you agree on.
  
• An implementation must be attempted during Django Dash. You get bonus points for completion of the implementation.
• The loser buys a drink for the winner at the next conference we both attend.

Barbara, are you okay with this side bet?

Jinja2 in zope

Getting Django Tempates inside of zope is not trivial. Yet getting Jinja2 to run inside of zope is easy! Use easy_install (or whatever) to fetch Jinja2 from PyPi and then from the zope debug shell:

>>> from jinja2 import Template
>>> t = Template('{{ name }} rocks!')
>>> t.render(name='Guido')
u'Guido rocks!'

So what does this mean?

It means that right now you can have Django style templates inside of Plone, Zope, Grok, et al. However, you can't mix TAL and Jinja2 in any way via template inheritence.

Off the top of my head I can think of two possible use cases:

1. You have the HATE for XML based template languages and just need something else for all rendering of all content including HTML. This is actually a very feasible option for Grok and pure Zope application development, yet functionally impossible for Plone. Too much of Plone is woven into TAL to make this work.
2. You are using lots of AJAX via KSS, JQuery, YUI, plain old JavaScript, etc and want something handy to help you write content coming from the server side. This is feasible in Zope, Grok, and even Plone.

The downside of using Jinja2 is that you are adding more complexity to the mix. I like simplicity. I would be very careful about using Jinja2 in Zope products, and would only consider option #2.

Thanks to the the indomitable Ian Bicking for suggesting Jinja2.

What I would change in Plone: Templates

These are my thoughts. My thoughts are not those of the luminaries in any community, but just your average Joe Developer. I formed these thoughts over Pycon 2009, and my experiences with Plone, Zope over 2 odd years and Django over the past 4-5 months. A lot of this is therefore inspired by my recent experiences with Django.

Item #1 - Plone/Zope needs Django Templates out of the box

You heard me.

I like the Template Attribute Language (TAL). TAL is stupid. Stupid in that it is very hard to encrypt my templates with business logic. I like this because I recognize I have no discipline when it comes to breaking the Model-View-Controller paradigm if a framework lets me. I also like TAL because it lets me create XML type documents very easily.

I like the Django Template Language (DTL). DTL is stupid. Stupid in that it is very hard to encrypt my templates with business logic. I like this because I recognize I have no discipline when it comes to breaking the Model-View-Controller paradigm if a framework lets me. I also like DTL because it lets me create non-XML type documents very easily.

I like both languages. Some people do not. Advocates of one template system style say very snarky things about the other template system style. Inevitably it comes down to that one is XML focused and the other is not. And from their point of view, they are completely right.

So I think that a version of DTL should be ported to the Plone/Zope world. And for Plone, it should be made readily available out of the box. This port should do all the default DTL filters and as many of the default tags as possible.

Which would I use? I would use either. More importantly, people who love XML based templates could use what they want and people who hate XML based templates could use what they want.

Everybody wins.

Item #2 - Plone TAL documentation needs a cheat sheet of the good functions

For Django, I can go to the DTL tag/filter page and find a handy reference on all the tags and filters available out of the box with Django.

Where is the same with Plone? The documentation has gotten really freaking good, but where is this sort of handy developer reference?

Yes, I have learned that you can use string methods on strings, and that there are various utility methods you can call if you know the name of the right portal object, but these are not in one easy-to-find location.

We Joe Developers need and love this sort of thing.

Forming a Northern Virginia Django User Group

There is already a Django User Group in the Washington, DC area (Django District). However, it looks like they meet in Silver Spring. Yuck. I live in Arlington, VA and I don't drive. I'm sure they are a nice bunch of people, but I want something more accessible.

I am a member of the Zope/Python User's Group of DC, which is organized by awesome Alex Clark. It has been going on for ages. Unfortunately they meet on Tuesdays which does't work for me or Katie Cunningham except maybe once a year.

See, what I want in a user's group is one that meets on a Wednesday or maybe Thursday once a month or so in a metro-accessible place with wifi in Arlington, Virginia. Katie agrees. So we are going to form what I nominally call the Northern Virginia Django User Group (shortened perhaps to Nova-DUG). We'll discuss and show off Django and the odd bit of python and pinax. We will be happy to run periodic events with Django-District, and we'll make sure not to fall in the same week as them. Should be fun for all!

More details as we cook them up.