My blog is moved

From now on I'll be blogging from http://pydanny.com.

See you there!

Silicon Beach Hackercast

Near the end of January of 2012 I was at the 10th Southern California Linux Exposition (SCALE 10X). I gave an Intro to Python talk, helped man the awesome Python booth, and hung out with a lot of awesome people. One of those awesome people I got to spend time with was Andrew Cholakian, a Ruby and Clojure developer, and the organizer of LA Hacker News. We've been meaning to do something together, and at the conference we decided to do a broadcast together.

Jump forward a few weeks and Andrew came over to our place in the San Fernando Valley for a day of brainstorming, web site building, and broadcasting. We outlined a show and started to record.

Download Now!

Topics of Discussion

• Python at Scale 10X
• New clients asking you to complete the ‘last 10%’
• A rant about job titles
• Why is clojure blowing up?

Links

• Silicon Beach Hackercast
• Silicon Beach Hackercast Episode #1
• Hacker News Thread
  

Art

Audrey was kind enough to illustrate the event for posterity:

The Future

The show was a lot of fun to do. We plan to do more of them, perhaps every few weeks. A better microphone is definitely in the works.

Capoeira at PyCon!

For several years I've wanted to have a Capoeira event at PyCon US. Think of it: a lively blend of martial arts, music, dance, and nifty moves all done with and for a large crowd. Well, this year it looks like it may happen! PyCon is giving us a room and my own teacher, Contra Mestre Xingu will be in town. We would love for you to join!

The only hindrance is before I can make this happen, I need just twenty (20) people to express interest in participating; either learn some beginner friendly moves, play in the Roda, or just play instruments.

We aren't going to make you do crazy tricks, in fact, anyone can do Capoeira. It doesn't matter if you've never done anything athletic, or you won the gold medal in Olympic gymnastics. Just fill out the form and we'll see you there.

Sign up for Capoeira at PyCon!

Updates to this event will be tracked at the PyCon Capoeira Open Space

Python Web Summit Questions needed!

I've been fortunate enough to land a moderating spot at the invitation only March 8th Python Web Summit that takes place at PyCon US 2012. The panel I'll be moderating is the somewhat contentious issue of code reuse across different Python frameworks. I'm working on some questions, but I would love input from the entire Python Web Community.

If you've got questions to ask, or ideas to suggest, please post them on this google moderator link.

How to justify attending PyCon sprints

Sad that the PyCon sprints fall on business days? Wishing you could stay but the boss/client won't let you and demands you back so 'you can work'? This is how you make it so that the sprints are something your management is demanding you attend every sprint ever.

1. Make it foremost in your mind that the wonderful thing about the PyCon sprints is that the odds are that anyone who knows anything about whatever you are doing in Python will be there.
2. Write up a list of the things that you are finding challenging, hard, or impossible to do with Python.
3. Now go to the boss and say something like:

   "Because the experts and leaders of the open source tools we are using are going to be there, I want to attend PyCon sprints. All my time at the sprints will be focused on sitting around them and working on our tools. I'll focus on things that directly impact our agency / company / organization, specifically things I wrote down on this list."

4. If the boss says,

   "Why not just use IRC or email?"

   Then you say something like,

   "Well, IRC/email is not the same as sitting next to these people. I'll be so much more productive there!"

Rinse and repeat.

Then, when you attend the PyCon sprints, follow through on what you said you were going to do. Sure, it might be more fun to work on project 'spam' even though your company uses project 'lumberjack', but if you prove how much you got done during the sprints, next year the boss will be much more encouraging. Even if a good boss says to go do what you want, at least spend some time sprinting on work related technology.

Also, once you get approval to go, consider sending your boss to this old rant of mine.

Don't forget to register for PyCon! Early bird rates end today (January 25th, 2012) which means today is the last day to get involved in the extremely unofficial PyCon Early Birds program!

Join the PyCon Early Birds program!

First off, I want to say that me and my fiancee will be attending PyCon US this year! Hooray! Can't wait to see old friends and make new ones. I'll be chairing one of the Panels at the PyWeb Summit on March 8th. We're absolutely delighted to see all the great talks, hang out in the hallway, and just be in the middle of Python for well over a week.

Now on to the extremely unofficial PyCon Early Birds program!

PyCon early registration ends on January 25th. If you register at the early bird rate that gets you the benefit of joining the elite PyCon Early Birds group. Being a member of the PyCon Early Birds gets you all sorts of incredible rewards and benefits.

• Most importantly, you get some serious bragging rights.
• A custom ribbon that says 'Early Bird' that you get to attach to your conference badge.
• A discounted rate from the regular ticket rate as according to the registration page.
• The confidence of knowing you have a ticket before they sell out.
• A tasty and rather edible store-bought cookie provided by myself and Audrey Roy.
• If the PyCon Early Birds program gets enough members, I'm going to challenge PyCon chair Jesse Noller to stump me with Yoga poses! There's no way he'll even consider accepting a challenge like this unless the PyCon Early Birds membership roster is big enough. So join and help me find out if his Bikram will beat my Capoeira!
• Other incredible things that are in the works!

Of course, PyCon has tons of other reasons to sign up besides the PyCon Early Birds program. Amazing tutorials, talks, and sprints, plus great hallway tracks, a vendor room filled with great schwag, poster sessions, and startup row. Sponsorship levels are unbelievably high, and since the event is non-profit that means the money just goes right back into the community - starting with PyCon itself. This year is going to be AWESOME!!!

So what are you waiting for? Sign up for the Pycon Early Birds before it's too late!

Tips for speaking

Last year I wrote a blog post called My tips for speaking. In the face of SCALE 10X and other events coming up, this is my refactoring of that talk.

My experience in giving talks? I've been presenting on the job since 1999. I've been presenting at conferences since autumn of 2008. This past year (2011) I gave a lot of talks, helped others with their talks, and did a number of cooperative talks where I shared the stage with another person. I've learned a lot, from my own mistakes and those of others. Since I like to share, here we go.

You aren't Steve Jobs

In terms of presenting, Steve Jobs was no mere mortal. He got away with wearing black in front of a black background. His slides were frequently dark. And he held up tiny objects in front of literally billions of people. He did all of this and got away with it.

You. Won't.

Steve Jobs was billionaire leading a multi-billion dollar company. His presentations were highly choreographed affairs where every detail from lighting to audio to who got to attend was tightly controlled.

For example, Steve could wear black because of amazingly proficient stage lighting techniques. Carefully watch a high definition video of him showing off the latest product while in a black sweater and you'll see that he's literally standing in a column of light. Odds are most conferences don't have spotlights or fancy lighting, and even if they do - the stage crew are not being promised a bonus for doing it right and financial ruin if they fail.

What this means is be very, very careful about using the attributes of Steve's talks for inspiration or arguments on how you are going to do a talk.

What to wear

Bring several presentation shirts to the conference so you can be sure that what you are wearing shows up in front of the background. Be a little classy and wear a polo shirt instead of that t-shirt. If the background is the same as your shirt, go and change. If you don't have a shirt that works, go buy one or wear your jacket.

The reason being is the same as why Newscasters don't wear some colors on video. They don't want to look like floating heads and hands. It happens to them, it happens at technical conferences, and it will happen to you.

Black text on white background

I said it last year and I'll say it again: High contrast slides please. If a designer, manager, friend, or spouse tries to stick colored backgrounds or text into your slides, politely remind them that what shows up nicely on the monitor or laptop will absolutely fail to be appreciated by the attendees of the talk. Social media and IRC will be filled with comments about the unreadable colors of your slides.

Remember, unlike Steve Jobs, you have no control over the AV team or whatever the conference venue provides when it comes to projectors and screens. So play it as safe as possible.

I really prefer the black text on white background. The people in the vision community (arts, theatre, animantion, and eye doctors) all agree with me.

This past year I've had technical people (all with jobs outside the vision community) quote some interesting material about the validity of white text on black background, but I personally find it hard to read those slides if I'm in the back. Also, if the room is brightly lit, the letters are barely visible.

The happiest alternative I've seen was at a Mongo event, and was a certain shade of dark blue with bright yellow letters. However, I think like the black background, it would be suspect in the wrong light.

One last note: Don't use color gradients in your slides. Please.

Cheat at the command line!

I'm giving a talk today at Southern California Linux Exposition 10x called Intro to Python. It's inspired by folks like Raymond Hettinger and David Beazley. And the big thing I'll be doing is avoiding the command-line like the plague.

This is because the command-line is treacherous in front of an audience. Conference networks are notoriously prone to going down at the wrong moment. Same goes for laptops, especially when connecting to projector hardware it's never touched before.

Also, you know what it is like typing with someone looking over your shoulder? Imagine that times hundreds, or thousands when your video is being uploaded so people around the world can view it.

So either use something like PLayer Piano to record a command-line session in a docstring, or do what I do and use slide transitions to mock typing.

Be rested, fed, and sober for your talk.

In 2011 I watched a guy fumble through a talk with beer in hand, a personal hero of mine blearily try to get through his talk after a late night of drinking, and a couple people try to present after all-nighters.

Respect your audience and take care of yourself before the talk. Get some sleep, eat well, and drink moderately. This will really help you get invited to more events.

Practice, practice, practice

In the past I've said I don't practice much. If at all. Ahem...

It has been pointed out to me that I practice. A lot. Maybe not in front of a mirror, but I'm constantly going through my slides and coming up with things to say. I'll go through all my slides one-by-one and mouth what I'll be saying, and work out the timing of things. Which means that while I haven't been projecting my voice, I have been practicing.

In 2011 I did the curious thing of actually trying to practice on a couple presentations. And I have to say that I've seen a huge amount of improvement. I certainly felt more confident and I feel like the talk goes better! For example, A talk I gave at PyCon AU 2011 that had some issues after a lot of practice went smashingly better at DjangoCon 2011.

Push questions and comments to the end

Your talk should have a flow, a pace as it were. And lots of interruptions will cause you to lose your chain of thought, or cause the audience to lose focus. If someone asks a question during your talk, ask them to wait until the end.

If they keep asking questions or giving comments, ask them nicely to talk to you after your presentation. People are decent and will respond nicely to your request.

Sign up for Pycon US!

Early bird registration closes on January 25th. Sign up for it beforehand and you'll have enough money for a really good dinner with drinks. What are you waiting for? Go do it!

My SOPA boycott

I'm against SOPA and PIPA.

I believe that those bills will kill not just free speech, but also business within the USA. Innovation will wither. I'm also of the belief that those companies trying to get SOPA into place don't realize that no idea is new and if SOPA passes they'll be hammered with an increasing amount of takedowns and suits against them for anything they do. Litigation based on SOPA won't be as easily handled as the current status quo.

I've signed the petitions, I've posted on Twitter, Facebook, and Google. That isn't enough. I have to be willing to make a sacrifice. And in this case I'm going to make the sacrifice my vote.

My vote sacrifice is a boycott. It's directed at any politician, local or otherwise:

My Boycott:

• If you as a politician vote for SOPA/PIPA then you lose my vote. Regardless of whatever other opinions you have or party you belong to, you've lost me as a supporter.
• If SOPA/PIPA passes you can get my vote back by voting for what bill that destroys SOPA/PIPA is nominated.
• If SOPA/PIPA fails you can get my vote back by voting against whatever bills are resurrected to replace SOPA/PIPA.
• I will ignore party boundaries. I will vote against my normal grain simply to get you removed from office.

Why?

Like campaign finance reform, controls of freedom of speech often have unpredictable repercussions. 

The terrible thing about these bills is that their supporters are bi-partisan. While it's wonderful to see political opponents working together, in this case, it's for a terrible cause. 

So I'm going to cross political boundaries too. I'm going to say that as a registered Democrat I'm going to vote Republican if a Democrat candidate at any level votes for SOPA/PIP.

About Me

I don't use pirated software. I don't read, watch, or listen to pirated content. I purchase everything legally or use open source equivalents. I make a pretty decent salary and am pretty much in the direct center of what is called the 'middle-class'.

I'm also pro-business and a rather patriotic citizen of the United States. I believe in our nation and what it represents, and I know these bills are going to be a dagger in the heart of what our founding fathers gave us.

Finally, as I said, I'm a Democrat willing to vote Republican, Green, Libertarian, or whatever to make my point. That's my sacrifice. My vote and role in this nation that took my family in over 100 years ago is now in the hands of politicians.

What's yours?

Resolutions for 2012

• Go to a Python related conference in North America, South America, Europe, Asia, Africa, Australia, and New Zealand.
• Attend at least one JavaScript related conference or event.
• Upload all my outstanding pictures to Flickr!
• Make Consumer Notebook profitable.
• Find more ways to make Audrey Roy happy.
• Pull off an Aú sem Mão during a Capoeira Roda.
• Attend my first Capoeira Batizado.
• See a place in the USA I've never been.
• Work out at least three times a week.
• Drop to a 32 waist
• Visit friends and family back east. Been over a year since I've seen my sister!
• Blog once a week. That is at least 52 blog entries!
• Visit a Theme park.
• Learn how to surf or snowboard.
• Implement something in node.js, backbone.js, and handlebars.js
• Take a high level Python class from the likes of Raymond Hettiger or David Beazly.
• Teach some Python or Django.
• Have a beer with Thomas, Andy, Andy, Tony, Garrick, Bernd, and the rest of Ye Aulde Gange.
• See my old DC area friends such as Eric, Chris, Steve, Beth, Sarah, Daye, Renee, Kenneth, Leslie, Whitney, Dave, and many others.
• Visit my Son.

New Year’s Python Meme

I love these blog memes, so I give you my version of Tarek Ziade's New Year's Python Meme.

1. What’s the coolest Python application, framework or library you have discovered in 2011?

For python libraries, that would have to be Kenneth Reitz' python-requests library. I've used it for an amazing amount of stuff and blogged about it. It took the grunge out of doing HTTP actions with Python. The API is clean and elegant, getting out of your way. It embodies the State of the art for API design, which closely matches the Zen of Python.

For applications, djangolint.com is awesome. It has helped me out so much on several projects. I would love to see something like this implemented and maintained for modern Python.

All the Python friendly PaaS efforts that have emerged are changing the landscape for those of us who want to launch projects but don't want to become full time system administrators in the process. Heroku, DjangoZoom, DotCloud, ep.io, gondor.io, and others are making it possible for developers to focus on development not server tooling. Google App Engine paved the way and it is wonderful to see the rest of the universe catch up with material that more closely follow core.

2. What new programming technique did you learn in 2011?

Event based programming! I've touched on it for years, but this year I really got a lot more more into it thanks to Aurynn Shaw kickstarting me and Audrey Roy expanding my knowledge ever since.

3. What’s the name of the open source project you contributed the most in 2011? What did you do?

I participated mostly as co-lead in the Open Comparison project, which amongst other things involved running the largest sprint at PyCon 2011. We maintained Django Packages and launched Pyramid and Plone versions of the project. We hope to launch a Python implementation in 2012.

I took a lot of notes this year at pydanny-event-notes - enough to make a book.

4. What was the Python blog or website you read the most in 2011?

Like Nick Coghlan, that would be http://planet.python.org.

5. What are the three top things you want to learn in 2012?

1. How to use whatever consistently maintained project that replaces PIL that works in Python 2.7.x and Python 3.x.
2. Really advanced Python as taught by Raymond Hettiger.
3. backbone.js

6. What are the top software, app or lib you wish someone would write in 2012?

A tool python-requests, but for shell access. Something like Unipath, but kept up-to-date and with nicely written documentation on Read the Docs.

A PIL replacement that is maintained, works for all modern Pythons, and is close enough to the PIL API to not cause too much confusion.

Something like Django Lint but for Python 2.7.x/3.x.

An open source project that tracks test coverages across PyPI and publishes reports of the results via an API.

Want to do your own list? here’s how:

• copy-paste the questions and answer to them in your blog
• tweet it with the #2012pythonmeme hashtag

Evaluating which package to use

In November of 2009 I wrote about which third-party Python Packages I'll use. Here is my modern take on it - much of it inspired by personal experience and the advice of peers and mentors:

Tag and release on PyPI

I really don't like pulling from tags on Github, BitBucket, or whatever. Or being told to pull from a specific commit. That works in early development, but it certainly doesn't fly in production.

I also get frustrated when people release on PyPI but then insist on hosting the release themselves. That is because invariably at some critical point in development when PyPI is up, the host provider is down.

A huge point of frustration is that I shouldn't have to leave the canonical source of Python package versions to hunt down what I should be using. I've seen too many beginning Python developers fall into the trap of using 3 year old packages because they didn't know they should be using trunk. I was guilty of doing it for a 6+ month old release in 2010, and for that I apologize and promise I won't do it again.

This also means your package needs to be pip installable. If you don't know how to do it, please read the The Hitchhiker’s Guide to Packaging.

Documentation

2011 is closing, which means your package needs to have Sphinx Documentation. And those Sphinx Docs should be on Read the Docs. Read the Docs is great because it doesn't just host the rendered HTML, it also lets you easily push to it from a DVCS push - and implements nice search and handy PDFs too.

Yes, I know there is packages.python.org but I don't trust it. It doesn't have the easy push/deploy workflow of Read the Docs, which means often the docs are dated because it's yet another step for developers. Plus, the lack of search outside of Sphinx makes it hard to discover documentation.

The same goes for hosting docs yourself. In fact, that's usually worse because when someone goes on vacation and the docs go down... ARGH!

Please don't mention easy_install in your docs. We are nearly in 2012 and ought to be unified on our package installer, which is pip.

Tests

You should have them. Otherwise any update you put on PyPI puts the rest of us at risk. We can't be sure your updates to the project won't break our stuff. So please write some tests! If you add in coverage.py and some kind of lint checker, it can even be fun! It certainly does earn you bragging rights having a high coverage rating.

Code Quality

Are you using new-style classes or old-style classes? Do you follow PEP-8? Do you keep meta-classes to the absolute minimum? Is the code on an available DVCS so others can fork and contribute? These are things that weigh in my judgement, and certainly the judgement of others.

Announcing Consumer Notebook!

Need a Python programming language book? Want to see a comparison of the ones I own and use? Check out my Must-Have Python Programming Books comparison grid.

Let's drill down and take a closer look at one of the items on the page, in this case Doug Hellmann's amazing The Python Standard Library by Example. The product detail pages include the ability to add pros and cons and attach said products to comparison grids and specialized lists like 'my wishlist' and 'my possessions'.

Speaking of wishlists, check out my own:

In order to add items, like footy pajamas,  I click on the 'add' button and paste the Amazon (or BestBuy) URL into the form:

At this time we just handle Amazon USA and BestBuy USA. In the future we plan on adding more affiliate providers, including non-USA providers to support our non-USA friends.

There's a lot more than that...

In addition to weekly infographics, comparison grids, lists, and products, Consumer Notebook also awards points, coins, badges, and a growing privilege set to participating users. We even implemented an energy bar which regenerates over time, designed to match the pace of human users and serve as one of the brakes on scripts and bots.

Technology

I built this with Audrey Roy using Python, Django, JQuery, PostGreSQL, Memcached, and RabbitMQ. I'll be blogging in depth about the technical side in an upcoming post.

Genesis

It was the summer of 2010 and we were brainstorming ideas for a coding contest called Django Dash. The one we settled on was a listing and comparison site for Django called Django Packages. The result has been a very useful tool for the Django community. Eventually, with the help of several dozen people, we turned the code into the Open Comparison framework and launched Pyramid and Plone implementations. Time permitting this year, we plan to do Python, Flask, Twisted, Node, JQuery, and other implementations.

Since then we've wanted to do something similar, but in the context of products. And we wanted to do it right - elegant design combined with an ad-free space. So we cooked up Consumer Notebook, launching today!

We'll be adding features and enhancements in the months to come. We've acquired a community manager, and even have a blog. We would love for you to check out the site, share it with your friends and family, and send us your commentary, suggestions, and advice.

Made Up Statistics

At DjangoCon my good friend Miguel Araujo and I presented on Advanced Django Form Usage. Slide 18 of that talk mentioned some made up statistics. Here they are for reference:

• 91% of Django projects use ModelForms.
• 80% ModelForms require trivial logic.
• 20% ModelForms require complex logic.

Important Disclaimer: These numbers were cooked out of thin air by yours truly. I determined them with zero research, they carry absolutely not scientific weight, and shouldn't be used in any serious argument. They are wholly my opinion, which is good or bad depending on your point of view and your own opinion of my opinions.

With that out of the way, I'm going to make a bar graph out of my fictional data:

You'll notice that my bar titles could be stronger. I actually did that on purpose in case anyone tries to use that chart in real life. In any case, if you thought that was interesting, then read on. I have many more made-up statistics. For example, here are more numbers I've cooked up:

Pydanny Made Up DevOps Statistics

DevOps is the new hotness. I know because every other Python meetup features someone speaking on it - just like every other Ruby, Perl, and PHP meetup. Anyway... numbers:

• 24.3% Python developers doing DevOps think they could have launched a PaaS (aka Heroku clone) before it got crowded.
• 46.3% Python developers doing DevOps spend all their time writing Chef/Puppet scripts and yet still claim to be Python developers.
• 14% Python developers are worried about so much of the backend being done in Ruby.
• 54% Python developers are just happy that there are many options now and don't care about the internal machinery that much.

This time, because I'm worried about the data being taken seriously, I've titled the bar chart in such a way that no one will reference it in anything important:

Pydanny Made Up Python Enviroment Statistics

Following the obvious logic flow (to me anyway) of DevOps to something else, let's go into Python environments, also known as the VirtualEnv vs Buildout debate, which adds up to an even 100% (making it good pie chart material):

• 77% of Python Developers prefer VirtualEnv.
• 13% of Python Developers prefer Buildout.
• 7% of Python developers rolled their own solution and wish they could switch over.
• 3% of Python developers rolled their own solution and are fiendishly delighted with how they have guaranteed their own job security forever. I know who some of you are and I can say with some confidence that when the Zombie apocalypse happens, no one is going to invite you into their fortified compounds. We hate you that much.

Pydanny Made Up Template Debate Statistics

The made up statistics in this post frequently touch on contentious topics. So let me add another controversial topic, this time the never ending template debate in Python:

• 70% python developers prefer non-XML templates
• 25% python developers prefer XML templates
• 5% python developers wonder why we don't just use the str.format() method and be done with it
• 50% python developers strongly disagree with my Stupid Template Languages blog post from last year.

The display for this data is a lovely pie chart as seen below. In order to make it appear more useful, I took out the part where people disagreed with me and also made it a 3-D pie chart.

Pydanny Made Up Python Web Optimization Statistics

I sometimes get asked how to best optimize a Django site. My answer is 'cache and then cache some more' but there are those who disagree with me and start switching out Django internals before doing anything silly like looking at I/O.  My bet is this same thing happens with other frameworks such as Pyramid.

• 20% developers argue switching template languages.
• 80% developers argue using caching and load balancing.
• 100% Django/Pyramid/Flask/etc core developers argue using caching and load balancing.

Of all the made up statistics in this blog post, I suspect this is the one closest to the truth of things.

Update: Alex Gaynor and Audrey Roy pointed out that the original line graph for this data was not appropriate. My weak defense was that I'm trying not to make things too serious but they stated that the line graph was so inappropriate it distracted from the rest of the post. Thanks for the advice!

Pydanny Made Up Framework Debate Statistics

Alright, let's conclude this article with some statistics I cooked up about frameworks in Python. I'm going to do more then just mention web frameworks, dabbling into other awesome things that the Python community has given us.

• 23.6% of us get web.py and web2py confused with each other.
• 42% Python developers think Pyramid/Flask have awesome names that don't get mispronounced the same way Django does.
• 28% Python developers wish they could find a way to get some SciPy into their projects.
• 22% Python developers wish there was a PEP-8 wrapper for Twisted.
• 49% Twisted developers wish that Python had accepted their standard instead of PEP-8.
• 90% Python developers wonder what they were drinking when they renamed it to BlueBreem and wonder if it is sold over the counter in their municipality.

No chart? Getting this one to look meaningful was turning into a herculean effort. I invite others to render this data into something that look attractive and doesn't lose meaning. Come up with something impressive and I'll put it into a follow-up blog post.

The Story of Live-Noting

Like a lot of people, I've got this thing I do when I attend conferences, meetups, classes, and tutorials: I take notes. My open source based ones are mostly written in RestructuredText and I've kept in a particular folder since at least 2006.

Putting notes in a DVCS

On September 13, 2009, I uploaded these notes to Github.com. I did that because I wasn't pleased with the workflow I established of moving items to Dropbox for backup. I use DVCS all the time and I figured why not just put my notes where I put my code? So I added my notes as a Github repo.

DVCS Notes Based Management System?

For a while I tried to use the Github folder README.rst trick to make a navigations system for my notes. But Github isn't designed for making a README into a dynamic custom content navigator, and it would make a silly feature request. I would rather the Github team work on Mercurial integration or other practical things before they honored a request to turn their system into my own custom Notes Management System. Eventually I just gave up on it and moved on.

Sphinx + Read The Docs!

In early July of 2011 I had a wicked fun thought. What if I turned my notes into a Sphinx project and posted it on readthedocs.org? Most of my content is in RestructuredText and I've gotten really fast at rolling out Sphinx documentation. The 'hard' part would be converting the few README.rst files into index.rst files, but on the flip side I could use fancy Sphinx directives.

I'm not exactly sure when I started down this path, bit this commit log entry leads me to think I had it working on or around July 8th. What that would mean is that every time I pushed up a change in my notes, within minutes readthedocs.org would publish the content to the world in lovely HTML markup.

The result?

Pydanny Event Notes

Here's a screen shot of the front page

PyCon Australia 2011 Test Drive

For the 2011 PyCon Australia I gave my new process a serious whirl. I found if I created the page before the talk and entered some basic data like author and title and tied it to the index then I could constantly check the quality of my output while taking my notes. It made my notes seem a bit more exciting and alive. I even tweeted about it cause I thought it was fun, and people around the world seemed to enjoy the effort I was putting into my notes.

Because I was committing constantly in order to get updates on readthedocs.org as soon as possible, I also adopted the habit of super-short pull request messages. That's because the content I'm writing overrides the need for verbose comments. So when you see me writing "moar" it's because every minute or so I'm doing something like:

$ git commit -am "moar"
 $ git push

Kiwi PyCon 2011

I did my rapid note taking again at Kiwi PyCon and it was fun. The downside was that sometimes I get rather critical in my notes and I had a couple speakers come up to me later to clarify their positions. This makes it a bit challenging because I want to put down my thoughts, but if my thoughts impact another person, what should I do? Especially since if my negative notes on someone turn up in a search it can negatively impact the speaker way beyond a single talk. This is now always on my mind when I take notes, and I'm trying to figure out a good way to handle this going forward.

In essence, I don't want to constrain what I write but I also don't want to write something that will haunt someone else later. Even with a caveat and all that stuff, it can still be problematic. There is a difference between me ranting about something and me taking notes, and the written word is such that things are all too often taken out of context.

Food for thought indeed.

DjangoCon 2011 and the invention of the term 'live-noting'

At the start of DjangoCon 2011 someone tweeted that they were planning to 'live-blog' the event. Suddenly I realized that what I was doing had a name for it, and that was 'live-noting'. So I tweeted that was what I was doing and it seemed to catch on.

Not only that, but I got asked if I would accept pull requests. After a good two seconds of deep thought, I responded that I would only consider corrections and clarifications, not new material. I received not just one, but two pull requests from good friends and left the conference pretty happy.

On top of that, I managed to get featured on the front page of http://readthedocs.org! (Thanks Eric)

Kenneth Love also took notes in a similar fashion: readthedocs.org/docs/djangocon-2011-notes

PyCodeConf 2011

I had the excellent fortune of being an invited speaker to Github's PyCodeConf. While I gave my talk, my lovely fiancée, Audrey took notes of my talk and submitted a pull request. Her contribution was the first time I accepted content I did not write, and I'll say right now she's the only one for whom I will accept such content. On the other hand, If you take notes when I present let me know and I'll link to them from my own notes.

Josh Bohde also took notes at the event in a similar fashion readthedocs.org/projects/joshbohde-event-notes and even as I write this post he shares the featuring of our notes on the frontispiece of readthedocs.org:

Closing Thoughts

I often use my notes as reference, and if you follow the commit logs you may even see me comment or clean up things I wrote down years ago.

The graphs and stats of this effort is really interesting. Fortran? And a total of
Five contributors!

All of this makes taking notes a lot more fun. I enjoy finding ways to enhance and improve my process, and find it exciting that others are following a similar pattern of effort. My hope is to make 2012 the Year of PyCon, where I find a way to go to a Python related conference on six continents (Antartica is too cold for my tastes) and take notes everywhere.

Going forward, should I document how I built this out? Would my steps and patterns be useful for others?

Redux: Python Things I Don't Like

Back in May of 2009, I wrote about Eight Things I don't like about Python. It was my attempt to come up with things I don't like about my programming language of choice. Consider this my update of that post.

1. Division sucks in Python

In Python 3 this is fixed so that 2 / 3 = 0.6666666666666666 but in Python 2.7.x you have 2 / 3 = 0. You can fix that in Python 2.7.x with doing a from __future__ import division before your division call. Can anyone tell me if a version of 2.7.x will natively support 2 / 3 = 0.6666666666666666 without that import?

Note: Chris Neugebauer pointed out that changing division in Python 2.7.x will break backwards compatibility. However that doesn't change that I don't like it in Python 2.7.x.

2. TKinter blows

Honestly, it doesn't really matter to me anymore. I either use command-line scripts or things delivered to the web. Also, thanks to Brett Cannon, I know if I need to make TKinter look good, I can use TK Themed Widgets right out of the standard library.

3. Lambdas make it easy to obfuscate code

I'm known for not liking lambdas in Python. These days, I do know of use cases for Lambdas, but those are far and few between. I might even try to turn that into a blog post this month - use cases for Lambdas in Python. Fortunately for me, these days I seem to work with people who mostly agree with me on this subject.

4. Sorting objects by attributes is annoying

This is still annoying for me. As I said, "... the snippet of code is trivial. Still, couldn't sorting objects by attributes or dictionaries by elements be made a bit easier? sort and sorted should have this built right in. I still have to look this up each and every time."

I've thought of proposing something easier as a PEP. Imagine that! Me submitting a PEP!

5. Regex should be a built-in function

Before I got to do Python full-time I was a go-to person with regular expressions. Languages without them were weak in my opinion. Since then (2006-ish) my skills have faded somewhat in regards to regular expressions. And you know what? It hasn't been a problem. Python's string functions are fast and useful, and when I really need regular expressions, I import the library and do some research. I'm considering this one closed.

6. Reload could be less annoying

Reload only works on modules. I want to be able to something like reload(my_module), reload(my_class), reload(my_function), or even reload(my_variable):

>>> from my_module import MyClass, my_function, my_variable
>>> mc = MyClass(my_variable)
>>> mc 
5
# I go change something in my_module.MyClass and save the file
>>> reload(MyClass) # reload just MyClass
>>> mc = MyClass(my_variable)
>>> mc 
10

My current fix is to use unittest as my shell as much as possible. And that is probably a good thing.

7. Help doesn't let me skip over the '__' methods

As I said way back when, "Python's introspection and documentation features makes me happy. And yet when I have to scroll past __and__, __or__, and __barf__ each time I type help(myobject), I get just a tiny bit cranky. I want help to accept an optional boolean that defaults to True. If you set it to False you skip anything with double underscores.

The See project is one solution to the issue. A different approach I've used is the Sphinx autodoc feature, but Sphinx is a lot of work and doesn't cover every contigency.

8. Not enough female Pythonistas

These days I know a lot of female Python developers. There is my own fiancee, Audrey Roy. Face-to-face I've met and talked to Christine Cheung, Jackie Kazil, Leah Culver, Katharine Jarmul, Katie Cunningham, Barbara Shaurette, Esther Nam, Sandy Strong, Sophia Viklund, Jessica Stanton Aurynn Shaw, Brenda Wallace, Jen Zajac, and many more I know I'm missing. And there are even more with whom I've had in-depth online conversations.

So why didn't I put a strike-through on this one? Because the numbers still aren't good enough. I know a lot of female Pythonistas, but how many do you know? And even if you know a decent number, what percentage of a meetup group you attend are women?

I can say that things are improving, but they could be better - for women or minorities. Find ways to pitch in, be it PyLadies events, PyStar workshops, or what have you.

One last note on this subject, I've heard some unsubstantiated statements that the .Net world has a higher female-to-male ration then the Open Source world. Are we going to take that kind of thing sitting down?

Loving the bunch class

Warning

This is me playing around with things in Python. It's not anything I use in real projects (except maybe the odd test). Please don't use these in anything important or you'll regret it.

Every play with a bunch class? I love 'em and make them protected or unprotected. I started using them early in my Python career, although it wasn't nearly about 2 years ago that I learned what they were called and the best way to code them. In any case, here is a simple, unprotected Bunch class.

# Simple unprotected Python bunch class
class Bunch(object):

    def __init__(self, **kwargs):
        self.__dict__.update(kwargs)

bunch = Bunch(name='Loving the bunch class')
print(bunch.name)

You can also make protected ones, that don't let pesky developers like me overwrite attributes, methods, and properties by accident:

# Simple protected Python bunch class
class ProtectedBunch(object):
    """ Use this when you don't want to overwrite existing methods and data """

    def __init__(self, **kwargs):
        for k, v in kwargs.items():
            if k not in self.__dict__:
                self.__dict__[k] = v

You can also write them to raise errors when a key is in self.__dict__. Or perhaps merely publish a warning. There are many ways to customize, but generally you want to keep these things as simple as possible. Anyway, let's get back to the main topic...

In the early days of my experiences with Python I found a small, nagging issue with dictionaries and objects. The notation wasn't as handy as what you got with JavaScript and some other languages I was using at the time. For example:

// JavaScript object notation
o = {};
o.name = 'Loving the bunch class';
o.name; // Calling with 'dot' notation
o['name']; // Calling with 'bracket' notation  

Unfortunately, in Python you can't do this with a normal bunch class:

# Python bunch class failing on bracket notation
class Bunch(object):

    def __init__(self, **kwargs):
        self.__dict__.update(kwargs)

bunch = Bunch(name='Loving the bunch class')
print(bunch.name)
print(bunch['name'])
Traceback (most recent call last):
  File "", line 1, in 
TypeError: 'Bunch' object is not subscriptable

The quick answer is found with a little trick I found in the comments of a recipie by Alex Martelli that gives you the ability to do:

# Fancy dictionary/object trick
class Buncher(dict):
    """ Warning: DON'T USE THIS IN REAL PROJECTS """

    def __init__(self,**kw):
        dict.__init__(self,kw)
        self.__dict__.update(kw)

bunch = Bunch(name='Loving the bunch class')
print(bunch.name)
print(bunch['name'])

I'm not the only one who likes Bunch classes. On PyPI I found a really complete implementation.

Of course, in a lot of cases you probably don't want this 'weight of code', right? Dictionaries being lighter than full objects and all that. Nevertheless, it's fun for noodling and playing around with code. Still, I'm thinking it might be a fun little project to take a group of bunch implementations and do performance checks on them versus each other and dictionaries. Maybe the 'performance hit' isn't so bad.

I should also dig into things like defaultdict and other constructs to learn more. Part of the fun of any programming language is the depth of even the 'simplest' components of the language.

PyCodeConf 2011 Report

As my fiancee said, "PyCodeConf is a new kind of Python conference with a radically different format. Speakers are invited to speak about whatever they desire relating to the theme ("The Future of Python"), in front of a room of round tables. In between talks there are long breaks to encourage discussion. As a result, talks are edgier, and you really get to know people and possibly shape the future together."

This summer (2011) I was invited by Chris Wanstrath to be an invited speaker at the first ever PyCodeConf, to be held in Miami, Florida. I've been using Git and Github since early 2009, and more importantly, I've known Chris since DjangoCon 2009. I've always appreciated his interest in not just providing tools for the community, but also his efforts across languages and platforms to improve the lives of developers and those who support developers. And that he and his partners seem to make a pretty penny at it and share what they make (drink-ups and now conferences) is only a good thing in my opinion.

So I accepted Chris' invitation. :-)

The theme for the conference was the future of Python, and I submitted a talk proposal about Collaboration. Chris helped me figure my talk topic, which was awfully nice of him. Shortly afterwards my fiancee, Audrey Roy, received her own invitation to speak.

Between the time that Chris invited me to speak and the start of conference, Chris Williams of JS Conf got involved.

Alright, let's review things...

Accommodations

Everyone stayed in the Epic Hotel, in downtown Miami. The rooms fit the name of the hotel, being Epic in size and having amazing stuff in them. The rooms had free/good internet if you signed up for a hotel mailing list. Since we rehearsed and polished talks the night before the conference we ordered room service and were pleased with what we ate. The hotel had a heated outdoor pool on the 14th floor, which I'll get into later. In any case, the only time I've been in a comparable hotel was the incredible arrangements provided by the PyCon New Zealand folks who put us up in the Museum Hotel for a few days.

Sure, $189/night is high, but 3 nights when you split it with 2 or more people makes it not so bad.

Result: Superb

Conference Meals

If you are serving me food, you can't go wrong with salmon, steak, good cheese, fresh vegetables, coffee, and juice. I can report that PyCodeConf did quite well in this regard.

Outside of the conference I really enjoyed the food. Andiamo was a crazy good pizza place. I also got some really nice grouper which you can only seem to get in Florida.

Result: Superb

The Conference Room

The conference took place on the 14th floor of the Epic Hotel. This was a single track conference, with all the talks were given in the same room. The room was large, but everyone had a comfortable seat at large round tables. That was a nice touch, because it encouraged you to socialize with everyone nearby.

That worked out for the most part, except for a couple of developers sitting at a table who had backs turned to the speakers and were talking loudly while pair programming or something. I asked them to quiet down but 30 seconds later they were back at it. We moved away, but in retrospect considering their rudeness, I should have politely asked them to take it outside.

Anyway, the acoustics were good, the temperature comfortable, and the seats comfortable.

Result: Superb

Speakers

Speaker selection was done magnificently well. There wasn't a dud within the lot of speakers. I normally expect that in any conference you'll get at least one dud talk per day, and pycodeconf didn't have that problem (unless my talk sucked).

Jesse Noller opened things up with a great encouraging talk, Raymond Hettinger gave a talk on Python basics that was so full of nuance that I'm terrified of attending an advanced talk by him, Alex Gaynor filled us with hope for PyPy, Tracy Osborn taught us how to bootstrap entrepreneur projects, Travis Oliphant wants Python core and PyPy to collaborate more with the Scientific Community, Audrey Roy gave up some of her community building secrets, David Beazly explained the issues of the GIL in terms mere mortals such as myself can understand, Gary Bernhardt gave an amazing talk comparing Python and Ruby, and Leah Culver made Django + backbone.js look easy, but if you talk to her you know whatever she does is sophisticated and not for beginners. Dustin Sallings sold me on a neat idea for testing to help catch edge cases and Armin Ronacher opened my eyes on WSGI.

The wonderful thing about these talks is that since everyone knew the upcoming tracks, or had seen the previous ones, we could relate to each other. So David Beazly, Travis Oliphant, and Avery Pennarun raised interesting concerns about PyPy that everyone got the chance to hear about. They weren't show-stopping issues, just raising awareness about things that Alex didn't cover in his talk.

I live-noted the event as much as I could, with notable gaps in Leah's talk (she talks fast and is very technical and wanted to give her my full attention) as well as Armin (his talk shocked me a bit - I'm still a WSGI newbie). You can see my efforts at my PyCodeConf live-notes.

Speaking of live-noting, Josh Bohde also live-noted the event and captured a ton of stuff I missed.

The gaps between talks were also a nice 15 minutes. That meant you could stretch your legs, get a drink, and talk to people.

Result: Superb

Parties

We (me and Audrey) missed the first party (hosted by New Relic at a place called DRB) on account of preparing for our talks. We always do our absolute best on talks, and both like to practice a lot. Also, Mark Pilgrim's disappearance had touched me and I wanted to talk about it. We heard it was a great party, so we'll assume that it was. :)

The next evening the party (hosted by Heroku) was on the 14th floor, which meant it was a pool party! There was great food, good drink, and a latin jazz band playing. One of the pools was heated, so most people stayed in there, and drank many watermelon mojitos served by the staff. The pool was a huge hit because it was comfortable and people just talked freely. No laptops, no phones, just talking. Chris Williams served us drinks himself, Chris Wanstrath got wet, and everyone just relaxed. I have to say, a heated pool party is something EVERY conference should give. MOAR POOL PARTEEZ PLEEZE!

The final night's party was at the News Lounge and was hosted by Github and Droptype. The drinks and people were awesome, and I have good memories of being in a circle listening to Chris Williams and Audrey Roy talk. I did go beyond tipsy, overdid the Capoeira, and tried to convince Chris Wanstrath to give up the whole DVCS hosting thing to do wedding planning. There was also a bunch of us getting kicked out of a Karaoke bar because of the antics of a Python core developer. Ha ha ha. It was a crazy night that took me two days to recover from.

I'm glossing some important discussions that happened while I was still sober at these two parties, and maybe in the future if things play out right I'll go over them.

Result: Superb

People

Part of attending conferences is to meet old friends and make new ones. I got to spend time with Mark Ramm, Jesse Noller, Nick Coghlan, Alex Gaynor, Ben Firshman, Armin Ronacher, Raymond Hettinger, Rachel Hettinger, Chris Wanstrath, and many other excellent people. I also got the chance to meet and befriend Kenneth Reitz, Chris Williams, David Cramer, Wayne Witzel, and Leah Culver. I'm missing at least a dozen more. It was great to put faces to people, and in some cases, hear their side of a story.

Also, I got to gush at programming heroes like David Beazly and Travis Oliphant like a total fanboy.

Believe it or not, I got a bit shy. I'm kicking myself over not introducing myself to more people. Next time!

Result: Superb

Summary

The conference was amazing. Like all conferences it had its own character and fun. Because it was a purely commercial conference I was a bit worried going into it, since I've heard about the corporate feel of these things. Those fears were completely mitigated by the open attitude and decentness of the conference organizers and sponsors. I look forward to attending PyCodeConf again in the future.

If you've got good senior technical staff and you want them to benefit from a conference, this is a good place to send them.

Overall Result: Superb

Conference Talks I want to see

I'm writing this the day after Github's pycodeconf ended. That was an amazing conference, and I'll be blogging it soon (I'll also be writing about PyCon Australia, PyCon New Zealand, and DjangoCon US). With all this conference experience very current in my head, things I've seen and done at them, and the deadline for PyCon US submissions coming up, here are some talks I really want to see happen in the next six months. If not at PyCon US, then please consider these for other forthcoming events!

Note: Couldn't do my preferred 'linkify' as well as I liked thanks to bad hotel internet. I'll clean it up later.

Advanced SQL Alchemy Usage

I think the uber-powerful SQL Alchemy ORM needs the same sort of treatment me and Miguel Araujo gave on Advanced Django Forms Usage. Not a 30 tutorial or overview or 'State of', but tricks and patterns by someone who has used it frequently on more than one project. Multiple projects is important because the speaker should have had the chance to try multiple approaches. Start with something simple like a TimeStampModel all model classes might inherit from, then go into deeper and and more complex technical detail. Finish the talk with something crazy hard from SQL Alchemy that is hard to explain. If that causes you to open a bug/documentation ticket, then you'll know that you've done the talk right.

Advanced Django Models Usage

Following the same pattern as my SQL Alchemy idea above, start with something simple like a TimeStampModel (including South migration of fields), then go into complex looks with Q objects, good patterns for Managers, Aggregation, Transactions, and then finish it with the craziest, hardest thing you can find. When putting together the closing material causes you to open tickets for broken core code/documentation, then you know you've done it right.

Python Code Obfuscation Contest

This certain-to-be-controversial talk idea would be where the speaker would solicit Pythonistas to submit a single arcane Python code module that would have to display the text of "Although that way may not be obvious at first unless you're Dutch." There would be a 'Expert' category which would forbid the eval/exec functions. The "Anything Goes Category" would allow use of eval/exec. The conference talk would be where the speaker announces the winners and comments on the brilliant insanity of submissions.

Django + Flask + Pyramid: A demonstration of useful things you can do with WSGI

At pycodeconf Armin Ronacher showed how with WSGI, he can run Django, Flask, Pyramid all from same server from the same domain. This surprised a lot of people, including me, and I want to see more of what Armin was talking about. I don't want any theory. I don't want anything obscure. I just want meaty bits I can implement the day after I hear the talk.

Zen of Python

Richard Jones gave his version of the talk at PyCon AU, and I want to hear other opinions about it. I'm happy to hear an expert give his view, and I would also be delighted to hear how a beginner (or relative beginner) feels about it.

Websites and OO Design Concepts: A Tutorial

For beginners, I would love to see a talk on a list of OO theories, and as each list item was discussed, examples designed in the context of a web site, how to do things right, plus identified anti-patterns would be presented. The web angle would be a good way to get the incoming Python web crowd to attend and identify with raised issues.

Quick conferences report: Presentations

My lovely Fiancée, Audrey Roy, was invited to be the opening keynote speaker at both PyCon Australia on Diversity in Python (video) and PyCon New Zealand on Python on the Web.

As for me, I managed to get talks into both of those conferences AND DjangoCon US. I co-presented on three of them, and I share all credit for success with my cohorts. The talks I gave at the conferences were (I'll post videos when they get up):

Confessions of Joe Developer (PyCon Australia, DjangoCon US)
The genesis of this talk was as a lightning talk at I gave at the Hollywood Hackathon. It is a talk about admitting that us mere mortals need to ask questions, take notes, and follow good practices in general. I gave it again at LA Django this summer, extending it to a full length talk complete with lots of technical content. At PyCon Australia I toned down the technical content because I was nervous, and while the response was positive, it  could have been much better. So for DjangoCon I ramped up the tech-talk and it worked much better. I've now given the talk 4 times, and I'm leaning towards retiring it.

 

Python Worst Practices (PyCon New Zealand)
This talk grew out of a SoCal Piggies lightning talk which I gave for the purpose of humor. Often we as Python developers are smug in the clarity of the language that we don't realize just how easily we can obfuscate code. In fact, I contend that Python is fully capable of a code obfuscation contest. This talk rejects a lot of crazy practices I've either done myself or had to debug from other people's work. For New Zealand I added a ton of content and tested things pretty diligently. The variable naming pages stumped some people I really respect and I was quite happy with that result.

 

Django Packages Thunderdome (co-presented with Audrey Roy, DjangoCon US)
Audrey did most of the work for this presentation. In this talk I helped review a horde of Django Packages across 7 different categories. It was nerve wracking because every part of our talk would get judged - but Audrey kept things really positive and made it clear we were providing constructive criticism. I think she got her message across to most people, and more importantly, it got a lot of people thinking about what ought to be normal community standards. I'll probably blog on those community thoughts and statements later, but I think Audrey (with help from me) accomplished what she aimed to do.

View more presentations from Audrey Roy

Advanced Django Form Usage (co-presented with Miguel Araujo)
Some time ago Miguel befriended me and helped resurrect the django-uni-form project. He graciously agreed to help me present on Django Forms and we decided to make the talk as sophisticated as possible. Previous Django form talks have been good, but focused on the fundamentals and we wanted to do something really different. This talk was hard because Miguel and I were on opposite sides of the planet, so we did a lot of github pull/pushes. In both doing research and presenting Miguel did an unbelievably good job and I hope he does more of this in the future. The response was extremely positive and I'm certain that our plan of getting our notes/work/transcript into Django core is well on it's way.

 

Ultimate Django Tutorial Workshop (DjangoCon US)
I got about 10 professional Django experts in a room, including Django core developers, and had them help me coach nearly 20 people through a modified version of the Django tutorial. Students seemed to learn tons, lots of socializing happened thanks to some happy accidents, and the experts got a chance to really see where the Django tutorial needs work. PyLadies organizer Esther Nam spent her sprint days working on something that ties the slides into the Django Tutorial - and for now I'm holding off on sharing my work until she says her work is done.

Summary
These were amazing opportunities to speak and will hopefully make a difference. I wouldn't have traded all of this for the world. It was a lot of work, and I doubt I'll ever go quite at this pace again. My plan is to do fewer talks and make them much better.

Responses to Github is my resume

Shortly after I posted Github is my resume the responses started coming in. They seemed to fill these categories:

"Github is a portfolio, not a resume!"

I think this is rather valid, being a much more accurate description of the role that Github and other social coding sites are having in getting developer jobs these days.  Two of the more choice responses in this category were posts by Gini Trapini and Andy Lester.

"In X years of hiring, I've never requested source code along with the resume!"

This comment raised the issue that personality, location, writing skills, etc were important. I agree that being able to not annoy your team into losing productivity is important, but it doesn't negate the frequent desire to be able to review the work of potential hires. Ignore the code at your own risk.

"Using only binary for calculations, how many ping pong balls fit in your car?"

A couple people said they prefer to ask programming questions or challenging problems in interviews to seeing portfolios of code. Personally, I think a few programming questions are okay but in my opinion 'challenging problems' all too often means sticking your interviewees with puzzles and trick questions that all too often have nothing to do with the day-to-day work of being a developer.