- Travel to Europe again.
- Travel to Asia or Africa.
- Visit a Disney park.
See a place in the USA I've never been.- Drop the waist size 2 inches and not break any bones.
Go to Pycon and present or teach.Go to DjangoCon and present or teach.Present at LA DjangoContinue my Muay Thai and Capoeira studies, get back into Eskrima, learn some more BJJ, andpractice the forms I know.- Work out at least three times a week.
- Go back east and teach martial arts for a day.
Finish some outstanding legal proceedings.Launch a site that does cool stuff and somehow brings in money.- Get to the point with LISP where I can do cool stuff in it without needing a textbook.
- Blog once a week. That is at least 52 blog entries!
Explain why I wrote Diversity Rocks.
Thursday, December 30, 2010
Resolutions for 2011
Recap of 2010
This has been an unforgettable year. I can't believe all this stuff happened. I've moved many times, met wonderful people, and seen my life change in ways I could have never predicted.
The good:
The good:
- Met a whole new bunch of wonderful people - relationships to last a lifetime. Arthur, Audrey, Celso, Cody, Daniel, Jacob, Jeff, Lahn, Moriah, Skyler, Todd and others I'm certainly forgetting to my undying shame. Thanks for your awesomeness!
- Presented and taught at Pycon 2010.
- Had a blast in the blizzard of 2010.
- Prepped my house of 9 years for sale and sold it.
- Paid off my debts. I sleep so much better now.
- Left NASA after 5 years and 3 months for freelancing pastures. Thanks Eldarion LLC, Revsys LLC, Holdenweb LLC, and Bryan Klein. Being exposed to some of the incredible code and developers on these projects has made me a better developer. Show me more!
- Flew enough in the spring to put me ahead of notorious traveller James Tauber for about 2 months in sheer mileage accumulated.
- Got to reconnect with a cousin and met his lovely wife and handsome son. Also hung out with my Uncle Al and Aunt Sandra.
- Got a tour of Ames Research Center thanks to Michael Sims. He'll probably be presenting at Pycon!
- Attended another NASA wedding by Mark and Ariel.
- Took a cruise to Mexico on a ship that caught fire just months later.
- Said goodbye to Virginia and moved to Lawrence, Kansas to run with the unicorns and 5 months later to Los Angeles, California.
- Road tripped across the Western half of the country! Twice! Spent 3 days in Las Vegas over Halloween!
- Learned fundamentals of Capoeira from Celso Wills. Wish I could learn more from him. :(
- Learned how to drive and bought my first car.
- Worked on another course for Holdenweb LLC.
- Launched Django Packages.
- Presented at DjangoCon 2010.
- Started Muay Thai classes in Los Angeles. I've done quite a bit of it, but these were the first regular classes I've had in about 16 years.
- Brought my son to Los Angeles so I could spend a week with him for the first time in 15 months.
- Almost up to 7 years without a broken bone!
- Worked another year in Python related technologies.
- Started learning LISP and played a lot more with JavaScript (especially JQuery and JQuery UI).
- Left my students in Virginia. The young ones were especially hard to let go.
- Had to say goodbye to friends in the DC area. Some people I'll miss are Alex, Beth, Brandon, Chris, Dave, Daye, Eric, Jim, Joe, Katie, Muhammed, Pilar, Rich, Renee, Ron, Sarah, Sebastion, Shawn, Steve, and Whitney. I know I'm missing some names!
- I'm far from all my immediate family: Carla, Ciana, Dad, Doug, Michele, Midori, Mom, Seth, and Timiri.
- Watched NOVA Django falter after I left the DC area.
- Had to wait 15 months between spending more than a day or so with my son.
- Didn't really practice much Eskrima all year.
- Didn't blog enough. Last year I blogged almost twice as much!
- Frustratingly, haven't been able to finish something that prevents me from writing about certain things. Yes, this is quite vague, and my hope is to clarify it in a few weeks.
2010 Resolution Summary
Items that are crossed out are completed.
Sell the house, pay off the remainders of my debts, and get my own place to stay. And get a car too.
Travel to another country.
- Take my son to another country.
- Drop the waist size 2 inches
and not break any bones.
Go to an amusement park, visit the beach, and also see a part of the USA I've never been.
Do more educational work for Python related technologies, and that includes getting the Django Education Foundation really rolling forwards.
- T
hrow away at least half my current stuff. I don't have much stuff now, and I want even less. Also, If I don't use or interact with any one of my non-book remaining possessions by 2011, I'm throwing it away.
- Move my blog to my own system and blog at least once a week.
- Get a mountain bike and have reasons to use it.
- Get back into Eskrima, focus more on BJJ,
get into Capoeira.
- Hire a maid. I'm not messy, but I want someone to do the fine tuning of my place.
- Go to
Pycon,DjangoCon, and a new conference.
- Have a beer with
Thomas,Andy, Andy, Tony, Garrick, Bernd, and the rest of Ye Aulde Gange.
Monday, December 6, 2010
Reactions to "Stupid Template Languages"
My blog post on Stupid Template Languages has had some excellently crafted responses by Armin Ronacher, Mike Bayer, and Steve Holden. I respect and admire each of these developers, and their combined projects have made a real impact on my career. I'm happy that our community is large enough to have a difference of opinion, delighted that our base language of Python allows us the power to play with different options so easily, and hope that we can debate our differences of opinion this year at PyCon. The first round of drinks are on me!
- Response of Armin Ronacher, creator of Jinja2 and other projects
- Response of Mike Bayer, creator of Mako and SQL Alchemy
- Response of Steve Holden, Chairman of the Python Software Foundation and creator of PyCon
Friday, December 3, 2010
Stupid Template Languages
For years I've been absolutely certain that I really prefer stupid template languages any time I'm generating HTML. The less the template language can do the better. Since I spend most of my time coding in Python you might assume this applies just to Python, but I think it also applies to anything where you have the power to readily mix HTML generation and code.
The biggest annoyance I have with smart template languages (Mako, Genshi, Jinja2, PHP, Perl, ColdFusion, etc) is that you have the capability to mix core business logic with your end views, hence violating the rules of Model-View-Controller architecture. While the web can be hard to match to MVC, in general you aren't supposed to do that sort of thing. I've made the mistake of putting core logic in the wrong places in the past, but I'm proud to say I've gotten good at avoiding that particular mistake.
I don't work in a vacuum.
I often work on projects crafted by others, some who decided for arcane/brilliant/idiotic reasons to mix the kernel of their applications in template function/macros. This is only possible in Smart Template Languages! If they were using a Stupid Template Language they would have been forced put their kernel code in a Python file where it applies, not in a template that was supposed to just render HTML or XML or plain text.
What it comes down to is that Smart Template Languages designers assume that developers are smart enough to avoid making this mistake. Stupid Template Languages designers assume that developers generally lack the discipline to avoid creating horrific atrocities that because of unnecessary complexity have a bus factor of 1.
So what is a Smart Template Language?
Next comes Genius Template Languages, which take things a step further. These template languages allow you to not only define functions/macros, but also let you embed unrestricted Python (or Java or Ruby or whatever) in the template. This 'feature' lets you code your entire application in the templates! In the Python world what comes to mind is Mako and Genshi, but I'm sure there are many other tools with this 'capability'.
I like Stupid Template Languages!
Stupid Template Languages don't let you define functions/macros. They don't let you embed Python code. They barely let you define variables and often have simplistic control architectures.
For Django efforts, which is about 70% of my work, I like the Django Template Language (DTL). Since it is used by a huge community, there are a ton of useful apps which have it as a dependency. Switching away from it would mean cutting myself off from a large ecosphere of tools I can use to not reinvent the wheel.
Back in my Zope/Plone days I really, really enjoyed the Template Attribute Language (TAL) because it was stupid too. If I needed an XML generation template language and could import it easily I might consider using it again, or perhaps Chameleon, which is a new, improved version . The downside is that they come paired with another tool paired with it, METAL, which gave it macros. My own experience with METAL is that it was all too easy to do what we developers do with Smart Template Languages.
But DTL and TAL are slow!
So what?
If you want to boost your performance, first try caching. There are a ton of tools you can use, with Varnish being one I keep seeing in action. Read the docs on your favorite web framework's caching engine and apply what you learn. And Djangonauts should read up on Mike Malone as much as possible.
If after all that the site still delivers slow content and it appears to be a template language issue, then identify the bottleneck content and consider alternatives for that one portion. My favorite response is a bit of AJAX. Use your framework to render the content as JSON and have JavaScript parse it into legible content, a task which JQuery makes trivial.
The biggest annoyance I have with smart template languages (Mako, Genshi, Jinja2, PHP, Perl, ColdFusion, etc) is that you have the capability to mix core business logic with your end views, hence violating the rules of Model-View-Controller architecture. While the web can be hard to match to MVC, in general you aren't supposed to do that sort of thing. I've made the mistake of putting core logic in the wrong places in the past, but I'm proud to say I've gotten good at avoiding that particular mistake.
I don't work in a vacuum.
I often work on projects crafted by others, some who decided for arcane/brilliant/idiotic reasons to mix the kernel of their applications in template function/macros. This is only possible in Smart Template Languages! If they were using a Stupid Template Language they would have been forced put their kernel code in a Python file where it applies, not in a template that was supposed to just render HTML or XML or plain text.
What it comes down to is that Smart Template Languages designers assume that developers are smart enough to avoid making this mistake. Stupid Template Languages designers assume that developers generally lack the discipline to avoid creating horrific atrocities that because of unnecessary complexity have a bus factor of 1.
So what is a Smart Template Language?
In my own vernacular, template languages that let you write functions/macros are what I call a Smart Template Language. Some of them are brilliantly executed, the example of Jinja2 comes to mind, but invariably I suffer through abuse of its Macro control structure as implemented by others.
Misery Cubed a.k.a. Genius Template Languages
Next comes Genius Template Languages, which take things a step further. These template languages allow you to not only define functions/macros, but also let you embed unrestricted Python (or Java or Ruby or whatever) in the template. This 'feature' lets you code your entire application in the templates! In the Python world what comes to mind is Mako and Genshi, but I'm sure there are many other tools with this 'capability'.
I like Stupid Template Languages!
Stupid Template Languages don't let you define functions/macros. They don't let you embed Python code. They barely let you define variables and often have simplistic control architectures.
For Django efforts, which is about 70% of my work, I like the Django Template Language (DTL). Since it is used by a huge community, there are a ton of useful apps which have it as a dependency. Switching away from it would mean cutting myself off from a large ecosphere of tools I can use to not reinvent the wheel.
Back in my Zope/Plone days I really, really enjoyed the Template Attribute Language (TAL) because it was stupid too. If I needed an XML generation template language and could import it easily I might consider using it again, or perhaps Chameleon, which is a new, improved version . The downside is that they come paired with another tool paired with it, METAL, which gave it macros. My own experience with METAL is that it was all too easy to do what we developers do with Smart Template Languages.
But DTL and TAL are slow!
So what?
If you want to boost your performance, first try caching. There are a ton of tools you can use, with Varnish being one I keep seeing in action. Read the docs on your favorite web framework's caching engine and apply what you learn. And Djangonauts should read up on Mike Malone as much as possible.
If after all that the site still delivers slow content and it appears to be a template language issue, then identify the bottleneck content and consider alternatives for that one portion. My favorite response is a bit of AJAX. Use your framework to render the content as JSON and have JavaScript parse it into legible content, a task which JQuery makes trivial.
Saturday, November 6, 2010
Release classifiers in distutils/pypi
Thanks to Doug Napoleone I'm now aware there is already a convention followed for the python and framework versions, but it appears that not enough people are aware of it. This post is pretty much a reposting of the second comment of the post immediately preceding this one and Doug gets full credit for this post. I'm just repeating his message:
The release classifiers in this post should be included in the standard distutils documentation. For the moment, you can see the full list of classifiers here:
http://pypi.python.org/pypi?%3Aaction=list_classifiers
For the python language version the classifier is:
With each version on it's own line. That way you can browse the repository by python version (see the bottom of the page):
http://pypi.python.org/pypi?:action=browse&c=214
There is also support for frameworks which you can see on that page as well. There it is done with:
There is Zope, Plone, and a number of other frameworks already there.
In the example you gave the proper, and supported way of writing the metadata is:
Now it becomes a matter of education and illumination. This should be in the standard distutils documentation and arguably the home page of PyPI (or easily found there). And Django Packages will be supporting this functionality in the near future.
The release classifiers in this post should be included in the standard distutils documentation. For the moment, you can see the full list of classifiers here:
http://pypi.python.org/pypi?%3Aaction=list_classifiers
For the python language version the classifier is:
Programming Language :: Python :: x.y.z
With each version on it's own line. That way you can browse the repository by python version (see the bottom of the page):
http://pypi.python.org/pypi?:action=browse&c=214
There is also support for frameworks which you can see on that page as well. There it is done with:
Framework :: Django :: x.y.z
There is Zope, Plone, and a number of other frameworks already there.
In the example you gave the proper, and supported way of writing the metadata is:
Programming Language :: Python
Programming Language :: Python :: 2.4
Programming Language :: Python :: 2.5
Programming Language :: Python :: 2.6
Programming Language :: Python :: 2.7
Framework :: Django
Framework :: Django :: 0.96
Framework :: Django :: 1.0
Framework :: Django :: 1.1
Framework :: Django :: 1.2.1
Framework :: Django :: 1.3
Now it becomes a matter of education and illumination. This should be in the standard distutils documentation and arguably the home page of PyPI (or easily found there). And Django Packages will be supporting this functionality in the near future.
A request for new pypi classifiers
This request is to help enhance Django Packages, PyPM Index, and other projects. This would also help the Python community at large.
Would it be possible that a standard be established for listing in PyPI classifiers which versions of a package is known to operate? Using James Bennett's django-registration at http://pypi.python.org/pypi/django-registration as an example (see my bolded, last two lines to understand what I'm trying to demonstrate):
I picked a Django package but this could be for Zope, Pyramid, PyQT, or anything.
If we had something like this in place then people could quickly identify on PyPI and other resources if a tool can be of use to them or if it needs to be updated to the latest code base. If this already exists, then can someone point me at the existing specification so I can promote it?
Edit: Noah Kantrowitz suggested I take a look at the 'requires' keyword which is part of the distutils spec. However, this does not show up in the PyPI API (http://wiki.python.org/moin/PyPiXmlRpc) and so doesn't fit our needs.
Would it be possible that a standard be established for listing in PyPI classifiers which versions of a package is known to operate? Using James Bennett's django-registration at http://pypi.python.org/pypi/django-registration as an example (see my bolded, last two lines to understand what I'm trying to demonstrate):
Development Status :: 5 - Production/StableThe metadata system I'm writing about in this blog post is specified on the distutils documentation page.
Environment :: Web Environment
Framework :: Django
Intended Audience :: Developers
License :: OSI Approved :: BSD License
Operating System :: OS Independent
Programming Language :: Python
Topic :: Utilities
Python Versions :: 2.4, 2.5, 2.6, 2.7
Django Versions :: 0.96, 1.0, 1.1, 1.2.1, 1.3
I picked a Django package but this could be for Zope, Pyramid, PyQT, or anything.
If we had something like this in place then people could quickly identify on PyPI and other resources if a tool can be of use to them or if it needs to be updated to the latest code base. If this already exists, then can someone point me at the existing specification so I can promote it?
Edit: Noah Kantrowitz suggested I take a look at the 'requires' keyword which is part of the distutils spec. However, this does not show up in the PyPI API (http://wiki.python.org/moin/PyPiXmlRpc) and so doesn't fit our needs.
Tuesday, September 21, 2010
DjangoCon 2010 report I
The conference was like a family gathering except without any oddly weird uncles. To my utmost embarrassment I got overwhelmed a few times and forgot names of people I respect and admire.
If I went through people I got to touch base with, I would have to list three digits worth of people and play favorites. That just ain't me, so I'm just going to say everyone there was awesome and I hope to see you again at Pycon!
Alright then, some reporting:
Django Software Foundation Panel
Russel Keith-Magee, the new DSF president hosted a panel with Ben Slavin, Sean O'Conner, Jeremy Dunck, and myself. Russ went over plans for DSF finances and the hopeful creation of a DSF ecosphere of applications.
My own contribution was talking a little about Django Packages, and bit more about "Why Django".
Why Django, or maybe one day "Enterprise Djangoproject" is an advocacy site for Django targeted not to developers but to decision makers. It is a work in progress, we are collecting case studies and articles, and building out the site.
Ben Slavin then challenged me about the code and data. Other Django community projects like djangoapps and djangosites are closed source efforts without an API and this causes problems for the community. If a site goes down or is unmaintained then the community loses.
I agree.
So I promised that for these Django community projects the code is public, APIs were being made to support people fetching data from them at any time, and I'm trying to figure out how to do data dumps without handing out even salted passwords. Furthermore, I would give full access to these sites and servers behind them to those that Russell appointing to the position of being watchdog.
And I'm working on that promise. Django Packages now has an API and a BDFL has server access to Django Packages. Why Django is still a work in progress, so things are still in flux. I'll post updates to my promise of openness.
Furthermore, I challenge anyone who puts together a site useful to the community, be it a new version of Django Apps or Django Sites to follow my own promise. By all means maintain and work your project, but be willing to publish all data and keep your code under an open source license. Also provide access to whom the DSF president appoints so that others can provide maintenance for your site.
Eric Florenzano's Keynote
Eric Florenzano gave the keynote on What Sucks about Django (and how we can fix it). He did a very good job of it. Watch his talk. It has some amazingly telling points.
Anyways, I'm challenging for Eric to eat his own dogfood. The only people I saw Eric talk to were the same people he is always around and conferences. How many times did he go up to a person he didn't recognize and start talking to them? How many new people did he meet at DjangoCon 2010?
Portland Views, Food, and Beer
On Saturday, as a break, I went with Eric Holscher, Ben Firschman, and Andrew Godwin to the park around the Portland zoo. Below you can see where a lady pointed out that we could see mountains 50 and 150 miles away. That left three geeks speechless while
Ben Firschman captured the moment. This picture is Ben's and all rights belong to him.
And then I had my action shot taken by Ben. Again, all rights to him.
I just have to say that Ben is an amazing photographer and I'm honored to be in a few of his pictures of DjangoCon. In addition to that he presented on class based views in Django, which I plan to blog about as well as incorporate into Django Packages in the near future.
Edit: Blogspot is acting really oddly which is why I took out the linking and other things.
Edit II: Eric ate his own dog food. Awesome!
If I went through people I got to touch base with, I would have to list three digits worth of people and play favorites. That just ain't me, so I'm just going to say everyone there was awesome and I hope to see you again at Pycon!
Alright then, some reporting:
Django Software Foundation Panel
Russel Keith-Magee, the new DSF president hosted a panel with Ben Slavin, Sean O'Conner, Jeremy Dunck, and myself. Russ went over plans for DSF finances and the hopeful creation of a DSF ecosphere of applications.
My own contribution was talking a little about Django Packages, and bit more about "Why Django".
Why Django, or maybe one day "Enterprise Djangoproject" is an advocacy site for Django targeted not to developers but to decision makers. It is a work in progress, we are collecting case studies and articles, and building out the site.
Ben Slavin then challenged me about the code and data. Other Django community projects like djangoapps and djangosites are closed source efforts without an API and this causes problems for the community. If a site goes down or is unmaintained then the community loses.
I agree.
So I promised that for these Django community projects the code is public, APIs were being made to support people fetching data from them at any time, and I'm trying to figure out how to do data dumps without handing out even salted passwords. Furthermore, I would give full access to these sites and servers behind them to those that Russell appointing to the position of being watchdog.
And I'm working on that promise. Django Packages now has an API and a BDFL has server access to Django Packages. Why Django is still a work in progress, so things are still in flux. I'll post updates to my promise of openness.
Furthermore, I challenge anyone who puts together a site useful to the community, be it a new version of Django Apps or Django Sites to follow my own promise. By all means maintain and work your project, but be willing to publish all data and keep your code under an open source license. Also provide access to whom the DSF president appoints so that others can provide maintenance for your site.
Eric Florenzano's Keynote
Eric Florenzano gave the keynote on What Sucks about Django (and how we can fix it). He did a very good job of it. Watch his talk. It has some amazingly telling points.
Anyways, I'm challenging for Eric to eat his own dogfood. The only people I saw Eric talk to were the same people he is always around and conferences. How many times did he go up to a person he didn't recognize and start talking to them? How many new people did he meet at DjangoCon 2010?
Portland Views, Food, and Beer
On Saturday, as a break, I went with Eric Holscher, Ben Firschman, and Andrew Godwin to the park around the Portland zoo. Below you can see where a lady pointed out that we could see mountains 50 and 150 miles away. That left three geeks speechless while
Ben Firschman captured the moment. This picture is Ben's and all rights belong to him.
And then I had my action shot taken by Ben. Again, all rights to him.
I just have to say that Ben is an amazing photographer and I'm honored to be in a few of his pictures of DjangoCon. In addition to that he presented on class based views in Django, which I plan to blog about as well as incorporate into Django Packages in the near future.
Edit: Blogspot is acting really oddly which is why I took out the linking and other things.
Edit II: Eric ate his own dog food. Awesome!
Saturday, August 28, 2010
New features for Django Packages
Since the Django Dash ended, the Django Packages team has been working to add new features and close out bugs.
"I Use This" added
Since we only want hard metrics on this site, we incorporated an "I Use This" button on the packages. This is so you can identify which packages you use. Please don't press this button for packages that you like, only the ones that are part of your coding efforts.
Added BitBucket support
We are still working out some of the kinks for coming up with stats from BitBucket. Most of the data we collect is fetched via the API, but a little is scraped off individual project pages.
Cache the commits
Originally the commit history was fetched live. But Github only provides the last 35 commits and BitBucket limits you to the last 50 commits. So now we store the commit history and update it nightly. Which means that the sooner you post your packages the better your commit history will look on Django Packages.
Rebuilt the package updater
Limitations on how many API calls you can make against Github (60 a minute) meant that we had to write some fun code to get around that problem. I think the problem is solved now, but I'm worried I might get to eat my words.
Added a help section
As much as we wanted a completely intuitive site, this will hopefully make it easier for people to figure out how to participate on the site.
Package Add/Edit form refactor
We completely rebuilt the Package add/edit form to make it easier to add packages. So far the response has been entirely positive.
Page cleanup and CSS Reset
We've been slowly cleaning up the HTML and resetting the CSS. Everything is looking prettier. Our goal is to make things more readable, so a lot of the changes are subtle.
Email verification works
It works, and now you get an email to confirm your account.
"I Use This" added
Since we only want hard metrics on this site, we incorporated an "I Use This" button on the packages. This is so you can identify which packages you use. Please don't press this button for packages that you like, only the ones that are part of your coding efforts.
Added BitBucket support
We are still working out some of the kinks for coming up with stats from BitBucket. Most of the data we collect is fetched via the API, but a little is scraped off individual project pages.
Cache the commits
Originally the commit history was fetched live. But Github only provides the last 35 commits and BitBucket limits you to the last 50 commits. So now we store the commit history and update it nightly. Which means that the sooner you post your packages the better your commit history will look on Django Packages.
Rebuilt the package updater
Limitations on how many API calls you can make against Github (60 a minute) meant that we had to write some fun code to get around that problem. I think the problem is solved now, but I'm worried I might get to eat my words.
Added a help section
As much as we wanted a completely intuitive site, this will hopefully make it easier for people to figure out how to participate on the site.
Package Add/Edit form refactor
We completely rebuilt the Package add/edit form to make it easier to add packages. So far the response has been entirely positive.
Page cleanup and CSS Reset
We've been slowly cleaning up the HTML and resetting the CSS. Everything is looking prettier. Our goal is to make things more readable, so a lot of the changes are subtle.
Email verification works
It works, and now you get an email to confirm your account.
Saturday, August 21, 2010
Django Dash Lessons Learned
Our experience with Django Dash 2010 was that it was an wonderful exercise in classic Django development, cowboy/cowgirl coding, and drinking copious amounts of caffeinated beverages. The result, Django Packages, is something we are happy with, are continuing to improve, and hope will improve the community.
Lesson: Fixtures are a must
Django gives you this amazing Admin control panel. As soon as you get your models in place and are entering test data, start creating fixtures. For the dash we named them initial_data.json and loaded them into the individual app directories. This meant that every time we blew away the database we got a reload with records in place. Sometimes this means you have to hand-edit JSON (or YAML if you swing that way), but the alternative is to waste time re-entering the same data again and again. Don't forget to change the names of your test fixtures before you launch!
From the command-line how to save a fixture pydanny-style:
One nice thing about fixtures is that when you do have the time/need, you can use them to help you write tests. And it makes development easier for contributors.
Lesson: Research ahead of time
In the days before the contest, we researched to see if our target repos (Github, ;BitBucket, and Google Project Hosting) each had an API and a python library to speak to that API. Github has both an API and python library, Bitbucket has an API but no library. And as far as we can tell, Google Project Hosting lacks both API and library (someone please tell me I'm wrong about Google Project Hosting lacking an API).
This meant that when we commenced coding we knew which code base to work with - we weren't trying to look up this or that random package.
We did the same thing for rendering charts.
Lesson: Get it working then optimize
Looking at some of the code makes us wince a bit. But we got it working. Now we can go back and do some code cleanup, maybe use an XML parser instead of regex to try to scrape content from PyPI, and generally feel better about ourselves.
Lesson: Plan out system architecture in advance
In retrospect it was really amusing, but the night of launch the site was serving via the Django runserver command. We were so dead tired and neither of us are crack system administrators that we did what we had to do to score the contest launch point. The next day Audrey got the site running underApache, and next week we'll be giving someone else system access to increase reliability. Next year for the contest we'll probably use something like this or get continuous integration running in the first hour.
Lesson: Don't be afraid to chat with others after the contest starts
Share your ideas, selected packages and frameworks with your competitors. The break from coding helps clear the mind and they might counter with a better idea/package/framework you can use.
Lesson: Fixtures are a must
Django gives you this amazing Admin control panel. As soon as you get your models in place and are entering test data, start creating fixtures. For the dash we named them initial_data.json and loaded them into the individual app directories. This meant that every time we blew away the database we got a reload with records in place. Sometimes this means you have to hand-edit JSON (or YAML if you swing that way), but the alternative is to waste time re-entering the same data again and again. Don't forget to change the names of your test fixtures before you launch!
From the command-line how to save a fixture pydanny-style:
./manage.py dumpdata package > apps/packages/fixtures/initial_data.json
One nice thing about fixtures is that when you do have the time/need, you can use them to help you write tests. And it makes development easier for contributors.
Lesson: Research ahead of time
In the days before the contest, we researched to see if our target repos (Github, ;BitBucket, and Google Project Hosting) each had an API and a python library to speak to that API. Github has both an API and python library, Bitbucket has an API but no library. And as far as we can tell, Google Project Hosting lacks both API and library (someone please tell me I'm wrong about Google Project Hosting lacking an API).
This meant that when we commenced coding we knew which code base to work with - we weren't trying to look up this or that random package.
We did the same thing for rendering charts.
Lesson: Get it working then optimize
Looking at some of the code makes us wince a bit. But we got it working. Now we can go back and do some code cleanup, maybe use an XML parser instead of regex to try to scrape content from PyPI, and generally feel better about ourselves.
Lesson: Plan out system architecture in advance
In retrospect it was really amusing, but the night of launch the site was serving via the Django runserver command. We were so dead tired and neither of us are crack system administrators that we did what we had to do to score the contest launch point. The next day Audrey got the site running underApache, and next week we'll be giving someone else system access to increase reliability. Next year for the contest we'll probably use something like this or get continuous integration running in the first hour.
Lesson: Don't be afraid to chat with others after the contest starts
Share your ideas, selected packages and frameworks with your competitors. The break from coding helps clear the mind and they might counter with a better idea/package/framework you can use.
Tuesday, August 17, 2010
Announcing Django Packages!
I'm part of a two person team that just launched that BETA site for http://djangopackages.com, a site that is designed to list all the Django Applications, Frameworks, and Packages created by the Django community. While there are already a few places to look for these things, it is quite easy to argue that they are challenging to navigate, don't give any hard metrics, or are woefully incomplete/unstable/closed. Our goal was to provide an attractive, easy-to-navigate, easy-to-add-data, stable site and share it with the world.
Also, this was our entry into Django Dash 2010, and was the culmination of a few days of brainstorming over paper, a lot of research, and two days of feverish coding/designing. The project was feature complete to our specifications at 5pm the second day, and the rest of the time was spent adding in UI tweaks, usability enhancements, and trying to deploy our creation.
Since then, we've cleaned up a the UI, improved the design, and got the site stable. The code is open source and on github, so fork and contribute!
Design Consideration: No 'Like' button or 'Rate my app' rating systems
We wanted hard metrics. So the package numbers are pulled from the repo sites such as Github, Bitbucket, and Google Code. Otherwise things get weighted funny. Sure, this system can be monkeyed with, but its a good metric for now. We've had suggestions from Django core developers of coming up with a quality check system, things like pypants and/or a formalized approval system.
Design Consideration: Grids
Early on we wanted to duplicate and improve upon the Django CMS Comparison page. There is also a version for Forums, but it would be nice to have a current one for blogs! In addition, recently I heard that 'tag clouds are the mullets of web 2.0'. This really struck a chord in my soul. Since we had metrics on packages, why not compare those metrics, and use those comparisons, which we call 'grids', instead of tags? In fact, we extended our idea and instead of traditional tabs we use grids in the top navigation area as seen below:
Design Consideration: Categories
The site groups packages into three categories, 'Apps' which are individual django applications. 'Frameworks' which are aggregates of apps and python modules. And 'Projects' which are implementations of Apps and Frameworks. We've thought about adding 'Tools' but weren't sure if there was anything that fit that concept, and we are leery about allowing regular Python efforts into the fold.
Design Consideration: Regex vs XML
Slurping data out of Github was easy, especially since I used python-github2. Bitbucket has a RESTful API as well that serves out JSON. I think Google Code does as well. PyPI does not and DOAP on PyPI seems to give little that is useful, so I was forced to do screen scrapes of version numbers and downloads. I'm much faster with Regex and string methods than XML juggling, and speed was of the issue this weekend. I'm not sure what benefit there is to redoing it in HTML5lib or lxml, since what I have works and appears to be stable.
Design Consideration: Leave caching and optimization for later
Besides a tiny bit of memory based template caching on the home page, there is/was no optimization. In time I plan to cache many things using a proper key/value store like redis or memcached. Perhaps not before more design and usability work is done.
Why scared of rabbits?
You wouldn't understand unless you live on the Kansas prairie.
Note: if you have any suggestions, issues, problems with Django Packages please use our issue tracker!
Also, this was our entry into Django Dash 2010, and was the culmination of a few days of brainstorming over paper, a lot of research, and two days of feverish coding/designing. The project was feature complete to our specifications at 5pm the second day, and the rest of the time was spent adding in UI tweaks, usability enhancements, and trying to deploy our creation.
Since then, we've cleaned up a the UI, improved the design, and got the site stable. The code is open source and on github, so fork and contribute!
Design Consideration: No 'Like' button or 'Rate my app' rating systems
We wanted hard metrics. So the package numbers are pulled from the repo sites such as Github, Bitbucket, and Google Code. Otherwise things get weighted funny. Sure, this system can be monkeyed with, but its a good metric for now. We've had suggestions from Django core developers of coming up with a quality check system, things like pypants and/or a formalized approval system.
Design Consideration: Grids
Early on we wanted to duplicate and improve upon the Django CMS Comparison page. There is also a version for Forums, but it would be nice to have a current one for blogs! In addition, recently I heard that 'tag clouds are the mullets of web 2.0'. This really struck a chord in my soul. Since we had metrics on packages, why not compare those metrics, and use those comparisons, which we call 'grids', instead of tags? In fact, we extended our idea and instead of traditional tabs we use grids in the top navigation area as seen below:
Design Consideration: Categories
The site groups packages into three categories, 'Apps' which are individual django applications. 'Frameworks' which are aggregates of apps and python modules. And 'Projects' which are implementations of Apps and Frameworks. We've thought about adding 'Tools' but weren't sure if there was anything that fit that concept, and we are leery about allowing regular Python efforts into the fold.
Design Consideration: Regex vs XML
Slurping data out of Github was easy, especially since I used python-github2. Bitbucket has a RESTful API as well that serves out JSON. I think Google Code does as well. PyPI does not and DOAP on PyPI seems to give little that is useful, so I was forced to do screen scrapes of version numbers and downloads. I'm much faster with Regex and string methods than XML juggling, and speed was of the issue this weekend. I'm not sure what benefit there is to redoing it in HTML5lib or lxml, since what I have works and appears to be stable.
Design Consideration: Leave caching and optimization for later
Besides a tiny bit of memory based template caching on the home page, there is/was no optimization. In time I plan to cache many things using a proper key/value store like redis or memcached. Perhaps not before more design and usability work is done.
Why scared of rabbits?
You wouldn't understand unless you live on the Kansas prairie.
Note: if you have any suggestions, issues, problems with Django Packages please use our issue tracker!
Sunday, August 8, 2010
More reasons to go to DjangoCon!
I was thinking some more about DjangoCon next month.
Capoeira - I'm known for doing one-handed cartwheels. But at last year's DjangoCon I got a chance to try my hand at Capoeira. Since then I've managed to get in some actual regular Capoeira training. My hope is to tag up with Oregon Capoeira and see how my Angola matches their Regional. Even if you've never done a cartwheel, it should still be a blast to try out!
Okay, now on to listing some more talks I'm looking forward to having at the conference:
Capoeira - I'm known for doing one-handed cartwheels. But at last year's DjangoCon I got a chance to try my hand at Capoeira. Since then I've managed to get in some actual regular Capoeira training. My hope is to tag up with Oregon Capoeira and see how my Angola matches their Regional. Even if you've never done a cartwheel, it should still be a blast to try out!
Okay, now on to listing some more talks I'm looking forward to having at the conference:
- Why Django sucks and how we can fix it - While Django is the king of Python web framewrks, Eric Florenzano is going to slam it hard. But he isn't going to just troll the conference, he's also going to provide us some possible solutions. The best thing of all, is that since Django is open source, we can all contribute to make it better!
- Pony Pwning - Django is a nicely secure framework thanks to the security focus of the community, but Adam Baldwin shows us how as developers we can make compromising mistakes. If you want to avoid being caught with your pants down on a day where you didn't wear clean underwear, go to this talk!
- State of Pinax - Pinax is a platform for rapidly building websites in Django that I've made contributions to off-and-on since about December 2008. Heck, my talks and tutorial at DjangoCon and Pycon were about Pinax! Anyway, Brian Rosner, arguably the second tallest man in the Pinax community is going to tell us the past, present, and future of this incredible tool set.
- From Slice to Site: Django in 30 minutes - Tired of hearing about how its easier to deploy a <insert-competing-but-sucky-language> site? Katie Cunningham of NASA SMD fame is going to show you how you can do it fast and easy. A great beginner talk from someone skilled at speaking and educating!
- How to sell Django - So your trying to convince the customer/boss that Django is the way to go. But they read somewhere that XML + Shell scripts is how to build an application used by millions or talk to a well-dressed salesperson selling antiquated/broken technology. How do you get past this issue? Well, I'm hosting a panel on how to convince the people who sign the checks that Django is the way to go. I've got Steve Holden, James Tauber, and either Jacob Kaplan-Moss or Frank Wiles, plus another person to discuss this on a panel.
- Large Problems in Django, Mostly Solved - Unfortunately scheduled at the same time as my own talk, my good friend Eric Holscher is giving a talk on some of the best applications written for Django. These are tools that solve a lot of problems for you. Ever wonder what is the best REST application? How to do proper searches? Migrate data? Also, want to make your pet application endorsed by the Django ecosphere? Then attend this talk!
Friday, August 6, 2010
Getting excited about DjangoCon US!
Djangocon starts in just a month. I'm looking forward to this event because of so many reasons. Lets go over some of them!
Friends - I'll get to meet with old friends and make new ones. Rather than list names I'm going to mark a sheet of paper with the alphabet and check off letters as I meet/make a friend with the first name that matches an unmarked letter. Which makes me wonder if their is a Django app for that in the making...
Portland - Great food and awesome beer at cheap prices. The wonderful thing about Portland is that the base ingredients are really good. I found I liked the simpler/cheaper things there more than fancy foods. The food carts alone are worth visiting the city.
Oregon State - It is a beautiful state and since words can't do justice here is an image:
The conference schedule rocks! - The talks they lined up all look really good. They range from the basics to the advanced, and include things that go beyond the technical. Some of my favorite picks from just the first day:
Friends - I'll get to meet with old friends and make new ones. Rather than list names I'm going to mark a sheet of paper with the alphabet and check off letters as I meet/make a friend with the first name that matches an unmarked letter. Which makes me wonder if their is a Django app for that in the making...
Portland - Great food and awesome beer at cheap prices. The wonderful thing about Portland is that the base ingredients are really good. I found I liked the simpler/cheaper things there more than fancy foods. The food carts alone are worth visiting the city.
Oregon State - It is a beautiful state and since words can't do justice here is an image:
Me at Mutnomah falls! |
- Typewar: A Case Study - James Tauber is an incredible speaker, not to mention the founder of Pinax and the CEO of Eldarion. His talks are best served live. Typewar is an interesting game but the story of its inception and maintenance is a gem for anyone looking to see how the masters do it.
- So you want to be a core developer? - Russel Keith-Magee, a core developer, lets us know what we need to know and do to work on Django core. He got approved for three talks so you know he has to kick butt as a speaker.
- Maps of Imaginary Lands - Malcom Tredennick, another core developer, shows how to use django.contrib.gis to set up fancy maps of non-existant places. One day I'll have to do this for a place I created in my youth, a place called 'Dannyland'. Anyway, Malcom also has three talks so he's got to have a lot of cool points.
- Django Forms: Tips, tricks, and ways to stay out of trouble - Shawn Rider of PBS fame might not be a core developer but he's another brilliant coder and great speaker. This is a great beginner talk and maybe a decent advanced talk too.
Thats just four great talks described. And its just the tip of the iceberg! This is going to be great!
Edit: Fixed a geography mistake. I am so embarrassed.
Edit: Fixed a geography mistake. I am so embarrassed.
Friday, July 30, 2010
Rainbows on the prairie
What is life like on the prairie? Well, under the right conditions it is like this:
About two months ago I rented a nice house a few miles outside Lawrence, Kansas. My original goal was to bike into town a few days every week. However, I'm leery of biking alone down rolling country roads so I bought myself a cheap little car. It gets me to and from Lawrence.
My fun new activity is Capoeira. It keeps me in shape and is something new and exciting for me to do. In order to support the Capoeira, I started up a local meetup. Yeah, at the extreme end of things you see some crazy acrobatics but for most people its the chance to have fun and get in shape. I should post some video online of my antics at Capoeira.
I'm also doing a bit of cooking. The kitchen in our house is really nice and I'm practicing my old culinary skills. My hope is to do a cook-off or a food camp with local friends at some point. Or maybe just relax and just cook for fun.
I tried a ballroom dance class in the area but was not happy with the quality of instruction. Also, that each class we learned a whole new dance form meant that you never really got comfortable with a single method. That said, I did learn some nice steps and hope to do them at a conference and found I really enjoy salsa.
The food here is generally cheap and good. Especially when it comes to core produce and meats. Getting some of the edgier ingredients means paying coastal prices or more. Fish and seafood seems to be hit or miss. I've heard the barbecue is outstanding but haven't been to any of the really notable places.
Life is a lot cheaper here. The dollar stretches amazingly far. My monthly rent on a really big house is a third of the cost of my mortgage or much less than what I was paying for a room in Arlington, Virginia. That means I do a few more of the things I really enjoy and also be able to save at a good rate.
About two months ago I rented a nice house a few miles outside Lawrence, Kansas. My original goal was to bike into town a few days every week. However, I'm leery of biking alone down rolling country roads so I bought myself a cheap little car. It gets me to and from Lawrence.
My fun new activity is Capoeira. It keeps me in shape and is something new and exciting for me to do. In order to support the Capoeira, I started up a local meetup. Yeah, at the extreme end of things you see some crazy acrobatics but for most people its the chance to have fun and get in shape. I should post some video online of my antics at Capoeira.
I'm also doing a bit of cooking. The kitchen in our house is really nice and I'm practicing my old culinary skills. My hope is to do a cook-off or a food camp with local friends at some point. Or maybe just relax and just cook for fun.
I tried a ballroom dance class in the area but was not happy with the quality of instruction. Also, that each class we learned a whole new dance form meant that you never really got comfortable with a single method. That said, I did learn some nice steps and hope to do them at a conference and found I really enjoy salsa.
The food here is generally cheap and good. Especially when it comes to core produce and meats. Getting some of the edgier ingredients means paying coastal prices or more. Fish and seafood seems to be hit or miss. I've heard the barbecue is outstanding but haven't been to any of the really notable places.
Life is a lot cheaper here. The dollar stretches amazingly far. My monthly rent on a really big house is a third of the cost of my mortgage or much less than what I was paying for a room in Arlington, Virginia. That means I do a few more of the things I really enjoy and also be able to save at a good rate.
Friday, July 16, 2010
Getting piston forms to play nicely with JSON
Critical Update 2012/05/10!!!
Critical Update 2012/05/10!!!
Critical Update 2012/05/10!!!
Except for a critical security patch, django-piston has been unsupported for about 3 years. That is an eternity, and the number of forks to address multiple issues is cause for alarm. Also, the original author has left the project. Because of that, in it's place at this time I recommend django-tastypie. It is up-to-date, has very good documentation, supports OAUTH, and scored second place in the Django Packages thunderdome (it got nearly 3x as many points!). Another tool to consider is Django Rest Framework, which is as good as django-tastypie but lacks the OAUTH support.
Back to the existing blog post...
A commonly used tool by Djangonauts is django-piston, which is designed to make building a REST API easier. It even works with Django forms to provide easily written PUT/POST validation, which should be pretty darn nice. Unfortunately, if you go with django-piston forms validation it doesn't accomodate the JSON (or XML or YAML) requests and if validation fails it responds in HTML. Even more unfortunate, making validation accept and return JSON with PUT/POST requests is not documented.
While one could argue that it is documented in the django-piston docstrings, in my opinion that is not sufficient.
Fortunately while working on a project for Revolution Systems we worked out a solution:
""" myapi/resource.py author: Daniel Greenfeld license: BSD This assumes your API accepts JSON only. """ import json from piston.decorator import decorator from piston.resource import Resource from piston.utils import rc, FormValidationError def validate(v_form, operation='POST'): """ This fetches the submitted data for the form from request.data because we always expect JSON data It is otherwise a copy of piston.util.validate. """ @decorator def wrap(f, self, request, *a, **kwa): # Assume that the JSON response is in request.data # Probably want to do a getattr(request, data, None) # and raise an exception if data is not found form = v_form(request.data) if form.is_valid(): setattr(request, 'form', form) return f(self, request, *a, **kwa) else: raise FormValidationError(form) return wrap class Resource(Resource): def form_validation_response(self, e): """ Turns the error object into a serializable construct. All credit for this method goes to Jacob Kaplan-Moss """ # Create a 400 status_code response resp = rc.BAD_REQUEST # Serialize the error.form.errors object json_errors = json.dumps( dict( (k, map(unicode, v)) for (k,v) in e.form.errors.iteritems() ) ) resp.write(json_errors) return resp
Usage in handlers.py:
from django import forms from piston.handler import BaseHandler from myapp.models import Article # We use our custom validate rather than piston's default from myapi.resource import validate class ArticleForm(forms.Form): """ This is best stored in forms.py but we put here for sake of clarity""" author = forms.CharField(required=True) title = forms.CharField(required=True) content = forms.CharField(required=True) class ArticleHandler(BaseHandler): allowed_methods = ('GET', 'POST', 'PUT', 'DELETE', ) model = Article @validate(ArticleForm) def create(self, request): # Create/POST code goes here. @validate(ArticleForm) def update(self, request, id): # Update/PUT code goes here.
Usage in urls.py:
from django.conf.urls.defaults import * from piston.authentication import HttpBasicAuthentication as auth # Import our ArticleHandler from myapi.handlers import ArticleHandler # Use our custom Resource class instead of piston's default from myapi.resource import Resource article_handler = Resource(ArticleHandler, authentication=auth) urlpatterns = patterns('', url( r'^articles/(?P(\d+))$', article_handler, { 'emitter_format': 'json' }, name='api_article' ), )
Of course, this assumes you are mapping Create/Read/Update/Delete (CRUD) actions to your API.
I'm interested to see other solutions people have used to handle this in django-piston, and what suggestions people have that could improve on the examples I'm supplying here.
Friday, July 9, 2010
I want to talk to Jython and Iron Python developers
I want to ask some questions of the Jython and Iron Python communities. I'm primarily looking for efforts that can go into publishable case studies. Experience in use of Django in the Jython and Iron Python environments is also very much desired. Please use my email address which is obfuscated below and can be decoded by simply running the code in a python shell:
''.join([chr(x) for x in [112, 121, 100, 97, 110, 110, 121, 64, 103, 109, 97, 105, 108, 46, 99, 111, 109]])
Tuesday, June 29, 2010
Apologies to the Python community
Apparently planet python is pulling all content off this blog to its RSS feed. Which meant a rant I wrote this morning that was completely off topic went to the entire Python community. I'm contacting the staff behind planet.python.org and asking that they point at the correct feed so this doesn't happen again.
I hate Mac & Cheese
I'm not a picky eater. I'm willing to eat every meat that is legal in the United States and wouldn't mind traveling to places where the laws are different. I have this strange desire to try pickled insects in a third world nation. I often eat things I don't like, such as eggplant.
However, the truth is that I really loathe mac & cheese. I hate cooking it, I hate eating it, I hate cleaning up after it. I've thrown away pots that have been used to cook it. I claim to be different because I despise this American staple, and silly as you may think it may be it is my preference not to have anything to do with cheesy pasta. Heck, it ain't even healthy.
Note: I sometimes like some cheese on top of tomato based sauces or chili that has pasta mixed into it. This is not the same thing as mac & cheese.
Yet the complication I face is that I have dear friends and family who think that their prize-winning homemade mac & cheese made from imported unpasteurized cheese will be something that I love. That I'll change and they'll get the chance to smugly say that it was the quality of what I've eaten that discouraged me.
That won't happen.
I've been forced to try everything from Kraft mac & cheese to gourmet cheesy pasta prepared by chefs I otherwise respect. I've had it dumped on my plate on picnics and in fancy restaurants. I've had the people proudly insist I try their recipe all while making it clear that they would take it as a personal insult if I didn't try it and would also take it as a personal insult if I didn't like it.
And you know what?
I hated it. Its vile. Its disgusting.
Please stop insisting that I try your concoction. You aren't going to change me.
However, the truth is that I really loathe mac & cheese. I hate cooking it, I hate eating it, I hate cleaning up after it. I've thrown away pots that have been used to cook it. I claim to be different because I despise this American staple, and silly as you may think it may be it is my preference not to have anything to do with cheesy pasta. Heck, it ain't even healthy.
Note: I sometimes like some cheese on top of tomato based sauces or chili that has pasta mixed into it. This is not the same thing as mac & cheese.
Yet the complication I face is that I have dear friends and family who think that their prize-winning homemade mac & cheese made from imported unpasteurized cheese will be something that I love. That I'll change and they'll get the chance to smugly say that it was the quality of what I've eaten that discouraged me.
That won't happen.
I've been forced to try everything from Kraft mac & cheese to gourmet cheesy pasta prepared by chefs I otherwise respect. I've had it dumped on my plate on picnics and in fancy restaurants. I've had the people proudly insist I try their recipe all while making it clear that they would take it as a personal insult if I didn't try it and would also take it as a personal insult if I didn't like it.
And you know what?
I hated it. Its vile. Its disgusting.
Please stop insisting that I try your concoction. You aren't going to change me.
Tuesday, June 8, 2010
My little unit test trick
This is an oldie but a goodie.
I love writing unit tests for Python code. It makes me so happy seeing the little dots go by. Add in some coverage.py and you can even make a game out of how much your code is covered. Of course, adding in Hudson just makes it even better.
However, sometimes when your unit tests get sophisticated it can be a pain to introspect via the Python shell (REPL) on one terminal shell and then go back to the unittest. Especially when the unit tests get even the least bit sophisticated. In the shell you can forget steps since you are entering things manually.
So as soon as things get the least bit complicated I simply start using the Python help() function and pdb library inside my test code. For example:
So what does this give you? Well, the help() function acts here exactly the same way it does from the Python shell. It stops the code processing and lets you do introspection. pdb lets you step through things with joy.
Try it out!
EDIT: Of course, you probably wouldn't use both help and pdb. Thats because you can call help() inside of PDB. My example just shows you available options. Thanks to Gary for pointing this out!
I love writing unit tests for Python code. It makes me so happy seeing the little dots go by. Add in some coverage.py and you can even make a game out of how much your code is covered. Of course, adding in Hudson just makes it even better.
However, sometimes when your unit tests get sophisticated it can be a pain to introspect via the Python shell (REPL) on one terminal shell and then go back to the unittest. Especially when the unit tests get even the least bit sophisticated. In the shell you can forget steps since you are entering things manually.
So as soon as things get the least bit complicated I simply start using the Python help() function and pdb library inside my test code. For example:
class MyTests(unittest): def test_pretending_to_be_complex(self): ... complex_object = really_complex_actions() ... # help demonstration help(complex_object) # PDB example cause everyone loves that too. import pdb pdb.set_trace()
So what does this give you? Well, the help() function acts here exactly the same way it does from the Python shell. It stops the code processing and lets you do introspection. pdb lets you step through things with joy.
Try it out!
EDIT: Of course, you probably wouldn't use both help and pdb. Thats because you can call help() inside of PDB. My example just shows you available options. Thanks to Gary for pointing this out!
Tuesday, May 11, 2010
Using modernizr to help with canvas
On my current project I've been using a little bit of the HTML5 Canvas element to provide a little bell/whistle. However, the problem with Canvas is that not all browsers support it. Out of the box though Canvas gives you a quick and handy way of dealing with that problem:
The problem with this approach is that if your layout expects to have an object there and your client's use of Internet Explorer doesn't include the Canvas extension then this could damage the overall feel of your layout.
And that is where Modernizr comes in to play. It is a trivial to use JavaScript library that makes it possible to detect if a browser can use Canvas or any other HTML control. So what I did was take the Modernizr Canvas detection documentation and apply it to my JavaScript. With that in hand I wrote this:
<div id="content"> <div id="demo-space-wrapper"> <canvas height="100" id="demo-space" width="100"> This text is displayed if the client browser does not support HTML5 Canvas. </canvas> </div> </div>
The problem with this approach is that if your layout expects to have an object there and your client's use of Internet Explorer doesn't include the Canvas extension then this could damage the overall feel of your layout.
And that is where Modernizr comes in to play. It is a trivial to use JavaScript library that makes it possible to detect if a browser can use Canvas or any other HTML control. So what I did was take the Modernizr Canvas detection documentation and apply it to my JavaScript. With that in hand I wrote this:
// check for canvas if (Modernizr.canvas) { // We have canvas so add a rectangle var demospace = document.getElementById('demo-space'); var context = demospace.getContext('2d'); context.fillStyle = "rgb(255,0,0)"; context.fillRect(10, 10, 10, 10) } else { // No canvas. Remove the layout space to preserve the layout. var ul = document.getElementById('content'); var li = document.getElementById('demo-space-wrapper'); ul.removeChild(li); };
Code highlighting on blogger
Thanks to Luka Marinko I have code highlighting now!
def python_funct(): a = a + b print "Hello Highlighted code" class Foo(Bar): pass
Friday, May 7, 2010
Steve Holden giving a talk on Python education
Steve Holden, Python Software Foundation chairman and all around decent guy is giving a webcast talk today at 1 pm PST (4 pm EST) about the O'Reilly school courses on Python and the upcoming O'Reilly Python certification programs. Check out the O'Reilly promotional page:
Some quick notes:
1. Steve Holden is a marvelous speaker and a great wit. Even if you don't do Python and aren't a geek its worth listen to him talk. Leaving the DC area means I won't get the chance to sit at his feet and absorb his magnificent wisdom.
2. In certain ways, I believe Python needs these kinds of certification programs. The lack of certification or any paper validation of python skills means that a number of large and conservative organizations are often hesitant to use Python. And one way to make those organizations more open to using Python is certifications. Its not the only answer, and its an answer that comes with its own set of problems, but I think that Steve and O'Reilly are a really good choice for overcoming these problems.
3. I helped Steve write the Python 1: Beginning Python. So by attending you will be supporting my work too. :)
Friday, April 23, 2010
Moving away from DC
Way back in 1969, when I was about a year and a half old, my family moved to Laurel, Maryland, which is about 20 miles north of Washington, DC. Since then, I've lived in central Maryland, Washington, and Northern Virginia. I enjoyed my friends and family, and the things that I've done.
Yet I always wanted to see the rest of our world, try new things, explore new horizons. Breathe different air, if you will. My ideal existence would be to live in a lot of different places, living six months for a time in place to place so I can get a good experience of the planet.
Not long ago conditions came together that made it feasible for me to do all this. My house is sold, my son is 18, I don't currently own any furniture, and I have a job that allows/encourages for telecommuting (and that is because of Python and Django).
So on May 5th I'm moving to Los Angeles, California for a short term stay. I'm there to hang out with a certain artist/developer and to go on a vacation that I've wanted to do for many months. More details on this move in another post.
In early June I'm moving again to Lawrence, Kansas. I've got a number of friends there, but much more important are the professional reasons. The entrepreneur for my start-up will be living not far from there, Revolution Systems offered me office space, and the cost of living is amazingly low. Another nice feature is that its in the central point of the USA, so flying anywhere on the continent doesn't take that long.
I'm very excited, but I'm also sad to go. I'll miss being near my son, my parents and siblings. I'll miss all my DC area friends, who are way too numerous to list. I will certainly miss all my students, young and old, not to mention the other teachers at OMAA.
I will be coming back in late June, and then perhaps in either late July or August. And who knows, maybe someday my travels will bring me back to the Washington, DC area for an extended stay?
Sunday, March 21, 2010
Leaving NASA
This has been a hard post to write.
This meant so much to me, and maybe because my first memories of television as a child were the moon landings of the early 1970s. I dreamed as a child of being an astronomer or astronaut, and sometimes I plot how I would redo my life to fit these dreams if I got a second childhood.
In the past year I've had some incredible opportunities present themselves to me. I've been presenting frequently on Django and Pinax. I've had the singular honor of writing course material for Holdenweb, LLC on behalf of the O'Reilly School of Technology. Representing NASA as a contractor to the Python and related communities has been both enjoyable and a great honor.
Yet all things, even good ones, must come to an end.
I've decided to become an independent consultant. My first project will be working with Revolution Systems (Jacob Kaplan-Moss and Frank Wiles) on a neat stealth project that looks very promising and once launched will help people. The project will be Python/Django/Linux based, and the client insists on accessibility, testing, and quality work. We'll be exploring the boundaries of what has been done with those tools and besides what must remain proprietary, a lot of our work will end up going back to the community. Sounds like my kind of thing!
My last day is April 1, 2010. I'm both excited to explore this new project, and saddened that my professional world for the past five years is coming to an end. Yet the overlap in technology and the participation of the NASA SMD python group in the open source world means that my work with NASA isn't coming to an end, its just transforming into something different.
Nevertheless, this is the end of an era for me.
Which is partly why I'm happy that I'll still be in touch with my fellow NASA SMD Python contractors such as Katie Cunningham, Chris Shenton, Chris Adams, James Saint-Rossy, and others. I also plan to be real friendly with the awesome Ames Research Center Python/Django/FOSS groups such as the intrepid Mark Friedenbach and the entire incredibly awesome Nebula team.
I'll miss having the pleasure of working with Leslie Cahoon, Jessy Cowan-Sharp, John Kasmark, Bob Ryan, Candace Solomon, Bill Keeter, Gamble Gilbertson, Meredith Mengel, Malik Ahmad, Jenny Mottar, Mike Brody, Virginia Butcher, Dawayne Pretlor, Michele Montgomery, Jim Consalvi,Siew Chin Hon, Hans Goetzelt, Shannon Lantzy, and many more.
Lastly, I'll miss the honor of serving civil servants such as Gretchen Davidian, Sharron Sample, and Ruth Netting and others.
Friday, March 5, 2010
Pycon 2010 report III: Sprints
My report for the sprints of Pycon 2010. This isn't a general review of the sprints, just how it affected me.
Pinax Sprint
I don't have a hard count of how many people participated on Pinax this year. Last year's sprint looked like we had a lot more, but last year the Pinax room was home to people doing other things, albeit mostly Django related stuff. In any case, this year we had probably about 10 people working on Pinax, or things that went directly into Pinax.
While James Tauber is the leader of the Pinax community, this year Brian Rosner stepped up and did an amazing job being both a technical resource and director of geeks. The mutual clarity of vision and obvious telepathy between Brian and James is truly a joy to behold.
I also appreciate the entire Pinax community. Besides coaching me on Git branches (work uses SVN so I just don't get enough Git practice) they also gracefully understood that I was a bit distracted this year.
My specific contributions to Pinax
One of the things that had bugged me about Pinax for some time were the individual project tag apps. These were a per project extension of django-tagging, and were used to simply display tagged data. I was never happy about the displays of the tags, the lack of pagination, or that you had to create/modify an entire project level application just to control a display.
So last DjangoCon I wrote django-tagging-ext. What it does is let you control the displays of tags via a root urlconf wiring. It still needs some work (cleaning a query, 100% test coverage, better documentation), but its a big step up from the alternative.
For Pycon I volunteered to go into all the Pinax projects that used tags and replace the tag_app there with django-tagging-ext wiring. I thought it would be relatively trivial, but in practice it was not.
The issue was that immediately prior to the conference, changes had been made to Pinax trunk that caused a small number of errors. All them passed existing tests but failed when you actually clicked through pages. Yeah, that does mean that Pinax code coverage needs improvement, but that is something we are working feverishly on. In any case, what that meant was that by implementing the new tag display system, I got the chance to fix a number of small but poignant bugs.
I also worked some to help the code coverage of tests. Brian Rosner worked with me and gave me some excellent pointers. I feel sad because I think we had a disconnect on what we consider pair programming, and want to assure him that the time he gave me was a high mark of the conference.
In regards to pair programming, I don't mind working with someone until you figure something out, but spending hours and hours sharing a computer drives me nuts. Once a person 'gets it', go do something else and let them go forward.
Volume of Contribution
Looking at the Pinax impact chart on github I can claim that I had the most impact during the sprint. Heck, I even jokingly claimed it on Twitter.
Except that claim is false.
The truth of the matter is that how many commits I did in a brief period is nothing compared to what Brian and James did to equip others to make commits. Or what they've done during the history of Pinax. Or what they might have done to side projects that touch Pinax. Also, during the sprints my work was really focused to a specific area of Pinax, and besides some work I did with Skylar Saveland, I worked mostly by myself.
So that claim is full of hubris and rather silly. If that claimed annoyed anyone in the Pinax community I'll buy your forgiveness with a beer.
Wednesday, March 3, 2010
Conferences are a double edged sword
This post isn't for developers, its for managers.
In the Python world once a year there is Pycon. In other languages, tools, and frameworks they all have that one big conference where all the top people go. There are classes, tutorials, scheduled talks, open spaces, keynotes, and tons of networking. All your developers will want to go, and the ones that do will rave about the time they had. So how do you as a manager handle a big conference?
Conferences are a double edged sword. If handled well they will be a huge boon to you and your organization. Handled poorly and they can be devastating to you, your projects, your team, and your organization. Knowing which way to swing the 'conference sword' in such a way that you don't get cut is a handy tool to have in your pocket.
First, remember that at the big conference there will be lots of networking. Make sure your developers are really happy with you before they go. Also have your developers go with business cards. Ask them to bring back business cards. Remind them of whatever referral policy your company might have. Why pay for a headhunter when your own staff can do the hiring for you?
Second, remember that people will be scoping out your staff for potential hire. Don't anger your developers immediately before, during, or immediately after a conference. Keep deadlines as far away as possible from the conference. Don't begrudge them leaving and task them with petty work. Because what would have been training plus an energizing break for your staff will now be considered job fair by your staff.
Third, try to cover even a portion of the costs of the conference. If your organization doesn't officially do conferences, then offer to pay for their time during the sprints just so long as that open source work even tangentially affects what they do back at the office. That way they don't use up so much leave and odds are they'll have lots of smart comrades around to help them through some of the tricky parts.
Fourth, when the developers come back, ask them what they learned. Ask if they can incorporate these lessons into current or future tasks. They'll gush about twenty new things and most of them will be inapplicable. But there will be one or two things that are a perfect match and you'll both be grinning at each other in regards to challenges now effortlessly overcome or new services you can offer.
Fifth, try to go yourself. In fact, try and organize the whole trip. If you are close enough, rent vehicles and drive down. Set up the hotel rooms. You'll be able to network with other manager types and your staff will rave about you. Hiring will become that much easier.
So lets look at two ways to handle a big conference like Pycon:
The Wrong Way
- Block any attempts the developers make in getting reimbursement for the trip.
- Schedule deadlines around the conference. If anything goes wrong, declare that this is why people should not go.
- Before the conference, grumble about developers going away and then when they come back grumble about time lost.
- Developer time lost at the conference should not be compounded by talking about the conference when they come back.
- Act surprised when your team comes back angry and at low morale levels. Act more surprised when they leave for positions elsewhere.
The Right Way
- Get business cards and fliers for your staff to bring to the conference.
- In the meeting before the conference, go over the company referral system for new hires. Also offer beer or a meal for new hires.
- At least get the developers paid hours for the sprints. Beg and borrow from your own manager to try and cover other parts of the conference. Act surprised with the gratefulness of your staff.
- Afterwards, dedicate a staff meeting to things learned at the conference.
- Go yourself and hang out in the vendor room. Hand out cheap but goofy swag. Bring thousands of business cards and hand them out. Visit a few talks. Invite developers from other companies to meals. Boast about your staff and company. Be amazed by the new hires you pick up.
Tuesday, March 2, 2010
Apologies to Katie
Katie Cunningham is a good friend of mine. We met about nine years ago through a mutual acquaintance. We had a number of similar interests all tied to general geekery and kids. I've always appreciated her honesty, humor, and work ethic. In December 2006 I got her a job at NASA, which eventually got her involved in Python and various related communities.
One thing I like about Katie is that she likes to cook and she always does so from scratch. Her focus is on a mix of various savory dishes and baking. Since she takes care to find out her friend's allergies and preferences, it is a rare moment when people do not like what she cooks. One of my absolute favorites is a blackberry cobbler that she makes, which she does not destroy with loads of sugar. She knows that when taken in quantity sugar adversely affects me and that I also love blackberries.
I'm good at entrees and appetizers and sides, but baking is beyond me unless I use a pre-mix. So I really respect what Katie does.
So anyway, recently on twitter when she was discussing bagels with Jacob Kaplan-Moss I called her out publicly for putting icing on cheesecake. As an Ashkenazi Jew from family who spent generations in the New York area (although I was raised in Maryland) I'm picky about cheesecake. Legal toppings are berries, and going crazy with toppings is having those berries in a heavy glaze.
Well, it turns out that the illegal toppings cheesecake instance was not done by Katie, but rather her mother. Katie's mom is a wonderful lady who is also a cook whose work is worth tasting and all around wonderful person. But she puts a thin layer of sour cream icing on cheesecake. While I'm sure somewhere there is a Jewish law against illegal toppings on cheesecake, Katie's mom's defense is that she is not a Jew. And I certainly appreciate her Southern style fare.
So, for what it is worth, this is my public, formal apology to Katie. She rocks and if you are smart, you'll convince her to bake for you.
Monday, March 1, 2010
Pycon 2010 report II
The first half day report for formal conference activities on February 21.
Plenary: Introduction
I helped in registration so I arrived late to listen to Van Lindburgh start the conference. Which was a shame because I like to listen to him speak. Nevertheless, I consoled myself with the knowledge that I had contributed my time and service to help him launch what turned out to be an amazing conference.
Keynote: Steve Holden
In December Steve had presented to the NOVA-Django a much earlier draft of his speech. As much as his stuff back in December was good, what he did at Pycon was right on the money. He was in fine form, and the conclusion was very much Steve Holden at his best.
The next night Steve was in rare fine form, but that is a story for another day...
Keynote: Guido Van Rossum
Being that I am Guido's biggest fan, and have trouble breathing in his presence, you might think I'm a bit biased. Alas, in this case, Guido's talk was not my favorite of the conference. If memory serves, at last year's Pycon he mentioned a desire to remove the "For Life" from BDFL and this year I think that showed a little bit. It was nice to have him field questions from the Eldarion supplied pycon.djangodose.com feed and the audience, but from Guido I guess I want vision and on-high judgements.
Leafy Chat, et al by Alex Gaynor
Alex Gaynor is great at public speaking. He is unafraid of crowds and speaks with good clarity. His talks are always informative, and this was no different. The downside was that for his talk he had 35 minutes to present a lot of information. So he didn't have the time to explore some of the technical hurdles they overcame in each effort. Nevertheless, it was informative, and by the end it seemed clear that one of the lessons re-learned was to focus on a simple core architectural design and make it work.
A Short Pinax Tutorial by Me
I've given variations of this talk at least three times previously. Once at DjangoCon, once for NOVA-Django, and once for Django-District. So I'm rather practiced at it, which turned out to be a good thing. Because, brand new good luck charm or not, this talk ended up having a host of problems.
First off, we couldn't get any Apple laptop to work with the projector. So we started 7-8 minutes late on a Windows machine. So I had to speak very quickly, especially if I was going to include the technical side of things.
So I finished the majority of slides with time to spare and then things went wrong again. The Windows machine... didn't do what was needed. Displaying Pinax and showing off its tricks just wasn't going to happen. So we ended up with an extended Q&A session.
Not my best presentation, but memorable.
In the works is me giving the talk a least a couple more times. Once with my local friends at ZPUG-DC and once at LA Django.
Managing the World's Oldest Django Project by James Bennett
James is always on the ball when it comes to presentations. His stuff is well researched and documented, and his voice is both relaxing and invigorating. It was good to hear about lessons learned, experiences, and some of the more interesting foibles they've been through.
An Underwater Python: Tortuga the Python Powered Robot by Joseph Lisee
Python powered underwater Robots? I was so excited by this talk!
Yeah, odds are it would have almost no professional applicability, but sometimes you just have to see what other people are doing with one of your favorite tools. I was impressed by the energy and creativity of the students at the Robotics group at the University of Maryland and am trying to figure out how to include robots in my next project.
Wednesday, February 24, 2010
Pycon 2010 report I
Pycon was an incredible learning experience and networking opportunity. I met many good friends again and made just as many new ones. In addition, this was the first time I presented and did so on Pinax two times. Furthermore, in the name of diversity, this instance of Pycon saw the premiere of the Financial Assistance Grant for Women. We also had a dedicated talk on Diversity as a Dependency. The benefit this focus on diversity was that...
um...
yeah...
like...
uh...
um...
Diversity Rocks
Did I learn a lot at pycon? Heck yeah. Networking was life changing. And unlike previous conferences, I'm in a position to take advantages of opportunities offered. The next few weeks and months will see a lot of changes and challenges for me.
Note: I've got to keep some things under wraps for now so I'm going to have to aggressively moderate comments. Feel free to comment, just don't take comment rejections personally.
Note II: For reference, this post is mostly about Audrey Roy and some about the job offering I got at PyCon 2010.
Note II: For reference, this post is mostly about Audrey Roy and some about the job offering I got at PyCon 2010.
Thursday, February 18, 2010
I've got ribbons!
Odds are this will be my last post on blogspot. After this it should be on my personal blog.
Anyway, I'm here at Pycon. I presented a Pinax tutorial and tomorrow I'm presenting a talk on Pinax. I'm also a session chair and plan to help with assembling things for registration. And what do I get out of it?
Ribbons.
RIBBONS!!!
My plan is to have more ribbons than anyone else.
Anyway, I'm here at Pycon. I presented a Pinax tutorial and tomorrow I'm presenting a talk on Pinax. I'm also a session chair and plan to help with assembling things for registration. And what do I get out of it?
Ribbons.
RIBBONS!!!
My plan is to have more ribbons than anyone else.
Wednesday, February 10, 2010
Eating my own Dog Food
For several years now I've hosted this blog here on blogspot/blogger. Its been both a good and bad experience. For writing out simple posts it has made things pretty easy. However, if I try to post code examples I've got to deal with the various quirks of the blog engine. How it escapes special characters and that you can't easily do color colorization has been really annoying. Yes, I know you can do some hoop jumping to make this work, but I decided a long time ago that if I had to do that then I would host my own blog.
Speaking of which, a few months back I got called out for not eating my own dog food. Yup, as a Python developer shouldn't I use something Python powered? I currently do Django professionally so those are in my tool chest so this should be trivial. Heck I also do some serious Pinax work so why not that?
Whatever I put it under, it would be a nice move. I would be able to format and control my blog to a much better degree. I could easily supply code examples. I could use the hosting to demonstrate pet projects or launch some of the things I've wanted to try for some time.
So which blog to use? Roll my own?
Heck no.
I've got a full-time job, I teach 10+ hours a week, and my consulting efforts eat up a chunk of my free time. So rather than use my energy to reinvent the wheel, I would rather rely on the hard work and labor of others.
With that in mind, I really, really like Django-Mingus. Out of the box it does everything that I want and is a breeze to get up and running. It has a large, active community and it even uses a project that has my name attached, django-wysiwyg (although Chris Adams did most of the work).
Some might ask the good questions as to why I'm not using Pinax for my new blog. In essence, I wanted to do something in the wild that wasn't Pinax powered and my concept projects all use Pinax. So don't worry, I'll be doing Pinax work for as long as I can foresee.
So I've begun working on it during the Blizzard of 2010. I'll have it up today unless we lose power or Internet.
Thursday, February 4, 2010
How I write my presentations
Tools
I tend to use just a few simple tools. I like to keep things simple because I'm focusing on my message. Complexity means I spend all my time with this or that widget and fundamentally I don't care about tools.
Google for Images
I look for images that have free/open licenses for reuse. Sometimes its hard to find something that is properly licensed, so I have to alter what the image is supposed to be.
Mac OS X Screen Capture
If I want colorized code, prettily formatted text, or simple shell displays, I use command-shift+4.
Google Documents Presentation
You really can't do anything too fancy with Google Documents, which might bother some but I find it perfect for my needs. Also, storing my drafts on Google makes a lot of sense because if Google has problems with storage then I'll be worrying about Zombies more than a presentation. And I also don't have to worry about losing a machine. Google Docs also exports to PDF or Powerpoint which is useful during the presentation or uploading to slideshare.
Also, Google Docs lets me easily share and collaborate.
NeoOffice Presentations
Actually, I don't like this tool but I use it during my long metro rides when I don't have access to the Internet. I can export from Google Docs a PPT file, edit it, and then upload later.
Methodology
How I write things out and present.
Black text on white background
For a while I played with various color schemes. I found that problematic because what displays nicely on my Mac often doesn't display prettily on the big screen. Things can become illegible. Also, keeping away from pretty color schemes means nearly every picture looks good. So why not go with the historical constant thats been used for centuries?
I am moderate in my use of bullets
I don't think bullets are evil. I do think a lot of them on the screen or reading them to the audience is evil. Bullets help trigger my dialogue. In addition, if you reference my presentation later it means my presentation actually has value. A huge set of short statements as slides doesn't do that as well, even if its more fun to do and more enjoyable for the audience.
Pictures are more fun than words
I like to mix bullets and pictures. Or just have pictures. I can speak more easily to a picture than a set of words. The picture helps me remember what I intended to say, which is nice because I hate checking notes when I'm trying to speak.
I keep my points and sentences short
This is really important. Going into recursive detail about your point and preferences bores the audience and they lose focus in what you are saying. After you make your point, move on. Or use an image to reinforce things.
Also, if you keep your points and sentences short, its easier to link things together. Not only will you find it easier to keep things in mind, but so will your audience. Pause between big statements to let things sink it. Remember, negative space is as important in a presentation as it is in art.
This doesn't mean I'm simplistic or I'm assuming that my audience is simplistic. Instead, I'm aiming for an aggregate whole effect, and I also assume that when I'm done people will be excited and racing to look at formal documentation.
No practice
Some people like to practice several times in advance and time things out. Not me. I like my presentations to have spontaneity. I watch the audience and from their visual and verbal cues alter my presentation to match. Sometimes that means I end up skipping slides or when I get to a slide its moot (so I use it to 'reinforce' my point). This works for me. I'm a spontaneous person who responds well under the conditions of speaking in public.
Post my stuff on slideshare.com
When I'm done I post my stuff publicly. People sometimes want to copy/use my material or reference it for their own sake. As long as I get the proper and legal credit I'm delighted to share. I also love feedback, positive and negative.
Which reminds me, I've got some more stuff to upload...