The OLPC is here

I haven't been keeping with news so I was surprised and happy to find my new(est) toy on the doorstep tonight - an OLPC XO. I have been trying to put it down for several hours now so that I can take care of, well, anything else but it's fascinating. Performance wise it is not snappy but I can get on the network, download things, type in the word processor (abiword, I think) and generally fiddle around. I want to download some new apps just to see and then I am going to try and leave it alone for a while. At least overnight..


Has anyone else in Dallas (or elsewhere) received theirs?

Starting a wireless interface properly

I added WPA-PSK to my router recently (the wireless connection has been open up to now) and that immediately caused problems because my Linux laptop (running FC6) didn't want to play nice. My shiny new MacBook had no problems (no surprise there).

After some rummaging around, I found that I needed to start wpa_supplicant so I tried that using the init script but had no success. After some more poking, I edited

1. /etc/wpa_supplicant/wpa_supplicant.conf (to set the passphrase generated by wpa_passphrase)
2. /etc/sysconfig/wpa_supplicant (to change the default driver from ndiswrapper to wext

and created ifup-pre-local and ifdown-pre-local to start and stop wpa_supplicant respectively. These get called from /sbin/ifup and /sbin/ifdown and can be contain anything you want run before bringing up the wireless connection. Mine look like this:

#!/bin/bash

. /etc/init.d/functions

cd /etc/sysconfig/network-scripts
. ./network-functions

. /etc/init.d/functions

cd /etc/sysconfig/network-scripts
. ./network-functions

CONFIG=`echo ${1} | cut -d- -f2`

need_config ${CONFIG}

source_config

if [ "${TYPE}" = "Wireless" ]
then
/etc/init.d/wpa_supplicant restart
fi

Presto! My wireless interface now comes up just like it should!
Now I have to figure how to setup wpa-supplicant for when I am away from my home network..

Simple System Monitoring with Munin

I have a machine that I compile code on and it is very slow, taking 3 hours and more to build. Using top, vmstat and iostat was fine for a quick examination but I wanted something simpler to set up and with good output. I have used Nagios before and found it to be great for serious monitoring but it requires some level of effort in setting up that I wasn't ready for here so I tried a more lightweight solution, Munin.

Munin installs as either a server or a node in about 30 seconds (from rpm) or a few minutes from source. Configuration is simple and straightforward, documentation is adequate and results are just what I needed. Plugins are written in Perl and graphs are drawn by rrdtool (like nagios and cactus).

My linux machine was set up as both server and node in about 3 mins but the Solaris box was a little more tricky. After fixing a server config file foobar and node config file foobar, all was well.

Worth a look..

PlotKit and MochiKit

I needed to draw some graphs on a web page dynamically from data from a command line application so I took a look at Plotkit which, apart from having some very nice looking output, has quick methods to draw a graph from the contents of a HTML table on a page (DatasetFromTable - perfect for what I want) or the EasyPlot method that draws a chart in one line of code (or two if you count the HTML).

Poking around in there, I took another look at Mochikit which is a requirement for Plotkit. Mochikit's motto is making "javascript suck less" which it succeeds at very well. It provides the $() found in prototype and has a simple, clean set of libraries for such things as DOM handling, event handling and asynchronous event handling. It also has the javascript equivalent of Python's itertools and a logging module that can send messages to the console in Firebug.

Firebug is a Firefox extension that with the ability to inspect CSS, HTML and Javascript and a visual debugger makes web development simpler than ever.

Tips and Tricks: strings and formats

What if you want to output string data in a formatted fashion? You could use a format string in the same way that you would if writing PHP, C, Perl etc.. where %s is replaced by a string, %f is a float etc then you could end up with something like:

print "%10s is a filename"%stringvar

which would print the contents right-justified in a 10 character field. What if you wanted left-justified?
Well, then you would replace %10s with %-10s. Obvious, huh?

You could take advantage of some of the useful methods that Python strings own such as:

• rjust - presents a number or string right-justified in a field of specified width (there is an ljust as well)
• centre - presents the number or string centered in a filed of the specified field width
  
• zfill - to present a number or string of a specified width padded out with zeroes as needed

and use them like this:

print "%s is a filename"%stringvar.rjust(10)

Of course, this is only useful if you are writing to a plain text file or a terminal. What if you are writing an HTML output? You could use named parameters like this..

print "%(noddy)s is a filename of %(max_len)d characters"%(max_len, noddy)

Notice the parameters are the wrong way round in the format specifier compared to the parameter list?
This can make constructing long format strings simpler. You can take this a step further by providing a dictionary instead of a list of parameters, like this

my_dict ={ 'name':'/var/log/message', 'code':1}
print "Examining %(name)s returned code %(code)i" %my_dict

Pyglet

Pyglet is a cross-platform windowing and multimedia library for Python.. uses OpenGL and has a clean API.

I'll take a look and write it up.

Indexing the Sky

Indexing the Sky
SETI may be the Search for ExtraTerrestrial Indexes.

Rectangle classes in python

This will be useful for mapping applications.. assuming it works well. I keep hoping to find time to build something with Google maps and Python people data. Maybe this will help.

If it looks like a Duck, it may be a Parrot..

Last Saturday (July 15), the DFW Pythoneers gathered together to hear wise words from Patrick Michaud. "Lo", he said, "it is not an ex-parrot (though it may be pining for the fjords.." and we listened in solemnity.

PM gave us another great talk and explained why Perl 6 is important/interesting/intriguing to Python people and how we can get pizza paid for by the Perl foundation while looking at Python.

Parrot is the VM for the upcoming Perl6. It also happens to be capable of 'running' Python (and a bunch of other languages, some of which you will wish you had never heard of if I told you about them so I won't. You're welcome) and Patrick gave us a run-down on how the Python sourcefile in all of its beauty gets transformed into bytecode for Parrot. It's less than simple so you can go ook it up yourself at parrotcode.org.

After the meeting, I checked out the latest Parrot code and tried the test suite. Yay, it worked (good start) and then I tried the command line prompt for the Python part of Parrot, pynie. It also worked but things got a little bumpy after that:

• longs appear to be broken but aren't
• floats are broken, along with imaginary numbers
• list, arrays and dictionaries are less than working

It's a work in progress and I am not sure how to fix the BNF grammar that's written in Perl6 Regex syntax so sent Patrick an email and called it a night.

Django Flat Pages are cool

Turn your ever-growing linear website into a Django website with all that leading edge buzzword goodness!

I have been helping a friend with a website and it, like Topsy, just growed. It went from being manageably small (not too many pages) to large enough to be a problem (too many pages). Think it doesn't sound so bad? Imagine having to fix the copyright date on each page. That's not a good example because I was careful to use a standard layout.

Obviously the solution is to use templates but I didn't want to edit the content and add the template on my machine and upload everything so I wanted server-side templating (as opposed to rendering the pages client-side and uploading them, like they do for python.org).

My preference is to use Python for, well, just about any reason and then it hit me! Use Django! Or aanother framework, after all there are 80+ at the last count. But use Django! It's sexy! The voices in my head finally convinced me and I sketched my requirements:

I wanted

1. really straight-forward templating
2. WYSIWYG editing in the browser so the client can fix the content (if they want)
   
3. access control (so bad people don't do bad things)
4. a well-explained system with a user-base
   
5. ease of implementation and support (this should have been nearer the top of the list)
6. Simple urls, somehow mirroring existing URLs. Don't want to edit lots of pages.
   

The voices seemed to be right! Use Django! But how to do number 2? Using flatpages we can get a plain text version of WSIWYG but that's not I had in mind. I cast about and found TinyMCE (not hard to find, really, it's everywhere and for good reason) which is a small javascript-based editor that can beconfigured to be about as compicated as you want. Followed a recipe on the Django website and, presto chango! Nice looking WSIWYG!

After I fixed the problem where TinyMCE decided to rewrite URLs (wanted to leave that to Apache to solve number 6), it was all uphill to the finish line. Apache rewrites most URLs to send them to django, except for the fixed asset stuff (like CSS, images, forms for download etc).

What I may do, for a giggle, is have Django write a copy of the rendered page to the filesystem and server all content as static. Hm.. maybe not. What might be good is to version the pages - add a post-save so that we keep some amount of history.

Issue Number 2

I discovered cssutils tonight.. I thought there must be a parser for CSS in the Python universe somewhere.. and it's called cssutils, available from the Cheeseshop.

I installed it (using the magic of easy_install) and then broke it using the css file from HelixPlayer (the open-source portion of RealPlayer) so I raised an Issue.

Issue No. 2 as it turns out.

Unit testing.. why wouldn't you?

Unit testing is a good thing, right? Especially if it is simple to set up, simple to run and simple to read the results. How do we do it in Python? Simple!
Here's a quick example using some pieces from the unit test file I checked in along with the python bitset:

import unittest
from pybitset import Bitset # import class or module under test

class bitset_testcase(unittest.TestCase):
   def setUp(self): # testcase setup
       self.bits = Bitset(10)
      
   def testSize(self): # some method to test
                       # use assert to compare the expected and 
                       # actual results
       assert len(self.bits.bitstring)==self.bits.size(), 
           'Incorrect Size'
  
   def testRepr(self): # some other method
       assert self.bits.size() == len(self.bits.__repr__()), 
           'Repr size incorrect'
      

# .. if you need to test exceptions then write some code to 
# provoke an exception and catch the exception and add an else 
# clause to deal with exception failure
          
   def testIndex(self):
       try:
           self.bits[40]=1
       except IndexError:
           pass
       else:
           self.fail("Out of range index expected exception")

# add the check for main so you can run from the command line
if __name__ == "__main__":
   unittest.main()

Presto! Try it out, it's easy.

If you don't like my notes, go and read this

Python bitset checked in (along with unittests)

I have just checked in the python version of bitset (and unittests!) into the boost_python project directory. Now I need to spend a little time making the API for the python version agree with the C++ version. Some discrepancies around init/constructor, nothing major.

I think I have enough functionality to start on benchmarks.

The python bitset uses a list internally to store bits as, can you guess, zero or one. The first crack used the string representation of a bit rather than a numeric which was not a big problem until I noticed that all operations except init and repr needed to convert the value somewhere.

One gotcha to note: the zero-th bit is rightmost, as it would be if you were writing out a number rather than the string layout where the zeroth bit is on the left.

Pure python bitset

As previously mentioned, I recently hacked up a C++ extension to allow Python to use the dynamic bitset in the Boost libraries and I wanted to compare its performance to a pure Python implementation. Tonight, I sat down and, in about 90 minutes, hacked out a decent looking Python version.

I'll check it into the DFW Pythoneers subversion repository in case anyone wants to look at it.

Next thing to try: some sort of benchmarks to compare the two and then maybe Conway's Game of Life.

Dynamic bitset for python

I started trying the Python part of Boost C++ libraries recently and found it surprisingly easy to use. In search of a mini-project, I started coding a Python extension module to expose the Boost dynamic bitset for use in Python. It started as a novelty for a quick talk at the DFW Python group but I got stumped by adding operators.. until last night.

Here's a recap:

The constructor can accept a string ('111000') or a number of bits or a number of bits and an initial value. You can have a very large number of bits - in the millions without a problem.

The logical operators (&|^) work as expected and even throw polite exceptions when the operands are of different sizes.

There is a count method to see how many bits are on, a test method that returns true if the bit at a given index is on, a flip (or toggle) method..

There are even docstrings!

Available at:
https://python.taupro.com/repo/Projects/boost_python

It may be interesting to build pure python version and then use both to implement Conway's Game of Life to see what the difference in speed is.

DFW Pythoneers Meeting

We are winding up the current meeting and I am trying to capture what we talked about so that when newcomers ask, I don't have to scratch my head and say "um, stuff..".

As always a lively session, mainly driven by Jeff Rush, it has been a lively one with presentations and ad-hoc chats about topics like:

• the Forrester Wave work that Jeff did (and I contributed to) about the use of Python for enterprise web applications, including a simple mashup
  
• a presentation of named tuples aka nuples (something like an immutable dictionary)
  
• a recap on the game that John Zurawski developed for a 48-hour competition
• a quick look at Gizmo(QP), a python framework (yet another!)
• an RSS feed reader
• a quick peek at Pyjamas
  

.. and we had a drop-in from the local PHP group.

About Python Advocacy: Is(was) Looking For Help

About Python Advocacy: Seeking Four Code Samples for Forrester Research Survey - Jeff was looking for some help with this so I took on the mashup and came up with this: Link

I know it is simple but most of the good work in a mashup is done in javascript and I didn't want to spend too much time distracted from the purpose of demonstrating python.

I want to get around to playing with Mochikit and Pyjamas (separately or together) and see if I can dress up the sample a bit. Oh, and WSGI.. and SVG..

UTF-8 transcoder

I needed a quick program to convert from one character set to another, specifically from shift-JIS (a Japanese encoding) to UTF-8. I didn't really need the result but I did want to know if it could be *legally* converted. So here what I came up with:

#!/usr/bin/python

import codecs
import sys

print "opening file %s"% sys.argv[1]

fi = codecs.open( sys.argv[1],'r', 'shift-jis')
data = fi.read()
fi.close()

print "Writing results.txt"
fo = codecs.open('result.txt','w','utf-8'
fo.write(data)
fo.close()

I later added an option to specify the source codec.. and then after trying several japanese codecs, one after another, came up with the bulk transcoder to cycle through all japanese codecs until finding a suitable one (ie one that didn't throw an exception).

Mad scientist creates robot fish

A team from Essex university has created a robot carp that can swim around a tank with real fish and avoid collisions. It hasn't learned to feed yet and the battery life is only 5 hours.

Code and chips, anyone?

TuxDroid and D-Bus

D-bus is a local IPC protocol that allows applications on a single machine to signal each other and request or consume each other's services. Examples include the little notificiation pop-up that tells you your battery is charged.

Tuxdroid is a robot version of the linux mascot, available from www.kysoh.com. Developer pages here: www.tuxisalive.com/

Here's a tuxdroid service that lets other application speak with the mighty voice of Tux:

#!/usr/bin/python
# Borrowed from http://webcvs.freedesktop.org/dbus/dbus/python/examples/example-service.py

import dbus
import dbus.service
import dbus.glib
import gobject
import sys
sys.path.append('/opt/tuxdroid/api/python')
from tux import *
class TuxObject(dbus.service.Object):
   def __init__(self, bus_name, object_path="/Tux"):
       dbus.service.Object.__init__(self, bus_name, object_path)

   @dbus.service.method("org.dfwpython.TuxInterface")
   def Speak(self, hello_message, voice='Male'):
       print (str(hello_message))
       if voice == 'Male':
           speaker = SPK_US_MALE
       else:
           speaker = SPK_US_FEMALE
       pitch=100
       tux.tts.select_voice(speaker,pitch)
       tux.cmd.mouth_open()
       tux.tts.speak( str(hello_message))
       tux.cmd.mouth_close()


session_bus = dbus.SessionBus()
name = dbus.service.BusName("org.dfwpython.TuxService", bus=session_bus)
object = TuxObject(name)

mainloop = gobject.MainLoop()
mainloop.run()

Snakeskin Pyjamas?

I was browsing around and I came across this project.. which promises that I can "build AJAX apps in Python (like Google did for Java)". A bold claim.. I'll have to try it out and see if it really is that easy.

It's not an encouraging sign that the webpage is borked (most links appear to point to something that went away) but follow the links to download and things improve.. Google Code is lurking back there.

Tip for UPS users

Plug things in the right side!
We had a power glitch last night but we were fortunate; currently there are 200,000 homes without power in the Dallas area.

So the power went out for a second or so and the UPS didn't keep the computer alive. Why not? Because I had the PC plugged into the surge protect side, not the UPS side. Really should check these things..

Running Win2K in Qemu

I need to build a Python package for installation on Windows using distutils but I run Linux.. so I thought I'd try Qemu.

I built a disk image for Qemu using qemu-image, made an ISO from a Win2K CD and then installed win2k in a window on my main linux machine. All went well and the whole thing was installed in a couple of hours. Then I tried to run Windows Update. Boom.. gone. Hm.. maybe I'll try XP.

A quick Python recipe for a validating XML parser

ElementTree is now included in Python standard libraries from version 2.5 but as good as it is, it has no support for XMLschema validation and limited support for XPath. For that you need lxml which builds on the foundation of ElementTree.

Here's a few lines of Python to validate an XML document using a schema document using lxml:

from lxml import etree

# Parse the schema document
xsd = etree.ElementTree(file = 'schema.xsd')

# Build an XMLSchema object from the parsed document
xsv = etree.XMLSchema(xsd)

# Validate the document using the schema
doc = etree.ElementTree(file = 'doc.xml')
xsv.validate(doc)

And that's it!

If you also want to perform Xpath operations then here's a few examples:

# continuing from above

# Find all nodes with a tagname amount
nodes = doc.xpath('//amount)

# Find all nodes with a tagname amount and attribute value with value 7
nodes = doc.xpath('//amount[value=7])

# Need a namespace? Supply a dictionary
nodes = doc.xpath('//cdf:amount, {'cdf' : 'http://uri.namespace.org/1.0'})

Later: lxml uses libxml under the hood to do its magic. Apparently, there are some bugs. When trying to validate XCCDF documents, errors are generated. This forced me into actually using C++ to build a schema validator which was kind of useful, seeing as that's what I was supposed to be doing in the first place.

What programming language are you?

Ok, a little bit geeky but I saw this on Steve Holden's blog and thought it amusing. Take a little quiz and find out which programming language you are most like. I resemble Perl (and Steve is Smalltalk).. let me know if you are Python.


Which Programming Language are You?

First, pass the post

I thought I'd start a blog about my use Python and things related.. so here I am. Getting started proves to be difficult but I figure if I start typing and then redact, something might come out..

On an occasional basis (free time is more scarce since the arrival of baby Meghan), I attend the Dallas-Fort Worth pythoneers saturday sprints at Nerdbooks in Richardson.

Yesterday's meeting was covered a range of topics; a programming challenge; extending python via ctypes, pyrex, swig and boost; population simulations in Python; creating S5 presentations using restructured text; and the normal variety of odd conversation topics.