Learning Python

I'ld like to apologize right now for the length of this post, but there's something about someone learning a new tool that immediately helps them do something that I really enjoy.
Amy: I think I have a python problem.

Blake:	Ooh, I should be able to help. 

Amy:	Well, it's a problem that could be fixed by python.

Blake:	Close enough. 

Amy:	Ah, it goes beyond help.  I still have to figure out where to
start.  Like, do I even have python on this machine?  And how do you
read in something from a file?

Blake:	"python -v" 

Blake:	and:
 myFile = open( "filename.txt" )
 for line in myFile:
   print line 
Amy:	Holy, if comes up with a million lines of... stuff.

Amy:	But then it seems to be 2.3.

Blake:	Perhaps "python -V" 

Amy:	That's better!

Blake:	What did you want to do with the lines in a file? 

Amy:	Well, I want to take a bunch of pieces of data, like company
names and phone numbers and stuff, and stick it into a specific HTML

Amy:	So I want to take in a file of data and output HTML.

Amy:	Do I want the HTML format hardcoded into the python or should
that be another file?

Blake:	Do you have a python prompt up? 

Blake:	Try typing :
 x = "Amy"
 print "Hello %s" % x 

Amy:	Ah hah.

Amy:	That's nice.

Blake:	So, I would do something like :
 myBigTemplate = """abc %s
 def %s
 ghi %s"""
 print myBigTemplate % ('1','2','3') 

Blake:	(triple-quoted strings can span more than one line.) 

Amy:	Oh, I see.  So I set up the formatting and then use 'print' to
spit out the HTML.

Blake:	Yup.  Oh, the other thing you can do is name the variables
you're replacing.  So this works:
 x = { 'name':'Amy', 'food':'apple'}
 print  "Hi %(name)s, do you want a %(food)s" % x 

Blake:	(Just don't forget the 's' after the closing bracket.) 

Amy:	So you think I should define the HTML format in the python
script itself?  It seems easier but somehow less clean.

Blake:	Yeah, for now.  You can always change it later.  :) 

Amy:	True.

Amy:	What's wrong with this:

Amy:	myTable = " %(coname)s "

coname = "Big Developer"

print myTable % coname

Blake:	You need to pass a dictionary in if you use the (name)
feature.  So, make a dictionary of variables, like x = {'coname':"Big

Blake:	Then pass that in.  You could call your dictionary "values",
or something meaningful. 

Amy:	Ooh, that worked.

Amy:	Now I have to figure out how to get the dictionary from a
different file.

Blake:	What's the format of the file? 

Amy:	Well, I guess it could be something like "Company: Big Developer

Blake:	It could be? 

Amy:	Well, it's going to be exported from Access so I guess I could
define the format?

Amy:	It sounds like you can anyway.

Amy:	I'm going to work on the assumption you can export text in a
format like that, for now.

Blake:	Okay, although it might be easier to just assume that the
first thing on the line is the company... 

Amy:	The problem is that each company could have a varying number
of employees.

Blake:	What do you want to do in that case? 

Amy:	I want to iterate through all the names, adding a new row to
my table for each one.  I wonder if it would be possible to include
the number of names in the dataset.

Amy:	Although if they are the last thing on the line I guess you
could just go through them until you get to the EOL.

Blake:	You could, or you could repeat the company name for each employee. 

Amy:	But I don't want to repeat the company name in my HTML.

Blake:	Ahhh...  Okay, I understand.  How about:
 Big Co, "employee1, employee2, employee3", BooYeah 

Amy:	I'm not clear on the function of the "BooYeah".

Blake:	Neither am I.  It's just whatever other data you need in

Amy:	Ah hah.  The confusion is because the employees are the end of
the data.

Blake:	Oh, okay. 

Amy:	Yeah, I think if I could get a comma delimited file with
company information and employees on each line, I could parse it.
Assuming it was always correctly formatted. :P

Blake:	But if you're generating it from another program, it should be
correctly formatted. 

Amy:	It should.

Amy:	"In theory..."  but let's assume it will be.

Blake:	So try "import csv" at the Python prompt. 

Amy:	It didn't do anything.

Blake:	Sure it did.  Type "dir( csv )" or "help( csv )" to see what
it did. 

Blake:	(There's a webpage at
http://www.python.org/doc/2.3.2/lib/csv-contents.html that has more
readable contents of the help. ) 

Blake:	(And as another hint, you probably want to use the DictReader
class with a restkey of 'employee') 

Amy:	I will copy that and paste it somewhere and hopefully soon it
will mean something.

Blake:	Feel free to ask me questions about whatever doesn't make

Amy:	Hah.  Part of the problem is that you're not working for me,
and part of the problem is that I don't even know how to begin asking
the questions.

Amy:	What's a sequence?

Amy:	As in "remaining data is added as a sequence keyed by the
value of restkey"?

Blake:	It's just a list. 

Amy:	Okay.

Blake:	x = [1,2,3] is a sequence. 

Amy:	Alright.  So I can iterate through it pretty easily?

Blake:	Yup. 

Blake:	I think I was wrong in my last explanation. 

Blake:	I think what they mean there is that you'll have a dictionary
with keys of "employee1", "employee2", etc... 

Amy:	Hm.

Amy:	I guess I could work with that.

Blake:	But a good way to find out would be to try running it on a
file, and printing it out. 

Amy:	How do I call DictReader?

Blake:	No, I take it back again.  I think my first explanation is
correct.  You'ld have an entry in your dictionary with a key of
'employees', and a value of ['Bill', 'Jane', 'Ted']. 

Amy:	Do I have to define something else to be a DictReader?

Blake:	First, you create one.:
 myReader = csv.DictReader( filename, ['company','whateverelse'], 'employees' ) 
Blake:	Then, you use it :
 for values in myReader:
     print template % values 
Amy:	It's too easy!

Blake:	http://www.python.org/doc/2.3.2/lib/node549.html 

Amy:	I give it the fishy eye.

Blake:	That's the beauty of Python.  If you think it's too easy,
you're on the right track.  :) 

Amy:	Ah hah, it's giving me an error!

Blake:	What's the error? 

Amy:	NameError: name 'data' is not defined

Amy:	Where 'data.csv' is the name of my file.

Blake:	What's the line you used? 

Amy:	myReader = csv.DictReader( data.csv, ['company','address','phone'], 'employees' )

Blake:	You need to put data.csv in quotes, too. 

Amy:	It doesn't say that in the manual!

Blake:	No, that's a syntax thing.

Blake:	Hey, can I post this to the weblog? 

Amy:	Uh, sure.

Amy:	How do I get the values in myReader to just output willy
nilly?  (I don't have a template yet, I just want to see if they're
reading in right).

Blake:	print values 

Amy:	I did that but it gave me another ... prompt.  I guess my
question is actually how do I end a for?

Blake:	Just hit return. 

Amy:	Oh, that really didn't work at all!

Amy:	Here is what I got:

Amy:	{'phone': None, 'company': 'd', 'address': None}
{'phone': None, 'company': 'a', 'address': None}
{'phone': None, 'company': 't', 'address': None}
{'phone': None, 'company': 'a', 'address': None}
{'phone': None, 'company': '.', 'address': None}
{'phone': None, 'company': 'c', 'address': None}
{'phone': None, 'company': 's', 'address': None}
{'phone': None, 'company': 'v', 'address': None}

Amy:	It's kind of funny how wrong it is.

Blake:	Oh, hah!  Yes.  Read the examples, and see what's different. 

Amy:	Yes, sensai.

Blake:	(Alternately, see what the companies spell if you read them
going down.) 

Amy:	Yeah, the filename.  That's the funny part.

Blake:	So you need to get it to read your file, instead of reading
the name of your file. 

Blake:	You can do that one of two ways.  Either use "open( filename
)", or "file( filename )".  They're the same, under the hood. 

Amy:	Oh, it worked!

Blake:	It did? 

Amy:	Yeah, when I asked it to print values it gave me this: 

Amy:	{'phone': ' 416-574-8372', 'company': 'Huge Builder',
'employees': [' Bob Smith', ' President', ' Joan Simpson', '
Vice-President Public Relations', ' Huw Thompson', ' Vice President
Technology'], 'address': ' 2002 Yonge St'} {'phone': ' 416-938-2837',
'company': 'Big Buildco', 'employees': [' Joanne Jones', ' CEO'],
'address': ' 19 King St'}

Amy:	Except for some reason it reordered the variables, but I don't
think that matters.

Blake:	No, cause you'll use them in whatever order you want in your
HTML template. 

Amy:	Yup.

Amy:	Cool!

Amy:	I could get this working before your dad gets back from his
golf game!

Blake:	The only other trick will be to get the employee data out.
For that I'ld use a separate template. 

Blake:	i.e. format the employees into a table first, and then add the
'employeeTable' to your dictionary. 

Amy:	A table?

Blake:	(To do that, assuming you've got the employee's formatted into
the variable "temp", you would write:
 values[ "employeeTable" ] = temp

Blake:	An html table.  Or however else you want to add the employees. 

Amy:	Couldn't I format them after I format the rest of the stuff?

Blake:	You could, but formatting them before makes it easier to
insert them into the rest of the stuff. 

Amy:	Okay...

Amy:	The whole thing is going to have to be inside the "for values
in myReader", right?

Blake:	Mostly.  You could define your templates outside, but the
rest, yeah. 

Amy:	Okay.

Amy:	Why didn't this work:
for values in myReader:
...   print " %(company)s "

Amy:	It didn't return anything.

Blake:	Because you didn't tell it where to get the company from.
(You need the " % values" at the end of the print. 

Amy:	Oh.  So "values" is a real thing.

Blake:	At this point, I think you want to switch to a script. 

Amy:	Yeah, just a second. :)

Blake:	So that you can run it over and over again. 

Blake:	Yup, everything is a real thing.  There's very little magic in

Amy:	That's going to take some getting used to.

Blake:	Hopefully it won't be too bad. 

Amy:	Oh, I can tell I'm serious now, i have two shell windows open.

Blake:	Heh. 

Amy:	How do I do comments?

Blake:	# Like this. 

Amy:	Can I do line breaks wherever?

Blake:	Almost. 

Blake:	For now, let's say "Yes", and if you run into a problem,
you'll find out. 

Amy:	Okay.

Blake:	(And I can help you figure out where to put the break

Amy:	Oh my god.

Amy:	It worked.

Amy:	Just like that.

Blake:	Heh.  Now I'm definitely posting this to the weblog.  :) 

Blake:	What did you do for the employee names and titles? 

Amy:	I didn't do that part yet. :P

Blake:	Oh, okay. 

Amy:	I'm just excited I got the company to work.

Amy:	Now I must eat more.

Amy:	I'm running out of food.

Blake:	Heh.  I'll have you pulling your data from the live database
any second now. 

Amy:	Aiy!  Don't even say that!  Your dad would be so excited.

Blake:	It's really quite easy...  :) 

Amy:	Okay, now I'm stuck on the employees thing.  It's a list
called "employees"...  can I just do "for values in employees"?

Blake:	You can, but it wouldn't be quite what you wanted. 

Amy:	Ah.

Blake:	The quickest way I've found to get a useful list out of it is
the following line (assuming you've put the employees list into a
variable named "x": zip( [y for (i,y) in enumerate(x) if i%2==0], [y
for (i,y) in enumerate(x) if i%2==1] ) 

Blake:	But that's nigh-unreadable, so perhaps we should try to do it
an easier way, huh? 

Amy:	Holy wha?!

Blake:	See what I mean? 

Amy:	Yeah.  

Blake:	Ooh, how about this:
 names = [y for (i,y) in enumerate(x) if i%2==0]
 titles = [y for (i,y) in enumerate(x) if i%2==1] 

Amy:	First, isn't my employees list in a variable called

Blake:	Just a sec. 

Blake:	Yes, so replace 'x' with "values['employees']" 

Blake:	Or add the line:
 x = values['employees']
 before those other two bits of code. 

Amy:	And then what do "names" and "titles" end up as?  Lists?

Blake:	Yup. 

Amy:	Hm.  That's not really useful because I want to use them in
pairs, the name then the title.

Amy:	I guess I can use an index to refer to the nth item in each
list, and they shouls match up.

Blake:	Yes, but you could then write something like:
 for name,title in zip( names, titles ):
   print name, title 

Amy:	Should I look up zip or just ask you what it is?

Blake:	(zip takes two lists "[a1,a2,a3]" and "[b1,b2,b3]", and makes
a new list with both "[ (a1,b1), (a2,b2), (a3,b3) ]" 

Amy:	Oh, okay.

Blake:	enumerate (while I'm here), returns the items in a list, along
with their indices.  So you could have written:
 for i, name in enumerate( names ):
   print name, names[i], titles[i] 
Blake:	and "name" and "names[i]" should have the same value. 

Amy:	So basically I'm taking the original employee list, stripping
it into two lists, and then folding it back into a new list with a
slightly different format.

Blake:	Yeah.  An easier to use format. 

Blake:	I suppose you could do it all in one go, if you wanted...
Something like:
 for i,name in enumerate( values['employees'] ):
   if i%2 == 1:
   print "name =", values['employees'][i], " title =", values['employees'][i+1] 

Blake:	Which makes more sense to you? 

Amy:	No, I don't like doing things all in one go!

Amy:	I like doing things slowly and methodically.

Amy:	Hm.  It doesn't like "x = values['employees']"  

Amy:	It says values is not defined.

Blake:	What's your whole script look like? 

Blake:	(That line, in specific, should be in the :
 for values in myReader:

Amy:	Right.

Amy:	Well, it did something that time!

Blake:	Excellent.  Not what you wanted, I'm guessing. 

Amy:	Nope.

Amy:	But it did what I told it to do.

Blake:	Heh. 

Amy:	I have this: 
employeeRows = " %(name)s  %(title)s "

and then
  for name, title in zip( names, titles ):
    print employeeRows % name, title

Amy:	But I'm not passing in the name, title values right.

Blake:	Yes, since you're not using a dictionary, you can't use the
%(name)s format. 

Amy:	Do I just use %s>

Amy:	?

Blake:	So, you can do one of two things.  Stick with the %(name)s
format and switch to a dictionary, or switch to %s and pass them in in
the correct order. 

Blake:	Switching to a dictionary, by the way, is as easy as changing
the "% name, title" to "% locals()" 

Amy:	locals()?

Blake:	It's a link to the local variables. 

Blake:	Try putting a "print locals()" at various points in your

Amy:	So the local variables are just whatever it's working with
right now?

Blake:	Pretty much, yeah. 

Amy:	Hm.

Amy:	Now I have to figure out how to stick the employee HTML into a
variable so I can put it in the rest of the HTML later.

Blake:	What's the format of the html you want to stick it into? 

Blake:	(As a hint, instead of printing it, use += to append it to a

Amy:	Pretty much what I had there, rows in a table.

Blake:	Let me know if you need any help with that, m'kay? 

Amy:	Do I have to define variables?

Blake:	Nope. 

Amy:	Hah, that was a trick question.

Blake:	(Well, kinda nope.) 

Amy:	Traceback (most recent call last):
  File "first.py", line 26, in ?
    employeeTable += employeeRows % locals()
NameError: name 'employeeTable' is not defined

Blake:	You can't just append to something that isn't there. 

Blake:	So start it with:
 employeeTable = "<table>" 

Amy:	That's better.

Amy:	I wonder what is wrong with my brain that I never remember to
put the close quote in.

Amy:	How do I tell it to put in a newline?

Blake:	"\n" 

Amy:	Or should I just triple-quote and put it in myself?

Blake:	That would work too. 

Blake:	Whatever looks nicer to you. 

Amy:	\n looks nicer

Amy:	Okay, I think I have the bones of it working.  Now I need to
put in the real formatting.

Blake:	Cool.  Could you show me some sample output before you do? 

Amy:	Sure.

Amy:	 Huge Builder 
  Bob Smith   President 
  Joan Simpson   Vice-President Public Relations 
  Huw Thompson   Vice President Technology 

 Big Buildco 
  Joanne Jones   CEO 

Blake:	No phone number? 

Amy:	I didn't do that yet.  I just assumed it would be about the
same as the company.

Blake:	(Just making sure it's not being overwritten by something

Blake:	Yup.  It will be. 

Amy:	Actually I think I will do the """ thing for the HTML
templates, so it looks like regular HTML.

Amy:	Uhoh.

Blake:	What? 

Amy:	If a value is empty I want to leave out a row in my table.

Amy:	I will have to do that in an if in my "for values in
myReader", right?

Blake:	What do you mean by "if a value is empty"? 

Blake:	Oh, if you don't have the title for someone? 

Amy:	Well, more specifically, if the company doesn't have a suite

Amy:	If it does I want a row with the suite number, if it doesn't I
don't want that row at all.

Blake:	Yeah.  Or you could build up a sub-template, like the

Blake:	Have a line that looks like:
 values['suiteNumber'] = "<tr><td>%(suiteNumber)s<td><tr>" % values 

Amy:	Either way I will have to break everything else up into
"before Suite" and "after Suite" templates, though.

Blake:	Not really.  If you added the above line, then you could just
use "%(suiteNumber)s", and it would output the whole <tr><td> for you. 

Amy:	Oh, I see.

Amy:	What if suiteNumber is empty, though?

Blake:	Ah, yes, so you would have something like:
 if values['suiteNumber']:
   values['suiteNumber'] = "<tr><td>%(suiteNumber)s<td><tr>" % values  

Blake:	So, if it was empty, there would be no row, but if it wasn't
empty, it would get a row of its own. 

Amy:	Ah.  Okay.

Amy:	This is going to be really swell if it works.

Blake:	It will.  One way or another. 

Amy:	Uh oh.

Amy:	One of my data fields has commas in it.

Blake:	A-ha!  Did it mess up? 

Amy:	I didn't try it yet.  Should I quote the data with commas?

Blake:	You shouldn't have to.  The export thing should do it for you. 

Amy:	Shut up!

Amy:	Wait, what export thing?

Amy:	From access or whatever?

Blake:	Yeah. 

Amy:	Okay.  I'm not using real data yet, I'm just making up
fake(ish) data.

Blake:	Ah, right.  I would just assume that your real data is
correctly formatted. 

Blake:	(The rules for CSV quoting are kind of odd.) 

Amy:	If I have single quotes within a triple-quoted section, is
that okay?

Amy:	Or do I have to escape them or something?

Blake:	Yup. 

Blake:	You can also have single-quotes in a double-quoted section, or
double-quotes in a single-quoted section. 

Blake:	And you can triple-single or triple-double quote stuff, if you
needed a triple-whatever-the-other-quote-was in it. 

Amy:	Ah, wait.  I meant single-double-quote, not single quote.

Blake:	Whatever. 

Blake:	It all works. 

Amy:	Hm.

Amy:	It's whining about something.

Blake:	What's the complaint? 

Amy:	Traceback (most recent call last):
  File "second.py", line 71, in ?
    print myTable % values
ValueError: unsupported format character '"' (0x22) at index 29

Amy:	Perhaps it is the %?

Blake:	It's probably the %.  To get a % in the output, you need to
type %%. 

Amy:	Hah!

Amy:	Ta da!

Blake:	It all works? 

Amy:	Kind of, except some values aren't right. 

Amy:	But it's formatting mostly right.

Blake:	Hmm.  Cool. 

Blake:	Back in a sec. 

Amy:	It's not reading the CSV properly -- it's the problem with
commas inside fields I was talking about before.

Amy:	According to this it should work.

Blake:	No? 

Blake:	What's the line it's failing on? 

Blake:	And is this actual data, or hand-created data? 

Amy:	It's my fake data.

Amy:	Ah.  It didn't like my spaces after my commas.

Amy:	When I got rid of them it worked.

Amy:	Whoo!

Blake:	Hurray! 


Amy:	You can put that in the blog.

Blake:	Oh, I will. 

Amy:	I'm sure I would have spent way more time looking for and
downloading and installing and testing a million graphical things, if
they even exist.

Amy:	Scripting is the shit.