2007 marked the second year that 3-hour paid tutorials were offered at PyCon, and going by the feedback forms the result was an unqualified success. There were 13 tutorials (7 in the morning, 6 in the afternoon) offered by 11 instructors. Links to tutorial descriptions can be found here: http://us.pycon.org/TX2007/Tutorials.
One of the hats I wore for PyCon 2007 was "Tutorial Coordinator", and one of my duties was to collect feedback. We handed out feedback forms during the tutorials, and most attendees filled them out (thanks!). I transcribed the data from the feedback forms into a database, and present the overall results here. (I designed the tutorial feedback forms with some helpful input from others, but I take sole responsibility for ambiguities, omissions, and any errors.)
183 people registered for tutorials, 155 for morning sessions and 167 for afternoon sessions. 139 people registered for two tutorials. Most of the tutorials filled up; all of the afternoon tutorials were full. We had to turn some people away because we just couldn't squeeze any more bodies into the rooms we had. It was clear that more space for tutorials is required. At PyCon 2008 we will have more rooms and many of them will be much larger. I will write more about tutorials at PyCon 2008 at a later date.
240 tutorial feedback forms were filled out, a response rate of 75%.
Here are the questions we asked, and the results:
- Where are you staying?
240 responses: 81% of respondents stayed at the conference hotel, 13% were local residents, and 6% stayed with friends or relatives, drove in, or stayed at a different hotel.
- Will you also attend the main conference (talk days)?
239 responses: 97% of tutorial attendees stayed for the conference; 3% came for the tutorials only.
- Will you also attend any post-conference development sprint days?
237 responses: 18% of attendees definitely planned to attend the sprints, 1% said maybe, and the remaining 81% did not plan to stay for the sprints.
- Did you get your first choice of morning tutorial?
239 responses: 90% of attendees did get their first choice of tutorial; the 10% that did not reflects the fact that the tutorials filled up.
- Would you attend a second day of tutorials in 2008?
227 responses: 70% said yes, 9% maybe (or "depends on the choices"), and 21% said no.
Along with the tutorial attendance and the reported satisfaction, this is an overwhelming sign that the tutorials are a useful addition to PyCon. We haven't booked a second tutorial day for 2008, so I'm not sure whether or not we'll be able to offer this. There are logistical issues (e.g., instructors and attendees who present or attend tutorials only on the first day wouldn't want to stay an extra day). This is definitely something the PyCon organizers will discuss.
The feedback form asked attendees to rate several aspects of the tutorials, choosing from [excellent, good, poor, awful]. A couple of people thought there should have been an "OK/neutral" choice, but I left that out intentionally to force people to pass judgement, positive or negative (I personally think "OK" is a cop-out). Even so, some people wrote in intermediate answers, which forced me to change my database schema from nice clean integer answers to floats to accommodate those darned nonconformists. ;-)
- The tutorial material
234 responses: 111 excellent, 1 very good, 112 good, 2 neutral, 8 poor. 96% positive, 3% negative.
This question could have been more clearly phrased as "rate the course content".
- The handout
178 responses: 87 excellent, 1 very good, 77 good, 1 neutral, 8 poor, 4 awful. 93% positive, 7% negative.
I intended this question to be about paper handouts, but didn't take into account that some tutorials would have no handouts. In these cases some people wrote in "not applicable" or "no handout", and others chose "poor" or "awful". What was interesting is that some people chose "good" or even "exellent" when there was no paper handout at all. I can only assume (and hope!) that these people were ranking the slides or online materials.
It's obvious that handouts are expected and ought to be supplied. Although some people like to take copious notes, not everybody does. Copies of the slides (at least) or an outline of the talk is a useful reminder later on. Details would be even better.
Should the complete "script" of a tutorial be part of the handout though? For my tutorial I prepared a nearly-complete script that I worked from, but I handed out an outline omitting most of my spoken "lines". I didn't want people to read ahead and find my jokes. And some instructors don't have a script; they just work off an outline.
- The instructor, overall
237 responses: 112 excellent, 2 very good, 117 good, 2 neutral, 3 poor, 1 awful. 97% positive, 2% negative.
Although the results were overwhelmingly positive for all instructors, the negative rankings just go to show that no one instructor or approach works with everyone.
- Instructor's preparation
236 responses: 132 excellent, 1 very good, 93 good, 1 neutral, 9 poor. 96% positive, 4% negative.
- Instructor's teaching ability
236 responses: 113 excellent, 2 very good, 108 good, 3 neutral, 9 poor, 1 awful. 94% positive, 4% negative.
- The room
237 responses: 62 excellent, 1 very good, 148 good, 3 neutral, 20 poor, 3 awful. 89% positive, 10% negative.
The most common complaint in the comments was the temperature: too cold. Next was "too crowded".
While it was chilly in some of the tutorial rooms, and in the conference rooms all weekend long, it was better than the alternative: too hot. Although we'll be at a different venue next year we'll recommend that people bring a sweater or a sweatshirt.
As for the overcrowding, we were in a tough situation. We had many people sending email pleading for us to squeeze in "just one more" person in a tutorial. We tried to accommodate as many people as possible, and filled some rooms to capacity. We didn't anticipate the space that the audio equipment would take up, and the need for storage (we had scores of boxes of sponsor T-shirts!). Rest assured, these requirements will be addressed next year.
- Overall satisfaction (value for your time and money)
226 responses: 104 excellent, 1 very good, 110 good, 2 neutral, 8 poor, 1 awful. 95% positive, 4% negative.
I didn't do any fancy analysis of the data; no attempts to correlate the answers. If you have any questions that aren't answered above, please let me know.
The feedback form also asked for suggestions for future tutorial topics, and general comments. I made the resulting list of future tutorial topic ideas into a wiki page. There were many good ideas and criticisms in the general comments, which have been duly noted for next year. Most of the comments were aimed toward individual instructors, so they will not be published. The comments have been passed on to the instructors though, and I will follow up with instructors individually, noting any areas of concern or indications of a need for improvement.
Although the responses were overwhelmingly positive, I place particular emphasis on the negative responses and associated comments. Hearing praise feels good but we really learn from our mistakes and from constructive criticism. In addition, there were a few people who felt their tutorial was at the wrong level; this tells me that we need to communicate the level and pre-requisites of tutorials better.
While we couldn't please all of the people all of the time, I think we came pretty close this year. Thanks to the great feedback we got, next year should be even better!
If you have any questions, suggestions for future tutorial feedback forms, or any feedback of your own, please write to me at <goodger at python dot org> or to <pycon at python dot org>.
Please note that as I will be chairing PyCon 2008, we'll need someone to step into the role of Tutorial Coordinator (any takers? hint hint). More -- much more -- about PyCon 2008 later.