DATA COLLECTION METHODS
Questionnaires
There are several different kinds of questionnaires such as Likert scale questionnaires (where participants choose a response on a continuum)‚ checklist questionnaires (where participants check of actions they do‚ values they share‚ information they are familiar with‚ etc.)‚ and open-ended questionnaires (where participants write answers to specific open-ended questions). Likert scale questionnaires are convenient data collection instruments as they are easy to use with large numbers of participants and offer clear‚ numerical data which is easy to analyze. Questionnaires can focus on both specific knowledge about teaching and pedagogical content knowledge (for an excellent example see Kennedy‚ Ball & McDiarmid‚ 1993) or teachers’ conceptions of and attitudes towards teaching and their content knowledge (Fang‚ 1996; Horwitz‚ 1985). When questionnaires are used to investigate teachers beliefs or conceptions of language learning‚ it should not be assumed that changes in these reported beliefs directly reflect changes in how teachers conceive of and carry out their teaching (e.g. MacDonald‚ Badger‚ & White‚ 2001) as studies have shown that general beliefs and those used for actual teaching are often divergent (Foss & Kleinsasser‚ 2001; Kennedy‚ 1996; Zeichner &Tabachnick‚ 1981)
In general‚ questionnaires should not be used alone‚ but triangulated with data from other sources in order to establish the credibility of the results (Fang‚ 1996; Kalaja & Barcelos‚ 2003; Kennedy‚ Ball & McDiarmid‚ 1993; Parajes‚ 1992). If used alone‚ they should have not only abstract‚ general questions‚ but should also include specific questions on knowledge of classrooms and the teachers’ classroom behavior (Attardo & Brown‚ chapter 5; Yaakobi & Sharan‚ 1985). Care needs to be taken when constructing questionnaire items and it is always a good idea to trial items‚ check if they are really testing what you want‚ and then revise items in the questionnaire (Brindley & Schneider‚ 2002‚ Yaakobi & Sharan‚ 1985). For example‚ the questionnaire used by MacDonald‚ Badger and White (2001) and Kerekes (2001) was not designed as a questionnaire tool and does not come from any data on teachers’ beliefs‚ but was based on general ideas about language learning that Lightbown and Spada felt were important for teachers to explore in the context of SLA research (Lightbown & Spada‚ 1999). The items in the BALLI questionnaire developed by Horwitz (1985) were elicited from teachers. However‚ Horwitz only elicited general‚ context-less‚ espoused conceptions of language learning which are very different from the kinds of context-bound‚ in-action conceptions which shape teachers’ plans and actions (Woods‚ 1996). One way of avoiding this problem is to use a questionnaire to provide information about a specific policy or document. For example‚ Allen (2002) investigated the extent to which teachers’ conceptions of language teaching were similar to the standards for foreign language education. In this situation‚ revising the standards statements into questionnaire items is relatively easy. For more information on constructing questionnaire items see Converse & Presser (1986)‚ Dörnyei (2003)‚ or Fowler (1995). For more detail on the kind of knowledge questionnaires tap into and the influence of item construction on this see Tourangeau‚ Rips & Rasinki (2000) or Sudman‚ Bradburn & Schwarz (1996). The internet is also making questionnaires easier to distribute and fill out as well as to analyze the data. If you are interested in using a web-based questionnaire‚ Owen and Edwards present and evaluate an example of a web-based questionnaire‚ while Dillman (1999) discusses ways of conducting survey research online.
A variation of the normal Likert scale questionnaires is the Q-Sort procedure or Q-Methodology (McKeown & Thomas‚ 1988). The Q-Sort procedure begins with statements much like a questionnaire. However‚ instead of rating each statement independent of the other‚ participants are asked to rank the statements on a scale showing the extent of their agreement with the statements. This ranking‚ however‚ has to take the form of a normal‚ bell-shaped distribution. For example‚ in the study by Corthran and Ennis the participants:
systematically rank-ordered a series of cards... [containing] 40 statements that reflected possible values for physical education. The participants were asked to sort the cards along a 9 point continuum from the most to least valued. The number of cards allowed in each of the 9 points along the continuum was 3, 4, 4, 6, 6‚ 4‚ 4‚ 3, respectively. For example‚ the three most valued items were places on the farthest right hand column with the next four most valued items in the next column (Corthan & Ennis‚ 1998: 313).
This procedure produces data that is easy to analyze statistically (because of the bell-shaped distribution) and allows for easier comparison of the items against each other.
An alternative to Likert scale questions is to provide participants with lists of activities they do (Dunn & Shriner‚ 1999) or reasons they have for certain actions (Li‚ 1998) and have them select all relevant items. Questionnaires can also contain open-ended questions in order to collect extended qualitative data. Pennington (1996)‚ for example‚ not only had the teachers she was studying rank different aspects of their teacher education program‚ but also to comment on why they gave a certain ranking.
