Methodology in Cognitive Linguistics
What Cognitive Linguistics actually does
Room for expansion in the use of an empirical methodology
Table 1 in Growing interest in an empirical methodology also suggests that the empirical approach, although on the rise, does not yet dominate the methodological landscape in Cognitive Linguistics. This impression may be further supported if we have a look at the program of the ICLC9 conference in Seoul, July 2005. Table 2 gives the distribution of the talks over three methodological classes: experimental research (which in this case is intended to include the use of all elicitation techniques, neurological evidence, and survey methods), corpus-based research, and traditional research. In terms of data gathering, the latter approach basically uses examples drawn from the linguistic knowledge of the researcher. The figures involve the plenary talks at the conference, and the talks from the general session, i.e. posters and theme session papers have not been included. Since the data are predominantly based on the abstracts of the talks (attending all of them would have been impossible), a certain degree of caution is necessary: it need not be the case that the actual method of research is well described in the abstract (and actual talks may differ somewhat from what is announced in the abstracts). Given this proviso, a number of interesting conclusions present themselves.
First, the figures confirm the conclusion drawn from Table 1 in Growing interest in an empirical methodology, with regard to the dominance of the traditional method: there is an outspoken interest in empirical methods, but they do not yet represent the majority of cases. Admittedly, it would be wrong to expect a 100% presence of empirical research: theory formation does require pure conceptual analysis, internal and external comparison of results, meta-reflection on methods and terminology, or synthetic overviews that do not immediately involve new research– and there will always be talks of that kind at conferences. There should be no empirical imperialism, then: one of the great things about Cognitive Linguistics is the room it has always left for creativity and the bold development of novel ideas, and it should remain that way if the approach is to keep its dynamism. However, at least half of the talks in the category labeled TRADITIONAL are not of that kind. They are descriptive talks that lend themselves to the use of an empirical methodology. In this sense, a further growth of the empirical trend in Cognitive Linguistics is definitely possible.
Second, the picture emanating from the plenaries is more conservative than that from the general session papers. The majority of the plenary talks, i.e. those of Eve Sweetser, Gilles Fauconnier, Ron Langacker, Len Talmy, Keedong Lee and Günther Radden were based on a traditional methodology of conceptual analysis, without much explicit attention for empirical data of the type discussed here. The plenary talks by Melissa Bowerman and Seana Coulson, by contrast, were firmly placed within the experimental tradition, while Suzanne Kemmer’s talk used state of the art corpus methods. Despite its promising title, John Taylor’s paper on “The mental corpus” basically used the corpus idea as a metaphor for the organization of knowledge in the mind but did not provide systematic indications how this metaphor could be linked to actual corpus research. As such, it is here counted with the traditional category.
Third, if we look more closely at the disciplines and domains of research covered by the different methodologies, we may note that experimental methods and corpus research are more firmly entrenched in some fields than in others. In foreign language learning, language acquisition research, and sign language and gesture research, for instance, experimental methods are well entrenched. This is a reflection of the importance of such methods in the fields in question at large: if experimental testing of the efficiency of a given teaching method is a standard practice in second language learning research generally, it is no surprise that cognitive linguists working in that domain will adopt an experimental methodology. Similarly, using corpora is normal in historical linguistics, language acquisition research, and text linguistics and discourse analysis. If we take this skewed distribution over disciplines into account, we may again conclude that there is room for a further expansion of an empirical methodology in what is usually thought of as the core area of linguistics, i.e. synchronic grammar.
Fourth, an analysis of the two empirical classes in Table 2 in terms of substance rather than frequency further supports the idea that an expansion of the methodological repertoire of Cognitive Linguistics is possible. To begin with, we may note that some methodological formats are relatively underrepresented. The experimental techniques that are being used predominantly involve elicitation in the form of production and comprehension tasks, plus some decision, association, and categorization tasks. Sophisticated methods like eye tracking or FMRI and other neurological imaging techniques are used only occasionally. This also holds for survey techniques in the form of interviews and questionnaires, or for direct observation as usual in some forms of sociolinguistics and anthropological linguistics. The corpora of observed language use that are being investigated are very often existing corpora like the British National Corpus, or readily available data like the ones you can get by googling, but compiling corpora with a specific purpose (like a corpus of advertisements or newspaper headlines) does occur. At this point, the borderline between elicitation and observation tends to fade a little. When a writing task takes the form of writing whole texts or narratives, the production task borders on “corpus elicitation”: pooled together, the stories of the participants constitute a mini-corpus. Overall, then, the range of empirical methods in Cognitive Linguistics is broad but rather skewed: a further expansion of the spectrum of available methods may be envisaged.
Fifth, and finally, that there is an opportunity for further progression along the empirical path also becomes apparent if we zoom in on the type of corpus analysis that we find in the corpus-based category in Table 2. There may be a lot of corpus use around, but does it live up to the general criteria for empirical research that we formulated? (Incidentally, the present paper tends to pay more attention to corpus-based approaches in empirical linguistics than to experimentation and the other relevant techniques, but that is a question of convenience rather than principle: more about experimentation may be found in Gibbs, this volume.)
In general, corpora appear to be often used as a simple data gathering technique and not as a empirical testing ground for operationalized hypotheses, that is to say, in a large part of the cases in the CORPUS-BASED category in Table 2, the corpus is primarily a repository of examples that are then analyzed in a traditional way. The examples you analyze are no longer derived from your own knowledge of the language, but the way in which you analyze them is still a predominantly intuitive one. This is the type of corpus use that Tummers, Heylen and Geeraerts (2005) call corpus-illustrated research. In a sense, it considers usage events as a data set for the selection of examples. The presence of an expected pattern in the corpus data is then so to speak interpreted as a grammaticality judgment, albeit to some extent a more reliable one than the usual, introspectively derived ones. Although corpus-illustrated work constitutes a big step forward in comparison with introspective work, there are two points with regard to which a more advanced way of dealing with corpus materials is possible, and indeed necessary. Both points, needless to say, link up with the basic characteristics of an empirical approach.
A first point concerns the systematicity with which the data are gathered and/or analyzed. Typically, the argumentation in corpus-illustrated research takes one of two forms. When a theory predicts a phenomenon, the corpus presence of the phenomenon is taken as a confirmation of the theory. When a theory excludes a phenomenon, the corpus presence of the phenomenon is taken as a falsification of the theory. In both cases, however, a problem arises when the frequency of the selected corpus data is low: what if the confirming or falsifying data are marginal with regard to the alternatives? Because corpus evidence is seldom a question of black or white (of one hundred percent versus zero percent of occurrences), a proper analysis of corpus materials requires that alternative patterns are systematically explored and that their frequencies are statistically evaluated. Grammatical models like Cognitive Linguistics and Construction Grammar in particular, which accept the existence of networks in which elements or levels may have different degrees of salience, should be particularly sensitive to differences in the frequency of alternative expressions. That is to say, a usage-based linguistics needs quantification and statistical analysis.
A second point with regard to which a more advanced form of corpus analysis may be envisaged is the interpretation of the corpus data. In a corpus-illustrated approach, the corpus data are used to restrict a process of intuitive interpretation: examples of a linguistic phenomenon (say, a construction) are retrieved from the corpus, and the set of examples is then analyzed largely intuitively. The availability of large corpora may allow for a more advanced approach. A given interpretation may be turned into a set of expectations with regard to the behavior of the entity under investigation, and these expectations may then be tested against the corpus. That is to say, advanced corpus research needs to pay specific attention to the oprationalization of hypotheses. If we interpretatively assume that a linguistic entity has this meaning or that function, what can we expect about its observable behavior in actual language use? The further steps for a corpus approach follow logically: how can we identify that behavior in the corpus data, and when can we say that the initial hypothesis is correct?
Within the broad domain of Cognitive Linguistics, a corpus-based methodology as meant here may be found in various fields (the references are exemplary, not exhaustive): in language acquisition research (Tomasello 2003); in language change research (Bybee 2001; Krug 2000); in language variation research (Grondelaers, Speelman and Geeraerts 2002; Speelman, Grondelaers and Geeraerts 2003); in functional discourse analysis (Sanders and Gernsbacher 2004), and in quantitative construction grammar (Gries 2003; Stefanowitsch 2003; Stefanowitsch and Gries 2003; Tummers, Speelman and Geeraerts 2005).
But if we take the Seoul conference as representative for Cognitive Linguistics as a whole, this type of advanced corpus research represents only a small minority of the CORPUS-BASED category in Table 2: not more than 10 or 15% of the corpus-based papers in the General Session would seem to move beyond the corpus-illustrated level as defined above. It is here in particular that the raw figures in Table 2 appear to be somewhat deceptive: yes, corpus materials are being widely used, but from the point of view of an “empirical revolution” in linguistics and the characteristics of genuinely empirical research, the sophistication with which they are being analyzed could still be improved.
In addition, note that the dichotomy between corpus-illustrated and more advanced forms of corpus-based linguistics obviously only defines the endpoints of a continuum. A lot of case studies occupy an intermediary position between both extremes. In particular, studies that do not yet use quantitative forms of analysis may display a lot of variation with respect to the systematicity with which the empirical data are collected and scrutinized, ranging from an anecdotal selection of examples from a corpus to a systematic scanning of a corpus to uncover the patterns and uses of a linguistic phenomenon. Good examples of what may be achieved with a thorough but not specifically quantified examination of corpus materials include Deignan and Potter (2004), Heyvaert (2003), Maldonado (2002)– and of course, there are traditional branches of linguistics, like historical lexicography, that testify to the value of in-depth qualitative analyses of large-scale corpus data (cp. Geeraerts 1997). But just like lexicography has, in recent decades, fruitfully enriched its traditional corpus-based approach with quantitative methods of information retrieval, it is to be expected that theoretical linguistics too might profit from further sophistication in the analysis of corpus data. The existence of a continuum, then, should not surprise us. If the spread of empirical methods in Cognitive Linguistics is a gradual process, it is normal that there will be researchers, and areas of research, that have progressed further in the empirical cycle than others.
