Levis, J. (2011). Assessing speech intelligibility: Experts listen to two students. In. J. Levis & K. LeVelle (Eds.).
Proceedings of the 2
nd
Pronunciation in Second Language Learning and Teaching Conference, Sept. 2010.
(pp. 56-69), Ames, IA: Iowa State University.
ASSESSING SPEECH INTELLIGIBILITY: EXPERTS LISTEN TO TWO STUDENTS
John Levis, Iowa State University
In order to use intelligibility as a guideline for assessing pronunciation, we need to
understand how judgments of speech intelligibility are made, and to what extent
judgments are consistent across different raters. This panel discussion asked English
pronunciation teaching experts from around the world to listen to the free speech and read
speech of a Spanish and Korean native speaker and to evaluate what features in their
English most impacted their intelligibility. The discussion touched on a variety of
themes, four of which are discussed: listener specific factors, identification of features
that seemed to be important, the scapegoating of foreign accent, and the consequences of
using read speech and free speech in assessing intelligibility.
The 2
nd
Pronunciation in Second Language Learning and Teaching Conference ended with an
interactive discussion about intelligibility based on recordings of the reading aloud and free
speech of two students, one from South Korea and one from Colombia. A panel of expert
participants at the conference were asked to listen to the recordings of the students, discuss and
take questions from the audience. The panel consisted of Bertha Chela (Venezuela), Tracey
Derwing (Canada), John Levis (US), Greta Muller Levis (US), Murray Munro (Canada), Marnie
Reed (US), Joanna Smith (New Zealand), Brian Teaman (Japan), and Beth Zielinski (Australia).
The participants were asked to participate because of their expertise in teaching pronunciation.
First, all participants (including the audience) listened to the speech of two graduate students: A
native speaker of Spanish from Colombia and a native speaker of Korean from South Korea.
Each of these students had previously been recorded answering a set of questions in an interview
format (which we will refer to as “free speech”) and in reading a passage specially constructed to
help to diagnose a wide variety of pronunciation errors (Celce-Murcia, Brinton, Goodwin &
Griner, 2010). We will refer to this as “read speech.” The reading passage and the questions are
found in the Appendix. Each of the students had previously been recorded when they had taken
part in an optional pronunciation tutoring component of an ESL methods class on the teaching of
oral language skills. These two students’ recordings were chosen deliberately for this exercise.
First, they both had large differences in their performance on the free speech and read speech
portions, with the Spanish speaker being fluent and relatively easy to understand in free speech
and fluent but hard to understand in read speech. The Korean speaker, on the other hand, was
halting in free speech and had trouble understanding the questions, but he seemed to read aloud
much more clearly. Second, they came from very different L1s which may or may not have been
familiar to panelists. Finally, they both demonstrated a wide variety of pronunciation problems
that could be candidates for instruction.
All participants listened to the free speech followed by the read speech of each student. None of
the listeners had heard the recordings before. Everyone was provided with an unmarked copy of
the reading passage for each student and with the questions the students were asked in the
John Levis Assessing Speech Intelligibility
Pronunciation for Second Language Learning & Teaching 57
interview. Each recording was played once (for time’s sake), and panelists and audience
members were asked to identify the features of the speech that most influenced the speaker’s
intelligibility or lack thereof. The task was quite difficult (even for the experts). Listening once
was not felt to be enough, and everyone thought that another opportunity to listen would have
been helpful. The panel discussion was video-recorded and transcribed, and themes were
identified from the discussion. Four themes will be discussed in this paper: The listener’s
dilemma, features thought to affect intelligibility, the scapegoating of pronunciation and accent,
and the value of reading vs. speaking freely in diagnosing pronunciation. Finally, some
conclusions about using intelligibility are offered based on the panelists’ discussion.
THEME 1 – THE LISTENER’S DILEMMA
It quickly became clear during the discussion that everyone was not equally effective listening to
both students. All listeners face challenges when speech is not packaged in expected ways, and
the greater the mismatch in expectations, the more trouble listeners had. These mismatched
expectations included things as mundane as not being able to clue in to the rhythmic structure,
stress patterns or vowels used by the students. They also included getting lost in trying to
understand and panicking, and a lack of familiarity with a particular accent.
Several panelists talked about having trouble with the Spanish speaker because the student’s
speech didn’t sound right. This usually took the form of comments about the rhythm or tempo of
his speech, which showed up in that he did not seem to distinguish between the length of stressed
and less stressed words very much. He was very fluent in both free speech and read speech, and
a number of panelists found that he was hard to follow because his fluency did not match their
ability to listen to him. This is how several panelists described what they thought was the source
of their difficulties.
Greta: The part in the free speech that I had trouble with and I think Beth did as well,
maybe like you said Murray, there were a couple of phrases that just completely lost me.
And we are guessing here that it was a lack of, that there were probably many syllables
that…seemed all the same to us, and Bertha’s idea that there was a lack of de-
emphasizing the stressed pieces in these phrases so that the prominence did not come out.
Murray: And my experience that was it was a kind of tempo problem. So maybe we're
talking about more or less the same thing. And I think I noticed that more in the reading
passage, though. But it seemed to me that there were sometimes when he would be sort of
going in a normal rate and then all of the sudden there would be a 'bada bada baa." A
rapid burst and I was just lost.
Greta: Yea, that's what we thought.
Later, the same theme came back again with an additional feeling attached to it: panic. This
reaction to having trouble understanding is not unusual in situations where listeners have to
understand and find they cannot. In this case, the experts were asked to be up front because they
were experts, and they were on the spot. They were supposed to be able to judge which pieces of
the student’s performance were causing the biggest problems, and they found themselves not
able to get a handle on the speech patterns. This is how they expressed this feeling.
John Levis Assessing Speech Intelligibility
Pronunciation for Second Language Learning & Teaching 58
Beth: Well, whenever I can’t understand anybody and can’t get any English out of it,
from experience, I think it's because I can’t latch on to anything and understand and quite
hear any of the differences between strong and weak syllables. And so that’s my
underlying premise: If I can’t hear any English at all, I’m not hearing any words, I can’t
hear any word boundaries. So I think that prominence thing, the difference between weak
and strong syllables is really important. And I also felt that he was pausing but in the
wrong places, and so he’d go,” dadadada.. dadadada”. That was my impression of it, but
maybe I was just panicking because I could not understand. I was thinking, “how am I
going to analyze this, I can’t understand it.”
Bertha: That was funny. The same thing happened to me with Korean. I could not
understand anything and I just switched off, you know, I was, “My God” (laughter).
To remedy this, one panelist thought that the student could just be taught how to slow down a bit,
so the listener could catch up.
Joanna: I think one more thing that we see about him. Because he was reading quite fast
that when he did make a mistake, as some of us know, I was just trying to figure out what
he said and I lost the next bit. If he had paused, like I said, at the end of the sentence and
took a breath, it would have given us time to catch up with him.. ok.. and carry on. And
that's something that is kind of quite easy to teach, even without changing any of his
pronunciation, just pausing so people can catch up.
One of the audience members disagreed about the effectiveness of this remedy.
Audience Member: Well…I’m a teacher, so, I mean, just for our speaker…I used to teach
him. I know him. So I was like, “Oh, that’s so good. I like it.” I understood every word he
said…And I remember having a conversation with him…trying to get him to slow. I
mean. I tried working with him…but you know, just trying to get him to slow down. But
it’s easier said than done. How do you get people to slow down? I think that’s what he
really needs to do, you know…how do you get people to slow down?
The listener’s dilemma includes one more point that is of interest. Beth, from Australia,
admitted that she simply did not have much familiarity with Spanish–accented English. The
North American listeners, on the other hand, found this particular accent very familiar. Similarly,
Bertha, from Venezuela, found the Korean speaker’s accent and errors unfamiliar, and as she
said, she simply switched off. It appears that familiarity with particular accents and patterns of
errors helps listeners to listen more effectively, and lack of familiarity can more easily lead to
panic, switching off, or even hostility. Fortunately, it is possible to become more familiar, and it
doesn’t take years of practice to do so, as Derwing, Munro and Rossiter (2002) found in training
social work students to interact with Vietnamese accented clients. However, the experience of
these experts indicates that losing the connection to a speaker’s message can result in panic or
switching off, even for the most open and well-intentioned listeners. For listeners who are not so
well-intentioned or open, reactions may be much more negative. Listeners are not all created
equal, nor are their reactions to not being able to understand.
John Levis Assessing Speech Intelligibility
Pronunciation for Second Language Learning & Teaching 59
THEME 2 – WHAT SEEMED TO AFFECT INTELLIGIBILITY
The panel was asked several times to address the question of what impacted intelligibility the
most. Judy Gilbert’s image of pronunciation teaching as triage was raised to ask the panel about
what they would work on if they had limited time and wanted to make the biggest difference for
these students’ effectiveness in speaking English. What was clear was that even the experts do
not completely agree on what most impacted intelligibility.
One of the things that was agreed on had to do with the Korean speaker, who was very halting in
his free speech and often did not answer the basic interview questions in a way that showed that
he understood what was being asked. Using a twist on Hinofotis and Bailey’s (1981) idea that
there is a threshold level of pronunciation that is necessary before a speaker can be intelligible,
panelists felt that the Korean speaker did not demonstrate a threshold level of general speaking
or listening ability, and that working on his pronunciation would not be effective until his overall
speaking and listening proficiency increased. This discussion is clearly related to Suzanne Firth’s
(1992) argument for a curriculum that focuses first on the big picture of general speaking and
listening, what she called the Zoom Principle.
John: Let’s talk about the Korean real quickly because he’s a very different case
especially in the reading passage, you probably could have understood many things even
without the text in front of you. So what are your feelings about his real difficulties and
what you would prioritize?
Joanna: Well, my first impression is obviously he does not have very much experience
speaking, either producing or listening because he did not understand the questions. He
was completely misunderstanding them. So. And I was very surprised at how much he
was able to actually read these things. Because I thought he's not going to have a chance.
But he, I think he knew a lot of these words, so he must have a lot of exposure to reading
and writing. But some of his pronunciations that we noticed like the word
“proNOUnciation” [aʊ] are things that just once he noticed how they're actually he said –
he could probably easily say pronunciation, but he just doesn't know what things sound
like – so my suggestion was I would probably put him on a, like a graded reader,
something that is fairly simple that had audio files. You could be looking and listening at
the same time, just actually get exposure to what words sound like when they’re said out
loud.
Greta: I had a kind of similar reaction that the problem in free speech was so much not
just pronunciation that the other things had to be addressed. That he couldn’t.. I couldn’t
fairly just say, “let's work on your pronunciation”, that we have to say, “how can we
work on your pronunciation in addition to your grammatical and vocabulary abilities”.
It’s just they have to be integrated together for this person.
Tracey: Yeah, I agree. His comprehension was really weak and he really needed a lot of
help with speaking and listening.
Greta: Well his speaking in the free speech piece was even more word-by-word than in
the reading, and it was just teeny-tiny bits at a time…I think in ideas and phrases rather
John Levis Assessing Speech Intelligibility
Pronunciation for Second Language Learning & Teaching 60
than one word ideas.
Brian: It reminded me of a common problem I have with my Japanese students that, well,
there’s a general rule that you can’t change something unless it’s moving, right? You
can’t steer a car that is standing still. And I feel like with students, the student that can’t
even produce, you know, sentences, more than two or three words at a time. You really
can’t do anything with them until they’re actually getting language out there for you to
work on. That would be something to, I do not know if you can call it fluency, whatever.
Something, actual speech, actually speaking
Someone: His mouth moving
John: I was reminded of your [addressed to Tracey and Murray] talking about the
Mandarin speakers [in your research] who are kind of in a tight-knit community, and I get
the feeling that this student is also in that situation. He’s married. His spoken English
when he came here was not very good. So, and he’s probably in a field where he can get
away with not speaking very much, but we think what he obviously needs to do is speak.
And I thought, well he’s married so you can’t tell him to get a girlfriend. You know, the
idea that he would get somebody that would give him input. But I remember that he also
had this sense that his company had sent him here and he had to work on his studies only
and he did not have time to go out and practice.
Tracey: Well, this is a huge problem for a lot of international students because if their
language levels aren’t quite up to where they should be, they’re not going to go and get
extra practice because…everything takes longer. So studying, getting their homework
done. All of that is going to take longer than for anybody else. And so the last thing
they’re going to do is sign up for a conversation class or go out with some of the people
in their class or a social event or anything like that. They’re just not going to do it
because they have to get that assignment done or study for that test that’s coming and so
they’re experiencing a double whammy.
Audience Member: Maybe in those kind of cases, what we could do is use the work to
kind of work on those skills. So if he knows he’s going to have a presentation, then we
can work on the presentation. If he knows he’s going to have to read a certain amount of
material and understand it, maybe I can read that material and then explain it to him or
something like that. And then in that way, we can couple the two together so that he can
get the practice but still say, "alright. I can get the practice but it’s not going to take all
that extra time. It's not going to take away from what I’m really here to do which is to
study. I mean, they’re paying me to get this degree."
Tracey: I think that’s a very good idea, but I think there’s a certain point below which.. If
your English level doesn’t reach a certain threshold, even that isn’t going to be sufficient.
And I think it is the bigger issue actually about universities accepting people whose
language levels are really at the point where there’re going to be able to pass
successfully. And it all comes down to money.
Greta: Getting back to the Korean, there was one example in here that struck me as “this
John Levis Assessing Speech Intelligibility
Pronunciation for Second Language Learning & Teaching 61
guy has done just about as he can do in isolation” because he had real /p/-/f/ problem in
free speech, but he had them perfectly fine in the reading. And it was like, “ok, he can
make those sounds, now he needs to do it, he needs to use it”. And it won’t happen until
he’s using it.
When it came to identifying particular features of speech, rhythm and tempo were talked about
several times as important, as was a related area, word stress. In an area of agreement, it was felt
that the pronunciation of stress and segmentals in English simply cannot be divided. Tracey
Derwing talked about this interdependence in relation to the Spanish speaker. The word stress
errors and the segmental errors together were closely connected to the wrong guesses made in
listening.
Tracey: …I think in the case of the Spanish speaker, I found him to be particularly clear
in the oral speech but in the reading passage all of the sudden he became very difficult to
understand. And there were parts that I just had difficulty following at all. And if I hadn't
had the written text there, I probably wouldn't have understood what he was saying. And
he went very very quickly to try to be fluent, I think, but in there in the mix he had
vowels that were off. So he said things like 'nAHtive' [ɑ] instead of 'native'. And that's
just not knowing how it is pronounced in English, right? Because he has the vowel. It's
just that in his own language, that's a cognate and he's mispronounced it. He's left off - as
Marnie mentioned - he's left off grammatical markers in places, so he said 'influence'
instead of 'influences' and 'concentrate' instead of 'concentrated'. So he's left things off
here and there. He has put stress in the wrong places several places. And that's really a
challenge for intelligibility, right? So things like 'aRAbic' and I do not know how he said
'accuRATE' [æ] or 'accuRAITE' [e]. - 'accuRAITE' [e] I believe he said. That's truly
challenging for a listener.
Murray: 'asSENT' [the pronunciation used for the word ‘accent’]
Many: 'asSENT', Yea.
Vowels came in for special notice with the Spanish speaker. One panelist joked that the student
could use a few more English vowels in his speech, while the Spanish speaking panelist pointed
out that his lack of reduced vowels seemed serious, and yet another noticed that vowel errors
were more evident in the read speech.
Deletions of sounds and syllables also came in for mention, sometimes connected to rhythm and
sometimes related to word endings, mirroring Judy Gilbert’s contention that final grammatical
inflections may be particularly important for speaking and listening. The comment about deleted
function words was called into question by one audience member, but it was quite noticeable that
the Spanish speaker did seem to leave a lot of words and syllables out because he was speaking
so quickly. The panelist also raises the suggestion that the student, in his fluency, has perhaps
decided that reduced function words do not really need to be worried about and that perhaps they
do not even need to be pronounced. Because he cannot hear them clearly, maybe native listeners
can’t hear them either?
Greta: … in the reading that there were a lot of deletions. He deleted a lot of articles and
John Levis Assessing Speech Intelligibility
Pronunciation for Second Language Learning & Teaching 62
things like that and I wonder if that could also be addressed with that same thing that they
are de-emphasized but they’re there.
Audience Member: I have a question, though. To what extent this affects intelligibility?
Even if he deleted articles, we know that deletion of certain function words does not affect
intelligibility.
Greta: They may or may not. That was a just a note. That’s just one example. There were
quite a lot of deletions as he was reading…that sort of run-away thing, and I wonder if that
bit in the spoken language that I didn’t understand. Yeah, articles may or may not be
important, but it’s just that sense of yes, the unstressed syllables are faster but they’re not
non-existent to me to keep the rhythm going.
Another panelist found final deletions most important, although she admits that this is particular
interest of hers and that her judgments may be biased.
Marnie: On these sweet little missing noun and verb endings. So those are very much on
my mind. And so when I listen, I automatically listen for the presence and absence of
these endings and whether or not they're mispronounced. For example, the extra syllables
they don’t pronounce. So potentially there's a little bias in my criticism. I was actively
listening for the presence of absence of those endings…with that Spanish speaker who
said “has too many place. The place are historical”. So maybe I just have a bias against
noun and verb endings but it’s spilling over now into sounding less educated less
competent than the speaker might be.
This comment about bias raises the issue of pet peeves that may or may not affect intelligibility,
but still are annoying to listeners. Even those who try to listen only for what affects
understanding can find themselves stuck on other issues. One of the audience members raised
this in regard to another aspect of the Spanish speaker’s performance.
Audience Member: Well I was just thinking about the degree to which somebody just
tests your patience.…does anybody get bothered by that rising intonation at the end?
dadaaA tazaaA…And I don’t know if that rising intonation at the end of sentences, every
sentence. Does that bother people? Is it just me? I don’t know. Maybe it does not matter.
Marnie: …And so, to me it goes back to the theme of our conference of intelligibility.
What about things that are simply off-putting? So I think a couple of issues have come up
here, if it’s not intelligible, it slows us down and then we’ve lost whatever else they were
saying while we were figuring out the first part of what they said. And then the other
things that are off-putting … which goes back I think to a question you were asking…So
there’s the intelligibility issue but also potentially the what’s stigmatizing and what’s off-
putting.
Another comment that came up had to do with particular words that caused problems. There is
no pattern here, but the words themselves (can/can’t) caused enough of a problem that panelists
agreed that it needed to be addressed in some way. While there may not be many words like
can/can’t, this discussion points to a more general issue: sometimes pronunciation accuracy on
key vocabulary is critical, even when there is not general pattern that can be addressed.
John Levis Assessing Speech Intelligibility
Pronunciation for Second Language Learning & Teaching 63
Joanna: One thing that actually Tracey was saying is that the big issue was actually his
pronunciation of, well I would say as “cAn’t” [a] but I think in American would be
“can’t” [æ]. But because he did not, because he - what did he say again?
Murray: I heard it as “accents cAn be changed” instead of can't. Did others get that too? I
heard exactly the opposite of what was said.
Joanna: Yes, but it was kind of a heavy functional load, cause if you get that wrong, the
whole thing changes. So if he knows that “can” was a weak form, then he would be able
to distinguish them easier somehow, the opposite.
Murray: Yeah, that's kind of unusual. Yes, that's sort of an unusual negative. Normally
you'd say “I can go” [ə] for the positive form and “I can’t go” [æ] for the negative
without the /t/ necessarily. “I can't go.” Here we’ve got “can’t be changed” as opposed to
“cAn be changed”. Yea, and it's stressed in both cases. And so you really need the /t/ in
this case, or a nice strong glottal stop. Something to make that distinction clear.
Bertha: For me, this is a point I have to make in my teaching to make clear that can, the
pronunciation of can from its negative.. it’s very very difficult.
Another issue was the social consequences of pronunciation errors. Regarding the halting speech
of the Korean speaker, and the fact that the student felt that his English might be good enough to
get by, this discussion occurred.
Joanna: I just want to comment also, it’s interesting the fallacy that he can get by. If he’s
going to work in an English-speaking company, even engineers need to have social
chitchats. There’s a lot of research done in Wellington’s Language in the Workplace
Project (http://www.victoria.ac.nz/lals/lwp/) and it’s all social chitchats. “I could do my
engineering, but I can’t for the life in me say hello or how are you to my workmates.”
And it affects their ability to function. So it’s, they need to know that they have to.
John: Jenny Miller’s work in Australia (Miller, 2003) also has that same thing that kids in
schools and in high school, immigrant teenagers, if they can’t break in to the social
milieu and be, she says, “if they can’t be heard” by their peers, then they remain
inaudible. And that the ability to do things like, she says, use things like ‘like’ and
understand humor, that that allows other people to see them as real people and that then
opens the opportunity to begin to define yourself and your identity and allows you then to
acquire the language more successfully.
Finally, panelists addressed the problem of multiple errors and their cumulative effect on
understanding. Although Munro and Derwing (2006) addressed the question of functional load
and frequency in a limited way, and Zielinski (2008) showed how stress and segmental errors
can compound effects on intelligibility, this is an area that is clearly ripe for further research,
even though such research may be difficult to carry out.
Marnie: I'll give one quick example of one of these if I may for a Korean speaker that
occurred with my co-author. A student came up to her during office hours to talk about
John Levis Assessing Speech Intelligibility
Pronunciation for Second Language Learning & Teaching 64
her thesis statement that she had to write and she plopped it down the teacher's desk and
said: "I need to talk about my thesis statement, it is occurred." So my co-author thought:
“well, I can see it has happened (laugh) what a strange turn of phrase.” The Korean
speaker, she was not trying to say it had occurred. She left out the glide, /w/ sound, but
she put the stress in the wrong syllable. She was trying to say: “it is awkward.”
Tracey: And one thing I think in research that we haven't looked at very much at this
stage is the interaction effect of different kinds of errors, right? So a lot of people have
isolated different types of errors and have looked at whether or not they influence
intelligibility or affect intelligibility one way or another. And there are a lot of really
good studies that have done that. But there is very little or I'm not aware of any actually..
any research that has been able to sort of take say three different types of errors, look at
those and then look at them together, and look at the interaction of those errors. And how,
you know, is this an exponential kind of problem when you add one? Is it additive or is it
worse than that compound?
THEME 3 – ACCENT AS SCAPEGOAT
Business writers sometimes talk about “The Peter Principle” in which people eventually rise to
the level of their incompetence. The Korean speaker’s graduate studies reflect this issue, in that
his weakness in speaking and listening made it more difficult for him to study effectively in
English. His pronunciation was clearly impaired as well, but listeners may hear the
pronunciation errors first and think that he had to work on his accent. In reality, his much more
serious problems had little to do with pronunciation. The tendency to attribute to pronunciation
what may be a very different problem was seen first in relation to the discussion about the
perceived speed of the Spanish speaker, but was also raised in relationship to vocabulary and
other errors that were evident in the free speech portions.
Audience Member: And also intelligibility is not only to pronunciation alone, and also
collocation use because sometimes we have some predictions what kind of words ..
students will say, a native speaker would say.. come up with an awkward combination
and catch our ears or interfere with intelligibility.
John: Yeah. A lot of intelligibility comes down to our expectations. And any time you
mess with expectations whether at the phonological level or at the lexical level or at the
syntactic level or at a cultural level, you can impair intelligibility. And all of those things
need to be addressed. It just seems that in the literature, and you can help me with this,
that pronunciation shows up as impairing intelligibility much more. And I think it’s not
because it impairs it much more, but it’s the first step. And then if somebody doesn’t
understand…they start to grab for causes and they say, “it’s an accent” or “they’re not
pronouncing right” or something like that. But it may be more pervasive in the kinds of
causes, and the listener is not seeing it.
This response led to a discussion of accent as a scapegoat for other problems, with the example
being given of presentation and teaching skills, an important issue in the training of international
teaching assistants.
John Levis Assessing Speech Intelligibility
Pronunciation for Second Language Learning & Teaching 65
Tracey: Absolutely. I think accent is the real scapegoat because it’s salient, it’s noticeable
and so people blame it on that. I’ve been involved in a couple of research projects in
Alberta where we were looking at international teaching assistants. And in one of them
we worked with these people who’d been identified as having “terrible accents” by their
students. And they had, they did have a terrible teaching evaluations, right? And so they
brought their teaching evaluations to us as a team, and we worked with them. We video-
taped them at the beginning of the term and then taped them at the end of the term and we
worked with them all term long. And, you know, each of these individuals had an accent
but actually that was not really the problem for most of them. Most of them really
weren’t very good teachers. And just didn’t know what to do, they did not know where to
start. So we spent a lot of time on presentation skills and pedagogical skills. And at the
end there was a lot of improvement, but their accent hadn’t changed. And one of those
guys got a teaching award at the end. And he was, you know, he was incredible, like the
difference was night and day, his pre- and post-video. You know, he was just so much
better because he had a better idea of how to present what he was teaching to his students.
And so, you know, his students had all blamed his accent. His accent didn’t change one
iota over the course of that term, but all of his presentation skills had changed, and he
was a lot more approachable, and he, you know, worked directly with the students, he
gave examples, he had handouts in advance. He went through concepts in advance
instead of talking to the board which .. before.. you know. And so the students have a
hard time discerning what’s really the problem, but they can hear the accent, so they just
blame it on the accent. But it isn’t necessarily the accent.
THEME 4 – ASSESSING WITH FREE SPEECH VS. READING ALOUD
We most often assess or diagnose pronunciation through the use of a written passage. This
method has a long history in pronunciation teaching, and it has many advantages. Teachers can
carefully construct a single passage to include and adequate representation of all the
pronunciation features that are considered to be important for instruction, and other elements like
vocabulary choice or grammar can be controlled.
Nevertheless, reading aloud is not speech, and it is a task that is full of pitfalls. The way that the
two students negotiated reading aloud and free speech was the subject of an extended discussion
regarding how we should evaluate intelligibility. Several assertions about reading aloud were
made: that it is a strange activity for most people who never have to read aloud, that it is neither
a reading skill nor a speaking skill, that it promotes worse performance than free speech for some
people but not for others, as well as raising questions on what the best way to use reading aloud
might be. Free speech, on the other hand, was implicitly thought to be better for assessment, but
this belief was not discussed in explicit detail. The main discussion centered on the problems
with reading aloud.
John: I'm curious what you think, so much of what we do in pronunciation involves
reading aloud. And with this Spanish speaker, a lot of the problem is that he is not taking
the written text and decoding it into a spoken form that is recognizable. How important is
it with especially with this kind of level of student at a graduate school level to work on
that kind of decoding? You know, the connecting of something that you can turn written
speech into spoken speech. Wayne Dickerson is not here, but it's something that he has
John Levis Assessing Speech Intelligibility
Pronunciation for Second Language Learning & Teaching 66
argued for many years. And he [the student] is a good example of someone who can't do
it.
Bertha: Well, as I tell many of my teachers, not to use reading aloud in class unless
you've prepared the students to read. You know, you tell them where to chunk, where the
stresses are, how you pronounce words, then you can read aloud. But judging students
from read-alouds is very bad. It’s, they’re concentrating on something else, you know.
And then studies of broadcasters in Spanish, Spanish broadcasters, that read the news and
they go berserk, “ok, oh my God, where did I get that?” So I say it’s worse for the EFL
students.
Murray: I think asking people to read aloud is asking them to something that’s kind of
weird. It's not something that most of us are called on to do on a regular basis. I mean,
it’s one thing if we were reading to our children or something like that, but most of the
time, it is not something that we do a lot of, most people, now some people do. And so
this is an alien thing. And when you’re asked to do an alien thing, it comes out weird
because you are not focusing on the meaning, you're not focusing on the processing,
what's being said. And so you pause in the wrong places, and you have reading
pronunciations, where you pronounce something simply the way it's spelled as opposed
to the way you have already learned that it’s actually produced because you’re not
making the connection between your internalized knowledge and what's on the page. So I
really agree with you.
Bertha: I said this to my teachers. One of my teachers was dictating something to a group
of students and she said.. she said, “she celebrated it on Thursday’. So she said, ‘did you
understand?” She said it again, “she celebrated it on Thursday." She realized they didn't
understand so she actually said, she said," she celebrated IT on Thursday." (laugh) So
everyone was pleased. And she said it three times after that, “she celebrated IT on
Thursday". I said," oh my God. The students will go out sounding like this." And this was
a teacher, you know, not a student.
Tracey: I think even if you have a native speaker or a group of native speakers and ask
them to read aloud, you'll find that most of them don't do a terribly good job in parsing
and, especially for evaluation (unclear). And I think it's an unnatural task unless you've
practiced it and unless you're going to be reading the news or an actor or something like
that then most people don't have a real need for that.
Marnie: So in our master’s program…we have a guest speaker who comes in. She's the
author of "Short Course on Teaching Reading Skills", and then she's also written
"Reading Power", "More Reading Power", "Reading Power for the Beginners" (laugh)
and all of these things. And she lists reading skills, she's got a list of 24 reading skills,
and reading aloud is not on it, but she'll talk about it as a good diagnostic tool. So it might
give us a first pass we are exploring some of the potential problems for our students, but
not to consider it a reading skill. And a good example about what would be, even for
native speaker to read something aloud and then you asked a comprehension question, "I
don't know, I wasn't listening to what I was saying" (laughter).
John Levis Assessing Speech Intelligibility
Pronunciation for Second Language Learning & Teaching 67
One audience member suggested that reading aloud was only valuable in certain contexts.
Audience Member: Wouldn't it really depend on the context that he will use English in?
So let's say he's never going to read aloud, and his free speech sounds ok. I mean, I don’t
know. Again, I think that just if you just go by reading, you’re kind of putting him at a
disadvantage, because, you know, if I had to read Spanish, I’m sure that I, the same
problems that he did. But if my fluency in spoken, in just regular conversational speech is
fine and that's the goal, then why mess with it? You know…if he can be intelligible in
conversational speech, then why continue to beat a dead horse, you know?
Not explicitly addressed in this discussion of reading aloud is whether teachers can effectively
make use of written text and predictive rules, as argued by Wayne Dickerson (1987). What
seems to be clear is that using written texts to assess intelligibility is problematic. Reading aloud
is not the same as speaking, and unless students are given sufficient preparation in the task, the
task itself is likely to create errors that do not show up in free speech. However, it is also clear
that in academic settings, students need to learn how to better connect spelling patterns to what
the words actually sound like.
CONCLUSION
A number of conclusions are suggested from the discussion. First is that intelligibility is a
moving target, depending on the type of listening being done, the familiarity of the listener with
the patterns of the speaker, the, the types of language features that cause conflicts with the
expectations that a listener brings to the interaction, and the type of speaking task. In addition,
intelligibility may be seriously impaired by a single error (as with can/can’t) or it may only be
impaired when multiple errors combine to violate expectations to the extent that listeners can’t
process speech successfully in real time (as in occurred vs. awkward). In addition, although
pronunciation may be frequently implicated in loss of intelligibility, pronunciation is clearly not
the only issue. It may be the most noticeable, but pronunciation never exists on its own in
communication, and vocabulary choice, grammar, fluency, listener internal factor, speaking
choices, and the information presented through other channels (such as the use of the board in a
lecture) all may contribute to a loss of understanding.
ACKNOWLEDGMENTS
Thank you to the panelists for making this discussion possible, and to Ghinwa Alameen for her
work in transcribing the video.
ABOUT THE AUTHOR
John Levis is Associate Professor of TESL and Applied Linguistics at Iowa State University. He
teaches courses in ESL teaching methods, the teaching of pronunciation, speaking and listening,
Introduction to linguistics, Sociolinguistics, and the history of English. He is the co-organizer of
the Pronunciation in Second Language Learning and Teaching Conference. His articles have
appeared in TEOSL Quarterly, World Englishes, Language Awareness, Applied Linguistics, The
Annual Review of Applied Linguistics, ELT Journal, SPEAK Out, System and TESOL Journal.
John Levis Assessing Speech Intelligibility
Pronunciation for Second Language Learning & Teaching 68
REFERENCES
Celce-Murcia, M., Brinton, D., Goodwin, J. & Griner, B. (2010). Teaching pronunciation: A
reference for teachers of English. Cambridge: Cambridge University Press.
Derwing, T., Munro, M. & Rossiter, M. (2002). Teaching native speakers to listen to foreign-
accented speech. Journal of Multilingualism and Multicultural Development, 23, 245-
259.
Dickerson, W. (1987) Orthography as a pronunciation resource. World Englishes, 6,1, 11–20.
Firth, S. (1992). Pronunciation syllabus design: A question of focus. In S. Avery & P. Ehrlich,
Teaching American English pronunciation, Oxford: Oxford University Press, pp. 173-
183.
Hinofotis, F. & Bailey, K. (1981). American undergraduates' reactions to the communication
skills of foreign teaching assistants, in J. Fisher, M. Clarke, and J. Schachter, (eds). On
TESOL '80 : Building bridges : research and practice in teaching English as a second
language : selected papers from the fourteenth annual convention of Teachers of English
to Speakers of Other Languages, (pp. 120-136). Washington, D.C.: TESOL.
Miller, J. (2003). Audible difference: ESL and social identity in schools. Clevedon:
Multilingual Matters.
Munro, M. & Derwing, T. (2006). The functional load principle in ESL pronunciation
instruction: An exploratory study. System, 34, 520-531.
Zielinski, B. (2008). The listener: No longer the silent partner in reduced intelligibility. System
36,1: 69-84.
Appendix
Free speech interview questions
1. What is your name and where are you from?
2. Tell me something interesting about your hometown.
3. What are you studying at ISU? Can you tell me a little about it?
4. If you could travel anywhere in the world, where would you go? Why?
John Levis Assessing Speech Intelligibility
Pronunciation for Second Language Learning & Teaching 69
Read speech reading passage (from Celce-Murcia et al. 2010, p. 481)
Is English your native language? If not, your foreign accent may show people that you
come from another country. Why is it difficult to speak a foreign language without an accent?
There are a couple of answers to this question. First, age is an important factor in learning to
pronounce. We know that young children can learn a second language with perfect
pronunciation. We also know that older learners usually have an accent, though some older
individuals also have learned to speak without an accent.
Another factor that influences your pronunciation is your first language. English
speakers can, for example, recognize people from France by their French accents. They can also
identify Spanish or Arabic speakers over the telephone, just by listening carefully to them. Does
this mean that accents can’t be changed? Not at all! But you can’t change your pronunciation
without a lot of hard work. In the end, improving appears to be a combination of three things:
concentrated hard work, a good ear, and a strong ambition to sound like a native speaker.
You also need accurate information about English sounds, effective strategies for
practice, lots of exposure to spoken English, and patience. Will you make progress, or will you
give up? Only time will tell, I’m afraid. But it’s your decision. You can improve! Good luck,
and don’t forget to work hard.
