| Pages in topic: [1 2] > | Off topic: Why do we hate questionnaires? Thread poster: Joanna Gough
|
|---|
Hello there,
I would like to invite members of this forum to view their opinions regarding participation in research surveys/questionnaires.
I was recently following a debate on the radio about the action of Green Peace which resulted in blocking numerous BP stations in London. The bottom line was that the protesters wanted to draw our attention to the fact that BP should invest more money in research into greener alternatives to crude oil. A very witty presenter succee... See more Hello there,
I would like to invite members of this forum to view their opinions regarding participation in research surveys/questionnaires.
I was recently following a debate on the radio about the action of Green Peace which resulted in blocking numerous BP stations in London. The bottom line was that the protesters wanted to draw our attention to the fact that BP should invest more money in research into greener alternatives to crude oil. A very witty presenter succeeded in proving to the angry listeners that indeed, the only way for Green Peace to halt our attention was to resort to such disruptive (and indeed annoying) action. We all know that much research is needed in order to move away from our reliance on crude oil, but few of us actually support organisations which desperately try to make a change.
By way of (not so parallel, but still relevant) analogy, much research is needed in our industry for a host of reasons and for various purposes and each year hundreds of students from all over the world are desperately trying to reach us in order to obtain valuable data for their research. And yet, our response seems to be extremely low. Even surveys carried out by big organizations like Common Sense Advisory or TAUS yield responses in the region of 350 and smaller surveys carried out by students rarely reach a hundred entries. How can any research proceed on the basis of such a small sample? By way of analogy with the free/open source movement, a study carried out in 2002 (!) yielded 2784 responses!
We are so willing to share our insights or give advice on matters pertinent to the minor or detailed aspects of our work, but when it comes to helping others in their academic work which is normally geared towards contributing to the 'bigger picture' we are showing signs of collective apathy.
I admit that some questionnaires can be a bit sloppy or even annoying, but these too are trying to contribute to the general pool of knowledge and it must be so demeaning for the researchers to see such poor return. Saying that, there are also many questionnaires out there that are of high quality and which give a great opportunity for translators to engage in current discussions pertinent to various aspects of their work. Sadly, these too generate only a small sample of usable entries.
Why are we so cold and uncooperative?
Best wishes,
Joanna ▲ Collapse
| | | | | I'm a questionnaire addict | Jul 28, 2010 |
I don't hate questionnaires - in fact I'm somewhat addicted to them and always take part if I have time. Some questionnaires are indeed annoying because they ask the wrong questions or the possible answers are not relevant to me, and some of them suddenly make the screen freeze and one can't complete them.
Meanwhile, ask away!
Jenny
| | | | PAS 
Local time: 21:31
Polish to English
+ ...
| Sloppy questionnaires | Jul 28, 2010 |
askaska wrote:
I admit that some questionnaires can be a bit sloppy or even annoying, but these too are trying to contribute to the general pool of knowledge and it must be so demeaning for the researchers to see such poor return.
It is more difficult to construct a meaningful questionnaire/ survey than most people imagine.
Many questionnaires I have seen contain questions which contradict themselves, or ones to which I could not provide a sensible response.
So how can I go about answering a poorly constructed survey? Lie? Guess? If I do that, the results will not be meaningful, although the "researcher" may be misguided into thinking that they are.
Bang! there's yet another scientific article not worth the paper it's been printed on.
Best,
Pawel Skalinski.
| | | | | Some problems | Jul 28, 2010 |
In my opinion, and after having translated a number of surveys aimed towards different purposes, I think there are several major factors here:
- Surveys are often too long. The people doing the survey think they need all the answers in one go, when they should probably make one survey with a maximum of a dozen questions, and then other sub-surveys with those willing to answer more questions.
- It is hard to make a good question. I often translate questions... See more In my opinion, and after having translated a number of surveys aimed towards different purposes, I think there are several major factors here:
- Surveys are often too long. The people doing the survey think they need all the answers in one go, when they should probably make one survey with a maximum of a dozen questions, and then other sub-surveys with those willing to answer more questions.
- It is hard to make a good question. I often translate questions that are too specific and are generally mistranslated (I mean that the translated versions are ambiguous) or not specific enough, leaving room for different interpretations and therefore mistranslation. In our role as translators, we should be more careful when dealing with surveys, as we need to convey the exact meaning of the question, not more, but not less, with absolutely no ambiguity.
- We distrust the final use of the surveys. We are proposed many surveys, but the final idea or goal of the askers is rarely revealed. Questions are usually biased so that the answers will always promote the askers' goals. For instance, in a survey about the construction of a new mall in town, you get agree/disagree questions like "A) Shopping centres damage shopping choices downtown." (from people against the mall), or "A) A new mall would offer more shopping choices to you" (from those planning to build the mall). We just want to know what we are answering, more exactly and in its context. Every survey should have a statement from the askers about its context and goal.
Recently I was called to my home by a survey company on behalf of our regional government. They just could not understand when I gave our government a bad note about some area, and asked me to repeat when I did so. They clearly put my replies among the exceptions and outside of the global picture they gave the government. They do want to keep doing work for the government, right?
This is all that jumps to mind. It will be interesting to see what others have to say. ▲ Collapse
| | |
|
|
|
Paul Dixon
Brazil
Local time: 16:31
Portuguese to English
+ ...
| I don't mind questionnaires, but... | Jul 28, 2010 |
I don't mind questionnaires, as long as they are not too long, but hate it when I'm stopped in the street and after 100 or so questions they come to that famous one "what is your telephone number?" or, worse, "what is your credit card number". I just turn my back and walk off - as it is just another marketing trick to get people to buy some bank insurance or encyclopaedias that they can't afford and don't need.
| | | | | Ok for the sloppy questionnaires, but what about the good ones? | Jul 28, 2010 |
Hello again,
Thank you all for your interesting insights. All you points are valid and without doubt result from bad experience. It is a shame that bad practices of those who expect great results without putting much effort into designing and piloting their questionnaires overshadow those well designed surveys with a clear purpose and with prospects of generating an end value.
Perhaps, as someone on Proz has already suggested, there should be a dedicated forum for post... See more Hello again,
Thank you all for your interesting insights. All you points are valid and without doubt result from bad experience. It is a shame that bad practices of those who expect great results without putting much effort into designing and piloting their questionnaires overshadow those well designed surveys with a clear purpose and with prospects of generating an end value.
Perhaps, as someone on Proz has already suggested, there should be a dedicated forum for posting questionnaires where addicts like Jenny (bless you - I love your answer!) could feed their addiction and perhaps rate questionnaires so that the best and the most valuable ones can receive due acknowledgement and generate results proportionate to their quality and importance.
I am obviously speaking from my own experience. It took me three weeks to develop my questionnaire with numerous consultations with my tutors, fellow students and, most importantly, people from the industry interested in my research. For all this effort and multiple and varied channels of distribution I found the response rate low. Well, low in comparison to the number of people who have seen it but decided not to participate. I'm relatively happy with the usable entries I have so far, but only because I know how hard it is to obtain every sigle one. I'm just curious why such disproportionately large number of people are just not bothering. Be it a bad questionnaire designed by a student or an excellent one developed by a professional researcher, the response rates are just incredibly low in comparison to the great numbers of translators who have happened to come into contact with such surveys one way or another. It is just sheer numbers that I'm baffled by. By way of an example, out of 164 viewings from one forum in just less than a day there was one, maybe two responses. That is 1% of all the people who have seen it. 1%!! And the questionnaire is about openness, sharing and collaboration, which only adds insult to injury!
Don't get me wrong, I'm not advertising my questionnaire here. It has been on Proz for some time. I'm just trying to understand the reasons why translators are generally not responsive in this area.
So apart from the fact that they are often too long, badly designed and biased, are there any other reasons that prevent you from even looking at them?
Once again, thank you all for your valuable comments and I'm looking forward to more opinions!
Best wishes,
Joanna ▲ Collapse
| | | | | academic research related to translation | Jul 28, 2010 |
Paul Dixon wrote:
I don't mind questionnaires, as long as they are not too long, but hate it when I'm stopped in the street and after 100 or so questions they come to that famous one "what is your telephone number?" or, worse, "what is your credit card number". I just turn my back and walk off - as it is just another marketing trick to get people to buy some bank insurance or encyclopaedias that they can't afford and don't need.
Paul,
I understand you point and yes, these kinds of surveys can be really annoying.
But let's narrow this discussion to surveys you can access on-line, on your own volition, in your own time and if you don't like them or they are not relevant to youe experience you just exit. Surveys related to our industry - i.e. translation. Surveys that don't require any personal details, are relevat to the work you do and offer to share the results. Surveys done mostly by students in order to feed their findings back to the academia or industry and which represent an attempt to contribute in a meaningful way to the knowledge we all share.
Why don't we like these?
Best wishes,
Joanna
| | | | lexical
Spain
Local time: 21:31
Portuguese to English
| a twist on an old saying | Jul 28, 2010 |
Those who can, do.
Those who can't, pose questionnaires at those who can.
| | |
|
|
|
| Nub and crux | Jul 28, 2010 |
askaska wrote:
So apart from the fact that they are often too long, badly designed and biased, are there any other reasons that prevent you from even looking at them?
Do you not think those are reason enough?
Before tonight (when, I confess, I hunted down yours and did it), I had to abandon the last 3 or 4 surveys publicised on here because they were utter crap. They usually start OK, but it soon becomes clear that most of the researchers haven't got a scooby about the industry in general or the nuts and bolts of the work in particular.
And since I'm been honest and a bit grumpy, that seems to apply to yours, in terms of the age brackets you used. I daresay 40 seems inconceivably ancient, and well able to be considered in the same category as those past retirement age. However, this is an industry with a fair old proportion of late middle-aged and older people in it. The joys of working for yourself, I suppose. No fresh-faced HR grads who think anyone over 45 smells of wee in charge of hiring and firing. Conversely, I would think the proportion of professionals under 20 with an opinion worth recording to be tantamount to zero. Indeed, I would argue professional translator under 20 is oxymoronic. But I am grumpy. Anyway, point being, I know a few people in their 40s who are right into all this web 2.0 stuff and could harp on about it for England. But you are lumping them in with the 70+ crowd who perhaps might be more reticent. Result (IMHO) your final age bracket might demonstrate very little in terms of buy-in or lack of it because it is way too wide, and probably encaapsulates well over 50% of the translator population.
And we (I) see that kind of thing in every questionnaire and we (I) just think "ah great, another clueless academic adding diddly squat to the sum of human knowledge".
And that is why I never usually bother.
| | | | Emma Goldsmith
Spain
Local time: 21:31
Member (2004)
Spanish to English
Charlie Bavington wrote:
I daresay 40 seems inconceivably ancient
Yes, I noticed the strange age groups in the survey. In fact it's the first time that I've ever ticked the box at the very end of the "how old are you" row
| | | | | I entirely agree | Jul 28, 2010 |
Charlie Bavington wrote:
Indeed, I would argue professional translator under 20 is oxymoronic. But I am grumpy.
I entirely agree with this kind of grumpiness.
| | | | | good point on age, but age was not my main focus. funny post though:) | Jul 28, 2010 |
Charlie Bavington wrote:
And since I'm been honest and a bit grumpy, that seems to apply to yours, in terms of the age brackets you used. I daresay 40 seems inconceivably ancient, and well able to be considered in the same category as those past retirement age. However, this is an industry with a fair old proportion of late middle-aged and older people in it. The joys of working for yourself, I suppose. No fresh-faced HR grads who think anyone over 45 smells of wee in charge of hiring and firing. Conversely, I would think the proportion of professionals under 20 with an opinion worth recording to be tantamount to zero. Indeed, I would argue professional translator under 20 is oxymoronic. But I am grumpy. Anyway, point being, I know a few people in their 40s who are right into all this web 2.0 stuff and could harp on about it for England. But you are lumping them in with the 70+ crowd who perhaps might be more reticent. Result (IMHO) your final age bracket might demonstrate very little in terms of buy-in or lack of it because it is way too wide, and probably encaapsulates well over 50% of the translator population.
.
Good points Charlie and thank you for your funny post. Please note that I also have a question about how long the respondents have been working as translators which gives me a much better understanding of their profile than age alone. The 40 mark was mainly set to indicate whether the respondents have 'been there' before CAT Tools, but I have now added an option to indicate whether they belong to the more mature groups, 51-60 and 60+. I admit the 20 mark was set too low. Anyway, I'm more interested in attitudes rather than age and these don't have to be necessarily age-specific, as you quite rightly point out. I don't make crude assumptions simply relating age to technology uptake. I'm interested in translators' own perception of where they are in terms of technology and whether they label themselves as innovators/early adopters, fast followers, late majority or traditionals and then I apply this to the rest of the results. This is much more interesting than looking at age alone. Also, please note that the questionnaire is not only focused on technology per se, but also on trends which are technology enabled, but are more of a sociological nature - and these are certainly not age related.
Anyway, I'm myself not far off the 40 mark so this might be an indication that age was not my main concern:)
Best,
Joanna
[Edited at 2010-07-28 20:11 GMT]
[Edited at 2010-07-28 22:23 GMT]
[Edited at 2010-07-28 22:24 GMT]
| | |
|
|
|
| Are questionnaires a source of knowledge? | Jul 28, 2010 |
First of all, I often look at the forums on Proz for a couple of minutes rest from translating, so I do not have the time to fill in a questionnaire at that moment, but really I'm not thrilled with the information resulting from questionnaires, are they really a souce of insight?
Again, without having seen your questionnaire, I do second the other criticisms of badly written or frankly biased questionnaires.
| | | | | do we gain insight from questionnaires? | Jul 28, 2010 |
Lesley Clarke wrote:
I'm not thrilled with the information resulting from questionnaires, are they really a souce of insight?
Lesley,
Questionnaires are only one way of obtaining data for a research project, and one has to be aware of the limitations of this method. There is always a great deal of compromise involved due to time constraints. Therefore questionnaires are usually complemented by other methods, in my case through feedback, following forum discussions and interviews, which add depth and extra dimensions to the research as well as allow to correct data obtained through a questionnaire. Insight is generated through analysis of all available data, and therefore will not be easily observed through looking at percentages alone.
The biggest limitation of any research is however the lack of comparable data against which one can measure change. This is precisely why I believe people should be more willing to participate in questionnaires as insight can be more readily gained when there is enough data to compare and analyse.
If we don't know where we are at the moment, how can we predict how things will develop? And if we don't want to have insight into how things are developing, how can we have a meaningful debate about the future of the industry? This is obviously related to my specific area of interest, but I'm sure all research projects are geared to look into the future of whatever they are researching and I'm sure that all of us would rather be the agents than the victims of change. And yes, things are changing in our industry, whether we welcome the change or not.
Thank you for your post which gave me yet another valuable insight as far as the perception of questionnaires is concerned.
With best wishes,
Joanna
| | | | Samuel Murray
Netherlands
Local time: 21:31
Member (2006)
English to Afrikaans
+ ...
| Not getting many responses | Jul 29, 2010 |
askaska wrote:
It took me three weeks to develop my questionnaire with numerous consultations with my tutors, fellow students and, most importantly, people from the industry interested in my research. For all this effort and multiple and varied channels of distribution I found the response rate low.
By way of an example, out of 164 viewings from one forum in just less than a day there was one, maybe two responses. That is 1% of all the people who have seen it. 1%!!
Well, there are numerous things that affect the response rate of a questionnaire.
Here's your post (that you refer to):
http://www.proz.com/forum/translation_theory_and_practice/175937-professional_translators_and_web_20_technologies_based_on_openness_sharing_and_collaboration.html
Firstly, the post you made (to which you got 164 views) had a title written in which words start with a capital letter. If you look at the other discussions here, you'll notice that that is not common. I almost did not read your post myself, because I was under the distinct impression that the post would most likely be a press release and not much else. Perhaps other people also thought that the post would contain valuable information... but instead the post is simply an invitation to visit some other web site.
Secondly, your post itself is one long, dense paragraph. Again, you were lucky that I read any of it, because I typically don't read such posts myself. I see it, and ProZ.com's system registers my visit, but I don't read it. If you had broken up your post into neat, topical paragraphs, it would have been not only visually appealing but also easy to skim read.
Thirdly, your actual post doesn't say anything extra to what your post's title says. Your post title is "Professional Translators and Web 2.0 Technologies Based on Openness, Sharing and Collaboration" and in your post you say you want to "ascertain the views of professional translators towards the new trends of openness, sharing and collaboration associated with web 2.0 technologies." So you're just repeating youself -- you didn't even bother to use different terms, or to describe or explain what those things are.
If you want people to take the survey, you have to hook them by appealing to their interest. Otherwise the only people who will take your survey would be survey addicts.
You should also distinguish between the type of things you say to your professors (who want you to be deliberately vague and overly humble) and your target audience (who need to know that you're not wasting their time). For example, your post says "I'm hoping to contribute to the general pool of knowledge and to a better understanding of where professional translators are in relation to the changes that affect industries globally." This is the type of mumbo jumbo that professors love to read from their favourite students, but for the general public it's just fluff.
With your post title you managed to grab the attention of 164 people in one day, but with the rest of your post you managed to make 99% of them lose interest in your survey. Don't blame the public for your own failure to capture and retain a more meaningful percentage of readers.
Here's a question for you: how many people who started the survey actually completed it? In other words, on which page in your survey do you lose most people? I assume you have statistics about how many people visited each page of the survey... you do, don't you?
| | | | | Pages in topic: [1 2] > | To report site rules violations or get help, contact a site moderator: You can also contact site staff by submitting a support request » Why do we hate questionnaires? | Trados Business Manager Lite | Create customer quotes and invoices from within Trados Studio
Trados Business Manager Lite helps to simplify and speed up some of the daily tasks, such as invoicing and reporting, associated with running your freelance translation business.
More info » |
| | Anycount & Translation Office 3000 | Translation Office 3000
Translation Office 3000 is an advanced accounting tool for freelance translators and small agencies. TO3000 easily and seamlessly integrates with the business life of professional freelance translators.
More info » |
|
| | | | X Sign in to your ProZ.com account... | | | | | |
Login to reply/comment 
