The problem with many balanced scales is that when evaluating events or people or objects raters tend to stick to the positive end of the scale. This is a particular bias in raters, especially untrained ones. When 70% or more of the responses are on a positive end then differentiation suffers. There are 2 ways to deal with this, either expand the scale to more levels, which may just give you a large number of levels not endorsed (the negative ones), or make the scale unbalanced. The latter method is more efficient. And by spreading out so many positive responses across more levels, better differentiation is achieved.

Paul Swank, Professor
Health Promotion and Behavioral Sciences
School of Public Health
University of Texas Health Science Center Houston

Sent from my iPad

On Apr 29, 2015, at 4:55 PM, Burke Johnson <[log in to unmask]<mailto:[log in to unmask]>> wrote:

Question for research and measurement experts,

Are any of you concerned about the unbalanced 4-point rating scale being used by the AERA to evaluate the recent annual conference? Here is the unbalanced scale they are using:

1=dissatisfied
2=somewhat satisfied
3=satisfied
4=very satisfied.

It seems pretty convenient and unfair to have 3 of 4 responses indicating satisfaction and only one indicating dissatisfaction in my professional opinion.

I informed the Executive Director of the AERA about this problem two years ago and she did not make a change to a recommended scale.

I had recommended that they use one of the two following scales:

1=Very dissatisfied
2=Somewhat dissatisfied
3=Somewhat satisfied
4=Very satisfied

or

1=Very dissatisfied
2=Somewhat dissatisfied
3=Neutral
4=Somewhat satisfied
5=Very satisfied

If you agree with me about this scaling problem, do you have any suggestions about how to get the AERA to change their scale?

Cheers,
Burke Johnson
Educational Research and Methodology Listserv ----------------------------------------------------------------- List Service Info http://listserv.uconn.edu<https://urldefense.proofpoint.com/v2/url?u=http-3A__listserv.uconn.edu&d=AwMFaQ&c=6vgNTiRn9_pqCD9hKx9JgXN1VapJQ8JVoF8oWH1AgfQ&r=8frmz39BMbPfozSCry7R2XF1zD3P8iT3dTcbzh5VWc8&m=niLm1r0FQ8i1g678LgMjCQmi53Msx9TB2IJCO1o2sYI&s=6rDOa4PShFkiwBAKt01NiZ4UeLjVgXZx3ywCTFBDXD0&e=> To cancel your subscription address click please do the following: SEND an email to the following address: mailto (colon) listserv (at) listserv (dot) uconn (dot) edu Your email should contain only the message UNSUB EDRESMETH-L. Address problems with your subscription to: [log in to unmask]<mailto:[log in to unmask]> -----------------------------------------------------------------

Educational Research and Methodology Listserv
-----------------------------------------------------------------
List Service Info http://listserv.uconn.edu
To cancel your subscription address click please do the following:
SEND an email to the following address:
     mailto (colon) listserv (at) listserv (dot) uconn (dot) edu
Your email should contain only the message
     UNSUB EDRESMETH-L.
Address problems with your subscription to: [log in to unmask]
-----------------------------------------------------------------