Just a general comment on non-representative sample.  Years ago and Old Professor said a good start if you want to know what fish are in a pond is to throw out a big net and look at a lot of fish.  While any estimates of the proportions of different types will be flawed, you may learn enough to serve your purpose and help design a more sophisticated survey.  Some questions about what kind of people favor what kind of test questions  could in some degree be answered by a self selected sample.


On Fri, Apr 18, 2014 at 12:47 PM, Rueben Ternes <[log in to unmask]> wrote:
While I agree with the sentiment that our comments were unhelpful (my comment was especially a bit snarky), I disagree that this is not the appropriate place to have these conversations.  In fact, I think this is as perfect venue to have serious discussions about the validity of self-reported data from convenience sampling!  It is a methods list serve, after all.

Though I snarked about it, in general, I think convenience sampling gets a bad rap.  Especially Mechanical Turks, which have been overly scrutinized in some settings.  I'd like to note that some studies do suggest that Mechanical Turks are broadly representative of generalized random sampling.  In fact, they may be more representative of a general population than say, college students, from which a great deal of psychology draws its participant pool.  To be certain, they are not perfect, but the idea that we can obtain true random sampling is pretty much bunk anyway. How many researchers can force participants to answer their surveys?  To the best of my knowledge: almost none.  In some ways, all samples are convenience samples to some degree.  I think it's near sighted to suggest that we cannot draw valuable information from such studies. 

Now, whether or not it's appropriate to send to a specific list serve dominated by highly-educated research methodologists is another question altogether.  But that could certainly be a valid population of interest.  And, to be fair, we have no way of telling whether or not the researcher is tracking our responses in that manner.  Certainly comparing the responses of educational methodologists with a more general population about educational testing systems could provide valuable insight.  

Just my two cents!

-Reuben


On Fri, Apr 18, 2014 at 12:20 PM, Barbara Helms <[log in to unmask]> wrote:
I do feel that this is not the place for you to comment.  If you find issues with this "invitation to volunteers", (1) don't volunteer and (2) send an email to the author (who has included her personal email address). Do not send them on the listserve. This is not the venue -- especially when making unhelpful jibes.

Barbara J. Helms, Ph.D.


On Fri, Apr 18, 2014 at 10:57 AM, Elaine Fitt <[log in to unmask]> wrote:

This is a self-selected survey and the results do not necessarily reflect the population.  I would certainly not trust any conclusions drawn based on this study.

 

Elaine B. Fitt  Ph.D.

 

 

Elaine B. Fitt  LLC

Phone:  215 913 4379

Email:  [log in to unmask]

 

 

 

This message, including any attachments, may contain confidential information intended for a specific individual and purpose and is protected by law.  If you are not the intended recipient, please delete it. Any disclosure, copying, or distribution of this message is strictly prohibited. 

 

From: Educational Research Methodology List [mailto:[log in to unmask]] On Behalf Of Rueben Ternes
Sent: Friday, April 18, 2014 10:44 AM
To: [log in to unmask]
Subject: Re: [EDRESMETH-L] ATTITUDES TOWARD STANDARDIZED TESTING: HUMAN SUBJECTS WANTED

 

Answer: All Mechanical Turks?

 

On Fri, Apr 18, 2014 at 10:12 AM, Dennis Roberts <[log in to unmask]> wrote:

On 4/17/2014 10:40 PM, Tanesia Beverly wrote:

You are invited to participate in a study investigating the perceptions a=


nd
attitudes about standardized test taking ability.


What I have noticed is that more and more folks are posting requests on the internet for "volunteers" to participate in some kind of a research study (usually a survey).  To me this is not a good way to define and obtain a sample ... though it appears to be a fairly easy one. My question is ... let's say the researcher gets n=87 to respond .... WHAT IS THE POPULATION TO WHICH THESE RESULTS (whatever they are) CAN BE GENERALIZED?



Educational Research and Methodology Listserv
-----------------------------------------------------------------
List Service Info http://listserv.uconn.edu
To cancel your subscription address click please do the following:
SEND an email to the following address:
    mailto (colon) listserv (at) listserv (dot) uconn (dot) edu
Your email should contain only the message
    UNSUB EDRESMETH-L.
Address problems with your subscription to: [log in to unmask]
-----------------------------------------------------------------



 

--
Reuben Ternes
Research Associate
Office of Institutional Research and Assessment
509 Wilson Hall
Oakland University
248-370-2559

Educational Research and Methodology Listserv ----------------------------------------------------------------- List Service Info http://listserv.uconn.edu To cancel your subscription address click please do the following: SEND an email to the following address: mailto (colon) listserv (at) listserv (dot) uconn (dot) edu Your email should contain only the message UNSUB EDRESMETH-L. Address problems with your subscription to: [log in to unmask] -----------------------------------------------------------------

Educational Research and Methodology Listserv ----------------------------------------------------------------- List Service Info http://listserv.uconn.edu To cancel your subscription address click please do the following: SEND an email to the following address: mailto (colon) listserv (at) listserv (dot) uconn (dot) edu Your email should contain only the message UNSUB EDRESMETH-L. Address problems with your subscription to: [log in to unmask] -----------------------------------------------------------------



--
Barbara J. Helms, Ph.D.
Education Evaluation Consultant
2144 Commonwealth Avenue
Auburndale, MA 02466
Telephone: 617-480-2260

 
Educational Research and Methodology Listserv ----------------------------------------------------------------- List Service Info http://listserv.uconn.edu To cancel your subscription address click please do the following: SEND an email to the following address: mailto (colon) listserv (at) listserv (dot) uconn (dot) edu Your email should contain only the message UNSUB EDRESMETH-L. Address problems with your subscription to: [log in to unmask] -----------------------------------------------------------------



--
Reuben Ternes
Research Associate
Office of Institutional Research and Assessment
509 Wilson Hall
Oakland University
248-370-2559
Educational Research and Methodology Listserv ----------------------------------------------------------------- List Service Info http://listserv.uconn.edu To cancel your subscription address click please do the following: SEND an email to the following address: mailto (colon) listserv (at) listserv (dot) uconn (dot) edu Your email should contain only the message UNSUB EDRESMETH-L. Address problems with your subscription to: [log in to unmask] -----------------------------------------------------------------



--
Wm W. Pendleton
Professor Emeritus Emory University
3 Breakers Ct.
Salem, SC 29676

Tel. 864 944-2238
Educational Research and Methodology Listserv ----------------------------------------------------------------- List Service Info http://listserv.uconn.edu To cancel your subscription address click please do the following: SEND an email to the following address: mailto (colon) listserv (at) listserv (dot) uconn (dot) edu Your email should contain only the message UNSUB EDRESMETH-L. Address problems with your subscription to: [log in to unmask] -----------------------------------------------------------------