Just a general comment on non-representative sample. Years ago and Old
Professor said a good start if you want to know what fish are in a pond is
to throw out a big net and look at a lot of fish. While any estimates of
the proportions of different types will be flawed, you may learn enough to
serve your purpose and help design a more sophisticated survey. Some
questions about what kind of people favor what kind of test questions
could in some degree be answered by a self selected sample.
On Fri, Apr 18, 2014 at 12:47 PM, Rueben Ternes <[log in to unmask]> wrote:
> While I agree with the sentiment that our comments were unhelpful (my
> comment was especially a bit snarky), I disagree that this is not the
> appropriate place to have these conversations. In fact, I think this is as
> perfect venue to have serious discussions about the validity of
> self-reported data from convenience sampling! It is a methods list serve,
> after all.
>
> Though I snarked about it, in general, I think convenience sampling gets a
> bad rap. Especially Mechanical Turks, which have been overly scrutinized
> in some settings. I'd like to note that some studies do suggest that
> Mechanical Turks are broadly representative of generalized random sampling.
> In fact, they may be more representative of a general population than say,
> college students, from which a great deal of psychology draws its
> participant pool. To be certain, they are not perfect, but the idea that
> we can obtain true random sampling is pretty much bunk anyway. How many
> researchers can force participants to answer their surveys? To the best of
> my knowledge: almost none. In some ways, all samples are convenience
> samples to some degree. I think it's near sighted to suggest that we
> cannot draw valuable information from such studies.
>
> Now, whether or not it's appropriate to send to a specific list serve
> dominated by highly-educated research methodologists is another question
> altogether. But that could certainly be a valid population of interest.
> And, to be fair, we have no way of telling whether or not the researcher
> is tracking our responses in that manner. Certainly comparing the
> responses of educational methodologists with a more general population
> about educational testing systems could provide valuable insight.
>
> Just my two cents!
>
> -Reuben
>
>
> On Fri, Apr 18, 2014 at 12:20 PM, Barbara Helms <[log in to unmask]> wrote:
>
>> I do feel that this is not the place for you to comment. If you find
>> issues with this "invitation to volunteers", (1) don't volunteer and (2)
>> send an email to the author (who has included her personal email address).
>> Do not send them on the listserve. This is not the venue -- especially when
>> making unhelpful jibes.
>>
>> Barbara J. Helms, Ph.D.
>> Email: [log in to unmask]
>>
>>
>> On Fri, Apr 18, 2014 at 10:57 AM, Elaine Fitt <[log in to unmask]> wrote:
>>
>>> This is a self-selected survey and the results do not necessarily
>>> reflect the population. I would certainly not trust any conclusions drawn
>>> based on this study.
>>>
>>>
>>>
>>> *Elaine B. Fitt Ph.D.*
>>>
>>>
>>>
>>>
>>>
>>> *Elaine B. Fitt LLC*
>>>
>>> *Phone: 215 913 4379 <215%20913%204379>*
>>>
>>> *Email: [log in to unmask] <[log in to unmask]>*
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> This message, including any attachments, may contain confidential
>>> information intended for a specific individual and purpose and is protected
>>> by law. If you are not the intended recipient, please delete it. Any
>>> disclosure, copying, or distribution of this message is strictly
>>> prohibited.
>>>
>>>
>>>
>>> *From:* Educational Research Methodology List [mailto:
>>> [log in to unmask]] *On Behalf Of *Rueben Ternes
>>> *Sent:* Friday, April 18, 2014 10:44 AM
>>> *To:* [log in to unmask]
>>> *Subject:* Re: [EDRESMETH-L] ATTITUDES TOWARD STANDARDIZED TESTING:
>>> HUMAN SUBJECTS WANTED
>>>
>>>
>>>
>>> Answer: All Mechanical Turks?
>>>
>>>
>>>
>>> On Fri, Apr 18, 2014 at 10:12 AM, Dennis Roberts <[log in to unmask]>
>>> wrote:
>>>
>>> On 4/17/2014 10:40 PM, Tanesia Beverly wrote:
>>>
>>> You are invited to participate in a study investigating the perceptions
>>> a=
>>>
>>>
>>> nd
>>> attitudes about standardized test taking ability.
>>>
>>>
>>> What I have noticed is that more and more folks are posting requests on
>>> the internet for "volunteers" to participate in some kind of a research
>>> study (usually a survey). To me this is not a good way to define and
>>> obtain a sample ... though it appears to be a fairly easy one. My question
>>> is ... let's say the researcher gets n=87 to respond .... WHAT IS THE
>>> POPULATION TO WHICH THESE RESULTS (whatever they are) CAN BE GENERALIZED?
>>>
>>>
>>>
>>> Educational Research and Methodology Listserv
>>> -----------------------------------------------------------------
>>> List Service Info http://listserv.uconn.edu
>>> To cancel your subscription address click please do the following:
>>> SEND an email to the following address:
>>> mailto (colon) listserv (at) listserv (dot) uconn (dot) edu
>>> Your email should contain only the message
>>> UNSUB EDRESMETH-L.
>>> Address problems with your subscription to: [log in to unmask]
>>> -----------------------------------------------------------------
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Reuben Ternes
>>> Research Associate
>>> Office of Institutional Research and Assessment
>>> 509 Wilson Hall
>>> Oakland University
>>> 248-370-2559
>>>
>>> Educational Research and Methodology Listserv
>>> ----------------------------------------------------------------- List
>>> Service Info http://listserv.uconn.edu To cancel your subscription
>>> address click please do the following: SEND an email to the following
>>> address: mailto (colon) listserv (at) listserv (dot) uconn (dot) edu Your
>>> email should contain only the message UNSUB EDRESMETH-L. Address problems
>>> with your subscription to: [log in to unmask]
>>> Educational Research and Methodology Listserv
>>> ----------------------------------------------------------------- List
>>> Service Info http://listserv.uconn.edu To cancel your subscription
>>> address click please do the following: SEND an email to the following
>>> address: mailto (colon) listserv (at) listserv (dot) uconn (dot) edu Your
>>> email should contain only the message UNSUB EDRESMETH-L. Address problems
>>> with your subscription to: [log in to unmask]
>>
>>
>>
>>
>> --
>> Barbara J. Helms, Ph.D.
>> Education Evaluation Consultant
>> 2144 Commonwealth Avenue
>> Auburndale, MA 02466
>> Telephone: 617-480-2260
>>
>>
>> Educational Research and Methodology Listserv
>> ----------------------------------------------------------------- List
>> Service Info http://listserv.uconn.edu To cancel your subscription
>> address click please do the following: SEND an email to the following
>> address: mailto (colon) listserv (at) listserv (dot) uconn (dot) edu Your
>> email should contain only the message UNSUB EDRESMETH-L. Address problems
>> with your subscription to: [log in to unmask]
>>
>
>
>
> --
> Reuben Ternes
> Research Associate
> Office of Institutional Research and Assessment
> 509 Wilson Hall
> Oakland University
> 248-370-2559
> Educational Research and Methodology Listserv
> ----------------------------------------------------------------- List
> Service Info http://listserv.uconn.edu To cancel your subscription
> address click please do the following: SEND an email to the following
> address: mailto (colon) listserv (at) listserv (dot) uconn (dot) edu Your
> email should contain only the message UNSUB EDRESMETH-L. Address problems
> with your subscription to: [log in to unmask]
>
--
Wm W. Pendleton
Professor Emeritus Emory University
3 Breakers Ct.
Salem, SC 29676
Tel. 864 944-2238
Educational Research and Methodology Listserv
-----------------------------------------------------------------
List Service Info http://listserv.uconn.edu
To cancel your subscription address click please do the following:
SEND an email to the following address:
mailto (colon) listserv (at) listserv (dot) uconn (dot) edu
Your email should contain only the message
UNSUB EDRESMETH-L.
Address problems with your subscription to: [log in to unmask]
-----------------------------------------------------------------
