HP1100 Critique on Research Methods

In this study, Hannah Bowles and Michele Gelfand hypothesize that people
with a higher status level will demonstrate lower rule compliance and show stronger
hierarchical enhancing beliefs. The independent variable of the research is the status
level of the participant and the dependent variables are rule compliance and
hierarchical enhancing beliefs.
Single factor
The dependent variables rule compliance and hierarchical enhancing
beliefs are measured on an ordinal scale of participants likeliness to deviate
towards certain behaviours. For rule compliance, Bowles and Gelfand used follow
directions, stick to the rules disregard rules for rule compliance as guiding
principles. For hierarchical dominance, the researchers utilized recognition as a
guideline to measure the tendency for participants to veer towards hierarchical
dominance.
Using an ordinal scale makes the information collated from the survey easy to
process, as Likert-type scales are quantitative rather than qualitative in nature. Also,
there is presence of convergent validity as there are two definitions being used for rule
compliance s sticking to the tules and the following of directions. However, they are
only from one source, adapted from Magee and Galinsky. It would be beneficial for
the researchers to be able to broaden their definition of rule compliance in order to
corroborate and observe if questions derived from different researchers would get
similar responses from participants, thus strengthening the papers convergent validity.
(Multiple item scale)
Although using a Likert-type scale helps in the ease of analysis and is
presumably less subjective in nature, the survey responses are self-reported. There are
no multiple judges in order to confirm the validity of the participants observations of
themselves. Additionally, social desirability can be a shortfall in this experiment as
participants would experience evaluation apprehension, increasing the likelihood of
participants giving politically correct answers. To solve this, the researchrs can assure
confidentiality and anonymity.
Reduction of effect of leading questions can be done by using wordings of the
opposite effects. Since definitions used for the designing of questions are direct
opposites of each other, this helps to reduce counter-balance the effects of leading
questions as participants will be guided in different directions for different questions.
At the same time, using questions with completely opposite meanings will allow
researchers to counterbalance the effects of acquiescent bias as they will cancel out.

The independent variable status is manipulated through a pen and paper

manipulation through an online survey. Participants are divided into two groups of
high or low status. In their respective groups, they are required to recount their
experience, and how they felt in that particular situation.

An advantage to the manner the independent variable was manipulated is that

the process that participants experience are the same. This eliminates the possible
difference in fatigue level, which could have been a possible confound.
Random assignment has already been done to the participants, which ensures a level
of internal validity. However, it would be better if the researchers has done a
matching process, where participants of similar characteristics set beforehand are
paired, then randomly assigned to either one of the two groups. This can help achieve
higher group equivalence, which improves the internal validity of the study.
As this research has a between-subject design, this reduces the chances of
sensitization of participants to the construct of the experiment. They go through only
one aspect of the experiment that is not repeated, making it less likely to be aware of
the construct.
There is the presence of an almost-equal number of males and females, but not. Not
matched based on gender. Males and females are inherently different (CITE). Would
thus affect survey results if males and females
Consider conducting a multiple group experiment with more than 2 levels./ This
would provide a comparison to the control. Identifies if the higher social sttus truly
causes higher hierarchical/rule compliance. This, however, is not very relevant as the
research hypotheses compares high social status to lower social status rather than to
the average.
Participants are American adult with work experience. They are randomly assigned to
their groups.
215 americans provide a very small sample size as a representation of the total
population of working adults in Singapore as there are ______ working adults
(SEARCH)
Mean working experience but could have those who just entered the job market with
less than a year of experience, while others could have more than 2 decades worth of
working experience. In the case where the range of working experience of participants
are not defined and age range is not defined as well, maturity or experience in the
working environment could drastically change the opinions of people at different
status levels.
Similarly, with the lack of an age or working experience range, the demographics of
the participants could be skewed in a particular direction, making the random
assignment conducted an ineffective one, as most people could possibly have a
different level of experience of maturity.
The outreach was conducted by a market research firm. This does not indicate how
participants are obtained from the market as well. It is unlikely that it is done based on
random selection, as the population is already limited to those who have a personal
computer. Those who volunteered may also have other reasons for doing so as they
could have possible benefits that come with doing the survey, attracting certain types
of participants that may have similar characteristics or traits. This can pose as a
problematic confound that would be difficult to eliminate. external validity.
Design.

Different experiences with high status or low status not enough of each? Different
opinions? More of one than another?
Small sample size also makes it hard to balance. It requires more time, people and
different methods, perhaps firstly from a laboratory-controlled environment where
participants can actually experience having a high or low status. Also, this would
allow us to record our observations of participants behavior with regards to the two
dependent variables, eliminating the problem of evaluation apprehension.
Hypothetical situation does not bring out how participants will react in a real
situation (SEARCH ANY EVIDENCE)

Independent variable: Status level

Dependent variable: Rule compliance and hierarchical-enhancing beliefs
Controlled variable: Presence of work experience, age-group, ratio of males to
females

study the effect of status level (independent variable) on rule compliance and
hierarchical beliefs (dependent variables).

Small sample size also makes it hard to balance the different opinions/ values from
people. It requires more time, people and different methods, perhaps firstly from a
laboratory-controlled environment where participants can actually experience having
a high or low status. Also, this would allow us to record our observations of
participants behavior with regards to the two dependent variables, eliminating the
problem of evaluation apprehension.

Reduction of effect of leading questions can be done by using wordings of the

opposite effects. Since definitions used for the designing of questions are direct
opposites of each other, this helps to reduce counter-balance the effects of leading
questions as participants will be guided in different directions for different questions.
At the same time, using questions with completely opposite meanings will allow
researchers to counterbalance the effects of acquiescent bias as they will cancel out.

As this research is conducted using a between-subject design, chances of sensitization

to the construct of the experiment is reduced. Participants go through only one aspect
of the experiment that is not repeated, making it less likely for subjects to be aware of
the construct. Also, an added advantage of a between-subject design would be the
reduction of fatigue, when compared to a within-subject design. Similarly, both
groups go through an identical procedure of recounting a past experience, ensuring
that the participants are conditioned in an identical manner. There is no difference in
time, effort and attention directed to the survey, which eliminates the possible

difference in fatigue level. Survey questions require the active processing of

information and possible confound in the experiment.
There is the presence of an almost-equal number of males and females, but not. Not
matched based on gender. Males and females are inherently different (CITE). Would
thus affect survey results if males and females
Research Design
Different experiences with high status or low status not enough of each? Different
opinions? More of one than another?

Also, two definitions are being used for rule compliance sticking to the rules
and the following of directions, making their . However, they are only from one
source, adapted from Magee and Galinsky. It would be beneficial for the researchers
to broaden their definition of rule compliance in order to corroborate and observe if
questions derived from different researchers would get similar responses from
participants, thus strengthening the papers convergent validity. (Multiple item scale)