Structured interviews: Follow a predefined set of questions, with both the topics and order of questions determined in advance.
Semi-structured interviews: Use a set of planned questions or themes while allowing the interviewer to introduce additional questions as the conversation develops.
Unstructured interviews: Do not follow a predetermined list of questions, allowing the discussion to develop naturally based on participants’ responses.
Focus group interviews: Involve asking questions to a group of participants rather than an individual, with the goal of exploring shared perspectives, discussions, and group dynamics.
Some variables have fixed levels. For example, gender and ethnicity are always nominal level data because they cannot be ranked.
However, for other variables, you can choose the level of measurement. For example, income is a variable that can be recorded on an ordinal or a ratio scale:
At an ordinal level, you could create 5 income groupings and code the incomes that fall within them from 1–5.
At a ratio level, you would record exact numbers for income.
If you have a choice, the ratio level is always preferable because you can analyze data in more ways. The higher the level of measurement, the more precise your data is.
The level at which you measure a variable determines how you can analyze your data.
Depending on the level of measurement, you can perform different descriptive statistics to get an overall summary of your data and inferential statistics to see if your results support or refute your hypothesis.
In psychology, inter-rater reliability refers to the degree of agreement between different observers or raters who evaluate the same behavior, test, or phenomenon.
It ensures that measurements are consistent, objective, and not dependent on a single person’s judgment, which is especially important in research, clinical assessments, and behavioral studies.
High inter-rater reliability indicates that results are dependable and reproducible across different raters.
There isn’t just one formula for calculating inter-rater reliability. The right one depends on your data type (e.g., nominal data, ordinal data) and the number of raters.
Cohen’s kappa (κ) is commonly used for two raters
Fleiss’ kappa is typically used for three or more raters
The Intraclass Correlation Coefficient (ICC) is used for continuous data (interval or ratio). This is based on analysis of variance (ANOVA)
The most used formula (for Cohen’s kappa) is:
Po is the observed proportion of agreement, and Pe stands for the expected agreement by chance.
