“significant” p-value ≠ “significant” finding: The significance of statistical evidence for the true X (i.e., statistical significance of the p-value for the estimate of the true X) says absolutely nothing about the practical/scientific significance of the true X. That is, significance of evidence is not evidence of significance. Increasing your sample size in no way increases the practical/scientific significance of your practical/scientific hypothesis. “significant” p-value = “discernible” finding: The significance of statistical evidence for the true X does tell us how well the estimate can discern the true X. That is, significance of evidence is evidence of discernibility. Increasing your sample size does increase how well your finding can discern your practical/scientific hypothesis.

Opt-in samples are about half as accurate as probability-based panels

A cognitive walkthrough is a technique used to evaluate the learnability of a system. Unlike user testing, it does not involve users (and, thus, it can be relatively cheap to implement). Like heuristic evaluations, expert reviews, and PURE evaluations, it relies on the expertise of a set of reviewers to assess the interface. Although cognitive walkthroughs can be conducted by an individual, they are designed to be done as part of a group in a workshop setting where evaluators walk through a task in a highly structured manner from a new user’s point of view.

Broadly, these feedback surveys can be categorised into five groups: the pointless; the self-important; the immoral; the demanding; and the downright weird:

EMERGE (Evidence-based Measures of Empowerment for Research on Gender Equality) is a project focused on gender equality and empowerment measures to monitor and evaluate health programs and to track progress on UN Sustainable Development Goal (SDG) 5: To Achieve Gender Equality and Empower All Girls. As reported by UN Women (2018), only 2 of the 14 SDG 5 indicators have accepted methodologies for measurement and data widely available. Of the remaining 12, 9 are indicators for which data are collected and available in only a limited number of countries. This assessment suggests notable measurement gaps in the state of gender equality and empowerment worldwide. EMERGE aims to improve the science of gender equality and empowerment measurement by identifying these gaps through the compilation and psychometric evaluation of available measures and supporting scientifically rigorous measure development research in India.

The Meta-Analysis Learning Information Center (MALIC) believes in equitably providing cutting-edge and up-to-date techniques in meta-analysis to researchers in the social sciences, particularly those in education and STEM education.

We are a global collaboration aimed at improving evaluation practice and theory through co-creation, curation, and sharing information.

research on health comm messaging effects

This guidebook helps media professionals of small media houses develop a better understanding of how to use data for improving their social media performance. Also includes worksheets and templates.

Breakthrough ACTION has distilled guidance on social and behavior change (SBC) monitoring methods into a collection of technical notes. Each note provides an overview of a monitoring method that may be used for SBC programs along with a description of when to use the method and its strengths and weaknesses.

We applied a Hidden Markov Model* (see Figure 1) to examine how and why behaviours did or did not change. The longitudinal repeated measure design meant we knew about food waste behaviour at two points (the amount of food wasted before and after the program), changes in the amount of food wasted reported over time for each household (more or less food wasted) and other factors (e.g. self-efficacy). By using a new method we could extend our understanding beyond the overall effect (households in the Waste Not Want Not program group wasted less food after participating when compared to the control group).

resources for mapping, assessing and weaving networks

Guide to outcome evaluation and development of logic models

Web Metrics, YouTube Basics and Mobile Metrics Guides