BBC Media Action conducted its first ever randomised control trial (RCT) on the impact of our health programming on audiences. In this blog, we explore some of the methodological challenges of conducting an RCT and ensuring randomisation in the field based on our work with pregnant mothers and women of childbearing age in Bangladesh.
This is the second blog in a two-part series on BBC Media Action’s Bangladesh RCT, read more about the results of the study in the first blog.
The ‘gold standard’ approach for being able to talk about an intervention causing an effect comes from the world of medicine: the randomised control trial (RCT). In this kind of study, one set of people – ‘the intervention group’ – receives the treatment while the other group – ‘the control group’ – gets a placebo. Only using this tightly controlled research methodology can we be certain whether or not the intervention caused the desired outcome.
Our RCT was interested in investigating how watching our Bangladeshi health programmes affect the key ‘drivers’ of healthy behaviour among women of childbearing age. These drivers include things like people’s knowledge of antenatal and early newborn care, their attitudes and beliefs around, for example, what to feed a newborn baby, and their intention to do things such as attending antenatal care sessions.
Our study involved 900 women of reproductive age as this group is the key audience we are aiming to influence with our programming, and took place over six weeks in February/March 2016 in two areas (Comilla in the South East and rural Mymensingh). Each day, a group of 30 women was sub-divided into groups of 10 (two treatments and one control) and were with us for around four hours. One treatment group watched our health drama Ujan Ganger Naiya (UGN) (Sailing Against the Tide), another group watched a closely related discussion programme that reinforced the health topics covered in the drama and a third control group viewed a television programme about a non-health topic.
Since mass media can reach anyone and everyone, evaluating whether a mass media intervention has had any causal effect on audience behaviour is notoriously tricky. Let’s look at some implementation challenges that our research team had to overcome:
First, we had to recruit a control group – one that had not been exposed to the treatment – which meant only recruiting participants who had never seen or heard anything about these shows. (See our previous blog for a description of the research design for this trial).
Another challenge was avoiding contamination – ensuring that people did not discuss what they had viewed. Each day, women were collected from different unions (local Bangladeshi political districts) so that there was no risk of anyone going home and speaking about the trial with a future participant. If participants needed to leave the room during the trial, they were escorted to make sure that no one conversed with each other.
The biggest challenge – ensuring randomisation – i.e., making sure the groups were more or less alike on all key variables – is a common difficulty in RCTs. To address this challenge, we created a randomisation matrix so that women were randomly assigned to the three groups and given a colour-coded wristband. There were three colour possibilities which referred to three treatment groups, i.e. one group was shown UGN and a non-health related programme, the second was shown UGN and Natoker Pore (NP) (After The Drama), the follow up discussion show, and the third group was shown an educational drama and discussion show – both on topics unrelated to health. Moreover, which colour band stood for which group was not revealed either to the participant nor the researchers at any stage and was also changed every day. This is known as a double blind process where neither the researchers nor the participants are aware of the treatment allocation to ensure there is no chance for bias.
This means we can be confident that the changes we saw were not due to some pre-existing selection bias such as education level or age.
“It was the most tense part of the study because everything hinged on achieving the randomisation which is difficult with 900 women and their children running around. We allocated most of our attention and resources to this during the fieldwork,” said Sanjib Saha, former Head of BBC Media Action Research, Bangladesh.
Besides all this, it is also vital to ensure comparability. To do this in our study, a health service provider gave the women a standard briefing on maternal health issues – the same as the one given by health workers when visiting women in their homes – to assure that there was a pre-trial standardisation of health knowledge. The briefing was identical each day across all groups. Those in the control groups were given a non-health related briefing.
Finally, a word is in order on ethics. As with any research we conduct, we took all steps possible to ensure that this research upheld the highest international ethical standards, at all phases of the research. Ultimately, we sought to ensure that all participants were protected from harm that might result from their participation in the study. This was a time-consuming study to be involved in, particularly for women with small children. We tried to smooth the process by arranging transport to and from the testing facility, making provisions for chaperones and providing lunch and child care at each of the test centres. All participants provided informed consent, were guaranteed anonymity and were apprised of their right to withdraw from the study at any point. We also made sure that a frontline health worker was available on site throughout the study to answer any questions.
The study was successful. We now know that our health programmes in Bangladesh are having a significant effect on some of the key drivers of health-related behaviours – especially knowledge and intent – in a laboratory setting. We are also reassured that, for knowledge and behavioural intent, watching a factual programme alongside a drama seems to be beneficial.
The BBC Media Action Research and Learning team manages a global cohort of more than 100 researchers around the world who inform, evaluate and generate evidence on BBC Media Action projects across the countries in which we work.
0 Comments