

Case study 1 - Response to NSW Government tender for prison program evaluation
Case study example for ethical guidelines
An Invitation to Offer has been issued by Corrective Services NSW for an evaluation of a prison anger management program that is specifically for Aboriginal and Torres Strait Islander males. The program is called Men’s Business and it has been running for two years in Broken Hill Correctional Centre and for one year at the Grafton Correctional Centre (which adapted the program developed for the Broken Hill Correctional Centre). Both prisons are medium security and managed by Corrective Services NSW. A local Broken Hill Aboriginal Elder, Uncle Bevan, and the Broken Hill Aboriginal Health Services designed and developed the program. The Men’s Business program runs for 10 sessions, comprising four hours, one day per week. It is offered twice a year and attendance is ordered by Corrective Services or prisoners can voluntarily choose to attend. Completion of the program is viewed favourably by the Parole Board when a prisoner applies for parole. On average 45 prisoners attend each session of the program across the two prisons and 22 prisoners have completed the program each time it has been offered. The program is designed to reduced anger arousal and improve a person’s control over their anger, particularly in a family setting. Most of the Aboriginal and Torres Strait Islander men who attend the prison program have been sentenced for domestic and family violence related offences. Many have substance abuse problems. The Men’s Business program is considered culturally appropriate because aside from cognitive behavioural therapy, it utilises creative healing techniques, such as performance, creative writing and painting to assist the men to express and understand their anger impulses. There are other anger management programs in other NSW prisons, including the Broken Hill Correctional Centre, which are not culturally-specific. These programs use cognitive behavioural methods to educate participants about why they get angry and encourage anger control.
The Invitation to Offer requires the successful bidder to evaluate the effectiveness of the program, including:
-
Whether the Men’s Business program is operating as intended;
-
Ways to improve the operation of the program;
-
To what extent the program is achieving a reduction in anger (in and outside prison), and domestic and family violence reoffending;
-
Whether the program is meeting the cultural needs of the participants;
-
Whether the program is cost-efficient.
You decide to submit a proposal and offer for the Invitation to Offer with a team of researchers from your university. The timeframe provided in the Invitation to Offer for the evaluation to be conducted is 12 months from start to finish.
What needs to be addressed in the proposal for the Invitation to Offer, in order to satisfy the ideas expressed in the four AIATSIS ethical principles, (which reflect the principles set out in the NHMRC Ethical Guidelines and the NHMRC Keeping Research on Track II compendium document, is outlined below. Before submitting the proposal consideration should also be given to whether you and your team are the right people to conduct the research. Consider the questions asked in the AIATSIS 'A Guide to Applying The AIATSIS Code of Ethics for Aboriginal and Torres Strait Islander Research'.
Applying the AIATSIS ethical principles
AIATSIS Principle 1: Indigenous self-determination
-
The Invitation to Offer needs to recognise and value the cultural distinctiveness of the Broken Hill and Grafton Aboriginal communities. Therefore, it may not be appropriate to develop an evaluation framework that treats the two programs as one unique program. This will depend on how the programs are designed and delivered at both prisons. This is required despite the fact that the men who participate in the programs may be from various locations throughout NSW, since prisoners are sent and transferred to prisons according to directives issues by Corrective Services, NSW.
-
Consideration should be given to what type of data needs to be collected to address the evaluation criteria. For example, is there a need to collect data about all types of post-program re-offending or behaviour that has attracted disciplinary action within the prison for each program participant? Or should it just be focused on domestic and family violence related offending or disciplinary action imposed due to anger-related behaviour?
-
An Advisory Group should be set up in both Broken Hill and Grafton consisting of key Aboriginal stakeholders, including the program leaders and designers needs to be established whilst preparing the Invitation to Offer. This is to reflect that the Aboriginal peoples of the two locations have distinct cultures and ways of knowing being and doing. The Advisory Groups should be consulted regarding best practice for obtaining consent from program participants, including advice regarding whether or not consent should be oral rather than written.
-
Best efforts should be made to include Aboriginal and Torres Strait Islander researchers on the Research Team, either by way of Chief Investigators or Research Assistants or both. For this consultancy, it would be advisable to include male Aboriginal and Torres Strait Islander researcher/s, demonstrating an understanding of cultural protocols and worldviews.
.
Principle 2: Indigenous Leadership
-
A collaboration agreement, setting out the role and place of all researchers, should be entered into between Uncle Bevan and the Broken Hill Aboriginal Health Services.
-
It would not be a conflict of interest for Uncle Bevan or staff from the Broken Hill Aboriginal Health Services to be involved in the planning of the research even if they are then included as interview participants. Using a collaborative participatory approach is recommended for Indigenous-focused research.
-
Methodologies that recognise the epistemologies and perspectives of the Barkandji and Bundjalung people and that adopt a decolonising methodological approach should be used. This would involve honest and open consultation with respected Elders and program leaders and designers about what data will be collected, how the data will be collected and how it will be analysed and reported. In ascertaining these parameters, there needs to be careful consideration of the aims and goals of the program and what outcomes can realistically be expected. Assessing outcomes such as impact on reoffending, when there are serious questions about the generalisability or ability of the data to definitively point to such outcomes should be avoided. Using a realist evaluation framework and mixed methods (qualitative and quantitative) approach is recommended. A qualitative approach that has been used to measure intervention impact on reoffending is one that focuses on pathways to desistance (see Daly & Proietti-Scifoni, 2009).
-
Ownership of the data that is collected should be clearly stated in the collaboration agreement and communicated to Corrective Services NSW.
Principle 3: Impact and Value
-
When engaging with the Advisory Groups, consideration of the value and impact of the evaluation is necessary. Questions need to be asked about whether assessing the impact of the program on reoffending is appropriate considering the original goals and aims of the program. For example, although reducing reoffending may be a long-term goal of the program, more immediate effects may be improving the wellbeing of the men who participate in the program.
-
Changing the focus of the evaluation will need to be negotiated with Corrective Services NSW, by explaining that using a realist evaluation approach the evaluation will be theory-driven and informed ensuring that there will be a more in-depth and comprehensive understanding of what works, for whom and in what circumstances.
-
If there is a risk that assessing reoffending using positivist, quantitative methodological approaches may lead to findings or results that do not support the underlying aims of the program, that sort of methodological approach should not be used.
Principle 4: Sustainability and Accountability
-
If data is collected that contains traditional knowledge or cultural expressions, permission to use that data is imperative. Ownership in, and therefore control of, that data remains with the Traditional Owners. In a project such as this one, traditional knowledge may include cultural matters discussed in the program workshops or personal information divulged by the men during workshops. Researchers need to be mindful not to disclose this type of information, particularly if there is a risk of identifying individuals or if it breaches cultural protocols.
-
The results and findings need to be disseminated in a way that is accessible, not only by stakeholders, but also by program participants. Due to the transitory nature of prisons, whatever approach is used may not target the program participants who participated in the project’s data collection. How best to disseminate the results and findings should be discussed with the Advisory Groups. Some examples include a one-page flyer written in accessible language that can be disseminated to the men in prison via the workshop facilitators; a presentation for the men during one of the workshops; the production of a short video that can be uploaded onto the Internet and also distributed in DVD format.