Impact of RAPP XXVIII - Pre- & Post-Test Analysis

The annual assessment of RAPP's impact through pre- and post-test analysis is completed (in first draft)!

Curriculum and Measurement

The flagship program of the Racial Awareness Program is the RAPP 9-month experiential social justice education program.  The 2012-13 class of RAPP was the 28th group of University of Cincinnati students to participate in this program.  As such, they are known as RAPP XXVIII.

The curricular objectives for the RAPP 9-month program were established in Fall 2008 and updated in Fall 2012 to be:

• Develop relationships with new people
• Develop ability to communicate across difference for learning and understanding
• Recognize that oppression is systemic, pervasive, interlocking, and imbedded
• Recognize own agency

The RAPP XXVIII experience was facilitated by the RAPP Program Coordinator Rebecca Lehman and RAPP Student Worker Brice Mickey.  Following the set of session goals that constitute the RAPP Curriculum, these facilitators developed 14 sessions and three overnight retreats utilizing social justice education resources and intergroup dialogue practices.  Three student volunteers also helped facilitate the RAPP XXVIII group process as peer educators and mentors – Farooq Alkhateeb, Tianshu Lu, and Kristen Myers-Young – having previously participated in the RAPP 9-month program and received pre-service and on-going training in social justice peer education.

The pre- and post-test instrument was developed and first implemented in Fall 2009 and has been updated annually by the facilitation team to reflect participant feedback and updates in the curriculum as well as issues found in assessment.  RAPP XXVIII is the fourth group on which it has been administered.  The pre-test was administered at the second meeting of the group and the post-test was administered at the twelfth (penultimate).

General Results

Twenty-eight students committed to RAPP XXVIII and 24of these graduated through the program in April 2012 (85.7% retention).  Twenty-one of these graduates completed both the pre- and post-test (87.5%).

All RAPP members committed to participating in all RAPP XXVIII meetings and retreats (each meeting is one session, each retreat has three sessions) and completing brief activities between sessions (e.g. "write the meeting dates in your calendar," "record a page of reflection in your journal"); RAPP XXVIII had a total of 22 sessions and 13 between sessions activities.  

The overall average for participation was attendance at 89.9% of sessions.  Previous analysis demonstrated that participating in 80% or more of sessions is critical to achieving the learning objectives; 21 participants (87.5%) of the group attended 80-100% of the sessions.

Key Findings

By section, the key highlights are:

• Identities – Participants reported an overall increase in racial and gender identity awareness.  When divided into dominant and subordinated group identities, the greatest reported change was among racial identity awareness of White/Caucasian students.
• Comfort in Discussion – Participants reported an increase in the number of people in the group with whom they report being comfortable discussing difficult topics.  Participants also reported any increase in their overall comfort discussing racism, sexism, cissexism, and heterosexism.  RAPP participation enhanced participants general awareness and knowledge related to oppression and their skill in having conversations around it.
• Key Words – Participants demonstrated an increased ability to define and quality of definition of key words related to social justice education (oppression, privilege, ally, social construct, and intersectionality).  The greatest demonstrated increase was for social construct.
• Challenging Others – Participants reported being more likely to verbally express disagreement with statements they found offensive made by people across all relationships surveyed.  The biggest reported increase was with expressing disagreement with co-workers and strangers.
• Communicating – Participants reported an increase in use of cross-cultural communication strategies with all relationships surveyed.  The greatest reported increase was in acknowledging multiple perspectives.
• Identifying Racism – Participants were slightly more likely to identify the connection with racism in institutional and social levels.
• Feedback – In response to open-ended questions on the post-test, participants provided feedback on the overall RAPP XXVIII experience.

o   Learnings/Enhancements:  All participants were able to describe something they learned or enhanced through their participation.  Most frequently noted were learning in line with the “each one teach one” philosophy, understanding privilege, recognizing oppression as complex, and communication skills.

o   Desired changes:  Participants provided a variety of suggestions for improving the RAPP 9-month process, most frequently noted were the meeting timeframe, providing social activities between sessions, and increasing the use of educational simulations in sessions.

o   Sense of connection with UC:  Four-fifths responded with an unequivocal yes when asked whether RAPP impacted their sense of connection with UC; all respondents described ways in which RAPP positively impacted their UC experience.

For full reporting of findings, please contact RAPP Program Coordinator Rebecca Lehman for the full report.  Included at the end are important-to-note limitations to the data and analysis.

Connected with Campus - Impact of RAPP XXVIII

Throughout the summer as analyses are completed on the 2012-13 groups in the RAPP 9-month, Accelerating Racial Justice, and the Social Justice League, I'll share insights from the results here on the blog.

A new question we added to the post-test for RAPP XXVIII was "Has participation in RAPP XXVIII impacted your sense of connection with the University of Cincinnati?  How and/or how not?"

Four-fifths of the group gave an unequivocal and literal yes - RAPP positively impacted their sense of connection with UC.  Here are some of their comments on how RAPP did this:

"I felt zero connection to the university.  I'm a non-traditional student who went to class and went home.  Now I know I have a family, a safe space, and the university is a living organism to me instead of a lifeless slab of concrete."

"At first when I arrived at UC I was another face in the crowd. Now I know people.  I'm not just a commuter and I do plan on joining more clubs."

"I'm glad I became a member of RAPP, I've expanded my knowledge about the world I live in, and I've also become a smarter and wiser person because of it.  Because of RAPP, next year I plan on joining more groups where I can apply the skills I've acquired through RAPP 28!!"

"I am proud that UC continues RAPP.  It lets me know that 'inclusivity' and 'acceptance' and 'diversity' aren't just filler words.  Some people are trying."

"I feel connected with students who are RAPP alumni and I get respect from faculty & staff.  I have a great understanding of the inequality in how this institution is run and leaders' positive and negative influence.  I have a greater awareness of my fellow students' demographics & issues and of programs related to RAPP."

Those who did not say a full "yes" to sense of connection with UC shared ways in which RAPP positively impacted their experience and and feelings toward the university:

"It has brought a positive to my feelings about the university."

"Not so much with the university itself but definitely with some of the students (RAPPmates) and faculty (facilitators)."

"It has made me realize that I am closer to the African American community on campus than I thought I was because many of our experiences are similar."

"I suppose. I'm NOT a school spirit person, but this program has a lot of value & is unique to UC."

While RAPP may do something people find risky for the institution - acknowledging that as an institution we practice the same oppression we're taught in our society - we also succeed at helping people connect with it.  We open up the complex ways that we work to combat the oppression we've been taught and are steeped in daily, just as we ask RAPPers to join us in doing as individuals.

It may initially seem like an unusual way to develop Bearcat Pride, but it certainly is a powerful and impactful way!

Impact of RAPP XXVI

Though it's taken me all summer to code and analyse the data, I'm ready to announce the  pre- and post-test comparison of RAPP XXVI!  This is our second year being able to provide this information. 

As with any survey-based study, there are definitely limitations to the information pulled from the data. The survey instrument itself and the way in which the data was recorded and analyzed are all up for question.

Below is the overview of the report.  The full copy - including data/analysis limitations - should be posted on the RAPP website shortly.  In the meantime, if you'd like a copy please send me an email to request it!  --Rebecca L

Curriculum and Measurement

The flagship program of the Racial Awareness Program is the RAPP 9-month experiential social justice education program.  The 2010-11 class of RAPP was the 26th group of University of Cincinnati students to participate in this program.  As such, they are known as RAPP XXVI.

The curricular objectives for the RAPP 9-month program were established in Fall 2008 as:

• Develop relationships with new people
• Develop ability to communicate across difference for learning and understanding
• Recognize that oppression is systemic, pervasive, interlocking, and imbedded
• Appreciate cultural difference
• Recognize own agency

The RAPP XXVI experience was facilitated by the RAPP Program Coordinator, an AmeriCorps Public Ally apprentice, and a Women’s, Gender, & Sexuality Studies undergraduate intern.  Following the set of session goals that constitute the RAPP Curriculum, these three facilitators developed 11 sessions and three overnight retreats utilizing social justice education resources and intergroup dialogue practices.  Five student volunteers also helped facilitate the RAPP XXVI group process as peer educators and mentors – these five Peer Leaders had previously participated in the RAPP 9-month program and received pre-service and on-going training in social justice peer education.

The pre- and post-test instrument was developed and first implemented in Fall 2009.  RAPP XXVI is the second group on which it has been administered.  The pre-test was administered at the second meeting of the group and the post-test was administered at the eleventh (penultimate).

New to the 2010-11 analysis is exploration of the impact of participation on results, measured by amount of sessions attended.  The group was separated into two portions: Those that attended 81% or more of sessions (n=24; referred to from here out at “high commitment group”) and those that attended 76% or fewer sessions (n=17; referred to as “lower commitment group”).

Also new this year was the addition of three open-ended questions at the end of the post-test to gather general feedback from the group in reflecting on the 9-month experience

General Results

Sixty-eight students committed to RAPP XXVI, 59 of these graduated through the program in May 2011 (86.8% retention).  Forty of the 59 RAPP XXVI graduates completed both the pre- and post-test (67.8% of graduates).

All participants committed to participating in all RAPP XXVI meetings and retreats.  Participation was explored in detail in each test section due to the unusually low participation rate of RAPP XXVI members compared to the previous two years in which data was kept.  While ten students (17% of group) attended every meeting and retreat and another ten missed only one meeting or retreat during the year, the overall average for participation was 78%.  Thus, in analysis, the group was divided into “high commitment” (higher than the average, or those with 81-100% participation) and lower commitment (lower than the average, or those with 76% or lower participation).

Key Findings

By section, the key highlights are:

·         Identities – Overall, participants reported an increase in racial awareness.  When divided into dominant and subordinated group identities, White/Caucasian participants reported a strong increase in racial awareness and cisgendered men reported a moderate increase gender awareness.  Students who identified as identities other than heterosexual reported a strong increase in sexuality awareness.

·         Key Words – Overall, participants demonstrated an increase ability to define the provided key words (oppression, privilege, internalized oppression, ally, social construct, and intersectionality).  The greatest demonstrated increase was for intersectionality.

·         Challenging Others – Overall, participants reported being more likely to verbally express disagreement with statements they found offensive. The biggest reported increase was with expressing disagreement with strangers and professors.

·         Communicating – Overall, participants reported a slight increase in use of cross-cultural communication strategies with acquaintances.

·         Exploring Racism – Overall, participants were more likely to identify the connection with racism in individual, institutional, and social/cultural scenarios.  The greatest change overall occurred with recognizing connections to institutionalized racism.

·         Participation – Participants' level of commitment demonstrated impacts in the following areas:

o   Identities – High commitment participants demonstrated a strong increase in racial awareness, compared to a decrease in the lower commitment group.

o   Key Words – On all key words except for internalized oppression, the high commitment group demonstrated a greater increase in ability to define the key words.  The high commitment group had a substantially stronger increase on intersectionality, privilege, oppression, and ally.

o   Challenging Others – The high commitment group reported an increase in expressing disagreement in all relationship categories except for roommate (which held steady from pre- to post-test).  The lower commitment group reported a decrease in expressing disagreement in four of the eight relationship categories, holding steady in three others and demonstrating a small increase with professors.

o   Communicating – When combined, the group reported an increase in use of cross-cultural communication strategies in only one of three categories (acquaintances).  When split along commitment, the high commitment group demonstrated an increase with all three (acquaintances, good friends, and close family members) whereas the lower commitment group reported a decrease in use of these strategies with all three categories.

o   Exploring Racism – Participation had a mixed impact in recognizing different levels of racism.

·         Feedback – In response to open-ended questions on the post-test:

o   The five most commonly reported words to describe the RAPP XXVI experience were educational, fun, eye-opening, enlightening, and life-changing. 

o   The most commonly reported dislikes within the experience were around the time involvement of three-hour meetings, attendance issues, and issues within the process. 

o   The most commonly cited take-aways were multiple perspectives on issues/experiences, relationships, knowledge, and a sense of agency.

Included at the end of the full report are important-to-note limitations to the data and analysis.

A word cloud (created with Wordle) of the descriptions participants gave of their RAPP experience.  Click for larger view.