Improving Research at RSA

Rather than documenting individual research case studies for specific brands, this piece highlights how I’ve taken my experience and knowledge and helped the UX team at RSA improve the standard, consistency and efficiency of their usability testing. Having worked with a very experienced Research lead at TUI, I was keen to take my learnings to the team at RSA and develop the maturity of the research practice.

At RSA I’ve lead and run usability tests for a variety of brands including MoreThan, Argos and Nationwide as well as on lower fidelity prototypes. From lab based testing in London, online testing, to guerrilla testing in local cafes and the office canteen, I’ve developed assets, practices and mentored members of the team to improve the insight we get.  

Lab testing

Lab studies are expensive, time consuming to organise and often have a lot of focus on them to run correctly. The ensure they’re as effective as possible, I’ve created templates for:

-Participant criteria for recruiters.
-The research plan for stakeholders to highlight the purpose & goals of the study.
-The discussion guide for the day which includes a flexible script.
-The pre-test communications ‘What to expect from testing’ for stakeholders.
-Instructions for observation room on how to document observations.
-The introduction facilitation script for observers.
-The post test documentation outline (to help drive a presentation deck)
-Co-creation of the presentation deck.

I’ve also produced checklists for the team member running the lab study to help them plan their time and make sure they remember everything. There are four checklists covering:
-3/4 weeks before the lab test.
-1 week before the test.
-On the day of the test.
-The week after the test. 

Guerrilla testing

To keep the guerrilla tests (with external participants) as consistent and quick to organise as possible, I’ve also re-purposed templates for:
-The email invite for participants.
-Discussion guide for the session which includes a flexible script.
Due to their nature, not all guerrilla tests I’ve run need as rigorous planning and administration but having them available has helped the team (and myself!) create a base level structure for introductions, settling the participants, setting the scene and asking questions.

The most challenging part of guerrilla testing for us at RSA has been finding willing participants to take part in the tests. To help with this I begged / borrowed / bought cakes for colleagues to create a participant list of internal staff. For local participants who don’t work at RSA I was lucky to have inherited the start of a list and have helped maintain and grow the list for future sessions.

To continue to grow the list we approached the marketing team and added a section to the MoreThan quarterly newsletter, promoting the sessions we run and inviting people to volunteer. This linked to a form on the MoreThan website which gathered key contact details and gave a bit more information on what we were looking to run. Once we had a list of names (we got over 100) we contacted them to help set expectations of how often we would be in touch and what kind of activities we would get them to complete. As many of the names were from all over the UK, the activities we planned were surveys, online tests as well as local studies where possible. Maintaining the list is an issue, as is running out of local participants so longer term we would look to use a recruiter to help us manage people.

Mentoring

As well as the documentation and checklists, I helped develop the experience and confidence of more junior members of the team. This started with inviting them to take notes at guerrilla testing, taking notes and helping facilitate observers in lab testing and helping organise lab testing. It led to practice reading through discussion guides with me, talking about questions and objectives of the tests then taking notes as they started to run a few guerrilla tests on internal staff. Following another a key lab test in London, I guided them through the process of analysing the findings of the tests and creating a presentation pack to play back to stakeholders. 
After getting more internal guerrilla tests completed, I then assisted them setting up a lab test and acted as facilitator as they moderated the 3 afternoon sessions.