Conversocial
Q1-Q2 2019
Project lead, working alongside a senior researcher—Responsibilities include project direction, defining research objectives and methods, participating in research activities, insight extraction and delivering executive report.
Agent workspace is an enterprise web app that allows brands to connect with their customers through a verity of different social and private messaging channels, at scale. It has powerful routing, priority and automation capabilities that help drive efficiencies for users in large customer service teams.
Company wide opinions were that the Agent Workspace needed improving. Any improvements should positively impact Average Handling Time, a key efficiency metric that brands use to measure the success of their customer service teams.
But what needed to be done? The design team embarked on a 3 month research project to find out…
San Francisco, New York, Cancun, Glasgow, Dundee and Birmingham
Across 5 Verticals
24 Users, 14 Managers
Across 5 teams
x3 to extract insights
The project comprised of three stages:
The Stakeholder and Data stages would inform the User Research stage, helping us identify gaps in our current understanding of users’ needs, expectations and experiences.
The team created research plans for each stage, defining goals, hypotheses and research methodologies.
Agents who are more satisfied with Agent Workspace are more efficient at responding to customers—data to support this would validate investment into improving agent experience as well as agent efficiency, which has been prioritised historically.
The team interviewed 16 internal stakeholders across various functions within the company, including the exec team, sales, marketing, product and engineering. Stakeholders can have a deep undertanding of the business and industry and give some steer to research direction.
Participants were asked for their opinions on:
Stakeholders thought that by reducing confusion, integrating with more systems, automating the product and creating better user flows, agent efficiency would improve for Agent Workspace
Stakeholders thought that automating features, enhancing UI, giving agents access to information they need and customising experiences, could differentiate Agent Workspace
Product Managers were able to help us identify key workflows and important features within them, guiding further research:
What insights could be gathered from data analysis? The aim was to measure the effectiveness, efficiency and satisfaction of flows & features for overall performance and use the data to benchmark.
With limited engineering resource, the team was not able to retrieve the data required. We pivoted, reversing the strategy by swapping the user research and data phases. Identifing user pain points first then checking data to confirm insights.
Agents (our primary users) were sent a UEQ usability survey that would help benchmark percieved ease of use and user experience for Agent Workspace. Key insights to note were below average results for efficiency and stimulation. Using SEQ, ease of use was scored at 5.7/7.
The survey was to be sent to users periodically to track usability improvements over time.
Scoring ‘Good’ for learnabilty, ‘Below Average’ for attractiveness, efficiency, dependability & stimulation and ‘Bad’ for novelty
Conversocial’s Data Science team helped by sharing important user flow data. The data gathered proved that there was no one true user flow for issue resolution, leaving some features unused, depending on the type of issue and brand.
Over the course of 2 months the team travelled to 6 different locations, globally to observe and interview users from 11 different customers. Each had very different customer service needs and practices, as seen by the contrast in workspaces.
Participants took part in sessions lasting ~45 minutes approximately. Each session consisted of observing the participant in their normal working environment whilst they went about their day, with the team taking notes and asking questions where appropraite.
This was followed by a short interview where the team could dig a little deeper into what they had just observed as well as asking questions about working habbits and perceptions about Agent Workspace.
Customer Service Agents Observed
Custome Service Managers Interviewed
User Pain Points Observed
Products or Features Requested
Insights were extracted from notes and recordings. Methods like Affinity Mapping helped the team find themes and patterns to help categorise and prioritise feedback. The categories below best represented perceived problems within Agent Workspace including severity level.
Below are some key takeaways with recommended solutions, based on insights gained from the entirety of the research project including stakeholder opinion, data and user feedback.
Different brands are resolving issues in different ways, creating their own user flows in the process. This is leading to some features not being used per brand and valuable real estate going to waste. On the flipside, agents are taking actions that are not necessary because reduntant UI is still available to them
Recommended - Configurable & Customisable UI, Guidance for Users, Side Panel Overhaul
33% of participants used other tools to store and/or gather information that they need to help the customer. Agent Workspace provides features that help agents with this but due to poor design they are neglected.
Recommended - Clipboard Overhaul, CRM Integrations Overhaul, New Suggested Responses feature
100% of participants were observed scrolling through message content to find details in every conversation. This is a huge efficiency loss.
Recommended - Conversation Summary UI
One example of this is Playmode, a feature designed to automate the distribution of converations and to stop cherry picking of conversations doesn not stop agents from cherry picking, resultung in customers being left waiting for responses. Other examples are Clipboard and Integrations.
Recommended - Playmode Iteration
Conversations splitting after 40 messages, spam content, and disappearing conversations are examples of bugs that slow agents. Manual actions like adding tags, refreshing conversations, setting up Agent Workspace and chasing approvals could all be automated to speed up agents.
Recommended - Review data to understand improvement benefit for each
Preconceptions were that research would inspire a completely new design paradigm for Agent Workspace, but that’s not what happened...
We identified many small to medium sized improvements that would collectively impact Average Handling Time and solve user problems. An iterative enhancement strategy was adopted. This gave the product and engineering teams the ability to focus and deliver quickly, as well as ensuring we could adapt to change and stay agile.
These features will define the product and design roadmap for at least 12 months.
Data collected supported the hypothesis that the more satisfied an agent the more efficienct they are at handling customer issues. Investment into improving product usability and agent experience will positively impact agent efficiency and should be considered in any product enhancements.