Case Study
How mapping the journey of multi-cloud users aligned teams across IBM to streamline the content journey across public and private cloud.
Situation
Objective: Help users find the right documentation by consolidating content and redefining information architecture.
Most customers are looking for hybrid (public/private) cloud services and cannot understanding the landscape of IBM cloud offerings because the content and experience are maintained by siloed orgs.
- Marketing covers all things cloud (ibm.com/cloud).
- Public and private clouds have their own portals.
- Documentation for cloud services is scattered between IBM Cloud Docs and two legacy knowledge centers.
- IBM had recently acquired RedHat. The introduction of new content around hybrid cloud solutions (Cloud Paks) was an executive priority.
Tasks
As the Lead User Researcher embedded on this and one other workstream, I was responsible for planning and conducting research to provide user insights to collaborating teams.
In the absence of a Design Lead or Manager at the time, I was also responsible for partnering with the Content Architect (owner of IBM Cloud Docs) to plan the project, and facilitated all Design Thinking workshops.
Actions
Over 3 months, I conducted research and facilitated collaboration across the company by grounding activities and discussions with research insights.
- User interviews
- Card sort and ranking survey
- Competitor review
- Workshop
- Rapid Iterative Testing and Evaluation
- Findability benchmarking study
Results
Coordination across workstreams; including alignment on priorities and roadmap commitments to address a known and neglected user pain point (due to the scale of coordination required).
- Introduction of a global navigation
- Updates to marketing pages that complement the new docs homepage
- Hubs for API Docs and Tutorials
Interviews to understand how our customers think of and search for cloud services.
What are they searching for when exploring new technology? Do they seek out content specific to public and/or private cloud? What content types are they looking for?
Users primarily look for documentation when they are learning something new or when something goes wrong. Customers are leveraging hybrid cloud solutions and dividing content between public and private cloud leads to redundancy and confusion.
FAQs are easy to scan and can quickly communicate an offering's capabilities.
Diagrams are quick to consume.
Code snippets help demonstrate and can be copied easily.
Providing screenshots within the text eases scanning.
Survey to determine which content types users are interested in based on where they are in their journey.
- Multiple recruitment channels: vendors, CSM or DevAdvocate clients, Sponsor Users
- 120+ responses
Created participant segments based on task frequency. Marketing pages target 'Explorers', documentation and knowledge centers serve 'Builders'.
- 'Explorers' ranked content types based on what's most important to them when they are exploring and evaluating new tech.
- 'Builders' ranked content types based on what's most important to them while they are building and maintaining cloud services.
- ‘Builders’ ranked system requirements by importance and tutorial content by level of interest.
Survey outcome: Ranked preferences to be used in redesign of IA and content hierarchy for marketing pages and product documentation.
Competitor review to see how direct competitors are organizing their cloud content.
- 3 direct competitors (enterprise cloud solutions)
- 1 indirect competitor (smaller platform, popular with devs)
All competitors have a single cloud landing page that points users towards their documentation, all with the same look and feel. Users can search across all docs from this cloud entry point.
Outcome: Executive stakeholder buy-in. *Required to move forward.
Workshop to align around how we will address user pain points.
- 29 people from 9 different squads
- User journey scope: learning what’s available with IBM Cloud through getting started
- Each squad presented their primary user’s journey
- Everyone took notes and we grouped stickies to identify actions to commit to roadmaps
Our team's roadmap commitments:
Complete tutorials, landing pages for offering subcollections, and provide image guidance for content contributors.
Cross-org roadmap commitments:
Improve product pages and linking to/from all content in ibm.com/cloud.
Applying the RITE method to revise designs quickly based on user feedback.
RITE stands for Rapid Iterative Testing and Evaluation. It's a partnership between Design and Research where usability improvements are made as soon as they are raised by research participants.
- Tasks: finding and navigating a tutorial in the IBM Cloud Docs redesign
- Using mid-fi prototypes
- 6 participants in 3 days
Outcome: Rapid design iteration and validation of tutorial navigation.
Improvements shipped!
New IA
New IBM Cloud Docs homepage
New subcollections landing pages
Tutorials hub
API & SDK hub
Findability benchmarking study to measure impact of our work.
- 2 rounds of testing (before/after)
- 10 participants (1/2 IBM users, 1/2 used direct competitors), same participants in both rounds
- 50-minute moderated sessions
- 9 findability tasks aligning to KPIs
Outcome: Reported positive impact through usability metrics, provided context with qualitative participant feedback, and discovered new usability issues to add to the backlog.
17% increase in task completion rate
Overall time on task decreased by 7 seconds
Average confidence rating increased by .1
Average ease of use rating increased by .5
16.82 point increase to SUS score
Improvements
All API & SDK references located in one place improved discoverability.
Participants were now able to navigate between marketing pages and Docs.
Issues
Need more visible entry points to the FAQ library. *We also monitored analytics and observed a drop in traffic.
Reflection
Project takeaways
Why I'm sharing this work
Required me to lead cross-functional partners from different orgs across the company.
This was a known user and business issue, but no one had been able to bring all parties together to solve it.
What worked well
Socializing our work often (slack blasts, weekly org playbacks, etc.) gained the trust of influential stakeholders leading to wider support.
Sharing templates to communicate the user journey helped collaborators document and communicate their users’ story.
Quantifying the impact of the redesign.
What I would do differently
Compare results from the content type survey with behavioral analytics.
Invest more time looking at competitors (run a UX competitive analysis or benchmarking study).
Thanks for reviewing a case study!
I'm happy to speak in more detail on this or other project work. Please reach out so we can continue the conversation.
Download Resume Connect on LinkedIn