Usability Testing | User Research | Experimentation
Role: launch plan, product marketing materials, A/B testing, user interviews, user engagement analysis
As a Design Research Manager on the digital product team for Knotel, a real estate startup offering flexible workspaces, I worked with a team of product managers, product designers, and engineers to research, design, launch and test a new product specifically for brokers, brokers.knotel.com. In an effort to increase corporate real estate deals through partnerships with brokerages, our team was tasked with the development of a site showcasing Knotel’s available properties tailored specifically for brokers. My role involved conducting user research, validating the insights, testing user acceptance, informing design iteration, and tracking marketing and product metrics for the site.
Project Context
At the start of the project, two versions of a listings platform of Knotel’s available spaces existed–one for internal Knotel Account Executives to share spaces with potential customers who reached out and one for Knotel’s potential customers, often the real estate teams of companies. Up until this point, the party not catered to by these two interfacing sites were real estate brokers, who are critical to the sales process as they are typically the first resource prospective renters turn to when considering renting commercial real estate space. To address this critical gap, our team embarked on a two month long project to understand the workflow of real estate brokers and reconfigure the listings platform to support their needs and facilitate sales of Knotel’s spaces.
For internal Knotel Account Executives
For Knotel Customers
User Research
User Interviews
To understand the needs of corporate real estate brokers, we conducted 10 interviews including 5 in-person sessions with an ethnography component. Interviews were conducted with brokers from the New York City market in-person and the West Coast and Europe markets virtually and included a split between brokers from large brokerages and smaller brokerages. Through interviewing the brokers, observing their workflow, and conducting a journey mapping exercise, we defined the broker journey to better understand at which points Knotel’s sales process could fit into their workflow. We then identified pain points in their journey to ensure our product features met their distinct needs.
User Journey
After synthesizing the user interviews and observations, our team defined a broker’s sales journey and identified the interaction points between brokers and customers, and between Knotel and brokers.
How can we support the workflow of corporate real estate brokers in order to increase customer engagement and Knotel sales?
Product Design - Translating the Insights
User Insights
The team translated the insights into key features of the product design enabling brokers to easily download relevant assets and find exclusive properties.
The Value of Pipeline - Brokers see the most critical value as access to properties off-market and in our pipeline (those we might acquire but even more so those that might come back into the market).
The Security of Assets - Brokers ask for flyers because they want the assets in them in order to make their own surveys for their customers. Their biggest struggle with these surveys is finding the assets (photos, information, layouts) that bring the spaces to life.
All the Data We Give for Free - Brokers don’t want to tell us who their clients are, but are happy to provide us their requirements in the form of saved searches.
The Importance of Branding - Brokers seemed reluctant to send our listings to their own customers because it’s not obvious who it is coming from (e.g. they may not even know Knotel) and they want the option to show our listings agnostic of our branding as some believe it may hurt the perception of the listing.
Value Proposition
After synthesizing the insights, a clear value proposition for the product was defined as a self-service site that provides exclusive access to Knotel’s off-market, pipeline, and active listings to enable easy download of assets and scheduling of tours.
Exclusive Visibility
Visibility into exclusive, off-market, and pipeline properties not listed anywhere else
Downloadable Assets
Quickly access and download assets including floor plans, space images, and flyers
Notifications
Easily access relevant listings and receive customized notifications as new listings are added or come back on market
Experimentation & Testing
Usability Testing
Before releasing a formal blast, I led a series of usability tests with 5-6 brokers (17 total) in each of three markets including New York, West Coast (SF & LA), and Europe (London & Dublin) as well as 5 internal stakeholders.
Rainbow spreadsheet used for tracking usability, success rate, time to completion, and satisfaction
Initial Walk-through/Familiarization - Taking an initial look at the listings site, what are your initial thoughts on this page? What would be the first thing you think you would be inclined to do/look at? Why?
Task 1: Filter for your most typical type of site (market, size, etc.)
Complete When:
Selects the market they typically work in
Filters by size of space
Selects a move-in date range
Selects a status (pipeline vs. vacant)
On a scale of 1-7, how would you rate the effectiveness of this feature? Why?
Noting major findings and tracking frequently occurring usability issues and potential solutions
Task 2: Download assets including photos and a floor plan of the desired space
Complete When:
Downloads photos of the desired space
Downloads a floor plan of the desired space
On a scale of 1-7, how would you rate the effectiveness of this feature? Why?
Task 3: Request more information about a specific site
Complete When:
Navigates to the ‘Space Detail Page’ for their intended site
Requests more information about that site
Submits a request for information
On a scale of 1-7, how would you rate the effectiveness of this feature? Why?
User Feedback
Complete walk-through of the broker portal
After talking to users, we had 3 main takeaways:
1. Localizing Sites - To make the portal functional for all markets in the U.S. and in Europe, tailoring needs to occur. Information such as transit options as well as measuring units should be curated for local culture.
2. Pricing Information - Brokers continuously cited a need to see a range of pricing information upfront even if this information required a log-in to access.
3. Relationship-Driven Industry - Brokers referenced the industry as very relationship-driven, desiring a way to receive email updates with new listings without having to save a search.
Additional Features - Developing Saved Searches
A key need our team saw to bring value to Knotel was through data collection from having brokers save searches - preset filters around their prospects’ requirements. To meet this need, we developed interactions to prompt users to save their searches and then receive emails whenever a new listing was added that met their needs.
With this functionality, the site could be used to establish an persistent, yet curated relationship with the broker, tailoring communications to their needs while keeping Knotel top of mind.
Product Marketing & Impact
Launch Plan
To launch the new broker portal, we put together a plan for staged blasts to brokers in order to be able to learn and iterate. The goal was to gauge initial reactions to the portal and pilot with select users to ensure the final largest blast was successful.
To measure the success of the launch, we identified key metrics to track such as number of visitors, click-through rate from emails with the new portal as opposed to typical broker blasts, and number of assets downloaded.
A/B Testing the Blast
In preparing for the final launch, I developed a blast to be sent out to brokers and two subject lines to see which would gain more traction:
Subject A: Find Flex Space Faster in 2020
Subject B: On-Stop Shop for Flex Space
Target Users: 500 brokers from 112 global small brokerages who have never done business with Knotel as a low-risk test case.
Goal: Test which email subject line has the highest open and click-through rates; Evaluate and monitor usage and core user flows for bugs and issues on FullStory
Duration: 2-4 days
Outcome: Subject line B (One-Stop Shop for Flex Space) had a slightly higher open rate at 8% compared to 5% for subject line A; Click-through rates for both subject lines were very low at <1% for both groups
Since the initial test blast, our team blasted the new portal to ~4,000 in New York City, the market that was top priority to generate business for Knotel in Q1 2020. After blasting the product, the site received ~1,550 new users in February 2020.
Tracking Metrics
A number of key metrics were tracked in order to gauge product success and reveal insights into user behavior. The team tracked metrics and usage using Heap Analytics. After a month of being active, the site was receiving 213 page views daily.
An important indicator that the team was tracking was the sales funnel from viewing a page to submitting a request for information. The funnel proved the value of assets with ~11% of visitors downloading assets directly from the home page and ~33% of visitors downloading assets from the space details page.