Project Scope

Client: The Portland Nursery

My Role: UX Researcher, Designer, Strategist

Team: Drexel University - Designing with Data - Team of Four Graduate Students

Timeframe: 4 weeks

Methods: A/B testing, Qualitative and quantitative research, data analysis

Tools: Figma, Microsoft Teams

‍

The Challenge

The Portland Nursery is a popular garden store with two locations in Portland, OR. The specialized in-person shopping experience makes the nursery unique. Plant enthusiasts visiting the store can speak to a garden associate to get personal advice. At the start of the COVID-19 pandemic, the nursery began offering curbside pickup as a way to mitigate the loss of in-person shopping. In the online shopping experience, the average plant buyer no longer has access to a garden associate to help them confidently make purchases.

The Portland Nursery website uses traditional, faceted search to help users find plants, but these require explicit knowledge of what the shopper is looking for. Important information such as USDA zone and lighting requirements might be unknown for users. One alternative is a guided search, which has the power to replicate the personalized feel of in-person shopping.

‍

Team

Sumita Mukund, Drexel University

Alex Wang, Drexel University

Jenny Weyant, Drexel University

Meredith Cullen, Drexel University

‍

My Role

As the researcher in a team of three, I led the exploration of this topic and designed a research plan to guide our direction.

‍

Research Goals

Design an A/B test proposal that aims to improve the online search experience for Portland Nursery site visitors.

‍

‍

‍

‍

Process

Hypothesis

For visitors of the Portland Nursery website looking to purchase plants online during the 2020 pandemic, implementing a guided search on category pages will improve transactions on the site. We believe guided search better replicates the in-person experience and does not require external knowledge from users. We will know this is true when we observe an improved conversion rate.

‍

Test Cells
‍

Control (A): Current state. Users searching for products on the Portland Nursery website will see the faceted search with no change.
‍

Test Group (B): Variant state. Users searching for products on the Portland Nursery website will see a guided search.

‍

Discussion

Our A/B test design takes a global and exploratory approach. We changed the search to find a different way to meet the product finding needs of users. The test proposal allows us to see how a guided search method impacts user behavior and understand search patterns. We want to know if a guided search helps users find what they are looking for, complete their task, and encourage product discovery.

‍

Potential Outcomes

‍

Among the potential outcomes for our A/B Test is a success scenario in which we learn a guided search significantly increases our conversion and engagement rates. In this scenario, our team would continue to refine the guided search design through further A/B testing. A suitable alternative would be to ramp up to 100% with the successful design and begin testing other variants against it. 

‍

If our hypothesis fails to be proven and the control experience (faceted search) performs better than the variant, our team would identify and fix any existing test design flaws. If there are no flaws, we would cut the variant and move on to other variants or abandon the hypothesis altogether.

‍

If the test concludes with flat results that indicate an inconclusive test, we would conduct some additional analysis to determine whether any segments performed better than others, discuss the findings, and surface opportunities to iterate or refine the concept for further testing.

‍

Limitations

‍

Although the guided search has potential to change the way people shop online, there are some limitations. Our design is focused on novice users with limited knowledge who will benefit most from the guided search recommendations. Further research will have to be conducted to understand the search needs of expert users.

‍

It is also important to note that back-end support for this new design will be required. In an inventory system with a faceted search, categories and tags organize the inventory. In a guided search, more thought must be placed into organization for the recommendation algorithm.

Considerations

‍

When conducting an A/B test, it is important to be aware of the things that could go awry. We may find that no Portland Nursery users searched for products, resulting in no data. A low number of novice visitors and higher ratio of plant experts who do little if any searching could result in a low yield as well. A poorly designed guided search could frustrate users, causing them to abandon the website, resulting in lost customers and business income.awry

‍

Financial Harm: We want users to discover products they didn’t know to look for, but if our guided search design is too persuasive, users could be harmed financially if they buy items they didn’t want or need just because it was suggested. If our guided search algorithm is poorly designed, users could also end up with products that aren’t suited for them (e.g., the wrong soil for their USDA zone).

‍

Social Harm: The recommendation algorithm would learn information about users to suggest products. It’s possible users would not want their environmental information or habits exposed. We believe our test mitigates the risks and has the potential to help Portland Nursery users.

‍

Conclusion

‍

The faceted search paradigm is what most consumers are familiar with: the user finds the product they want by adjusting a series of search filters. A guided search on the other hand is like a step-by-step tutorial, asking the user a series of questions and at the end presenting the relevant results. A local plant store like the Portland Nursery is a good candidate to A/B test the efficacy of guided search because of their large inventory and strong customer base. The results of the experiment will determine the design of the search experience. While speculative, this project sharpened my ability to design for ambiguous experiences. These skills translate to my work with clients in industries ranging from automotive to fine jewelry.

‍