Upcoming research at Stack Overflow

Last month, we shared our upcoming priorities from a product roadmap point of view. This included initiatives like better onboarding, Staging Ground (our experiment with a shielded experience for new users), Discussions, and technical investments. Today I want to share the research the User Experience team, which I lead, will be focused on over the next quarter and how you can help. At Stack Overflow, our researchers are focused on strategic initiatives and big, complex questions. They typically run ahead of the product roadmap by several months or even multiple quarters. Please keep that in mind as you navigate what we are looking to learn this quarter.

Last year, one of our researchers set out to create a more comprehensive framework of user roles and learning pathways on Stack Overflow. This research was done through cluster analysis of site behavior data, a survey, a Meta Stack Exchange post, workshops, and more. From this work, we built hypotheses around five motivations, including what users need or want to learn and what elements of our site might appeal to them. These motivation groups were used in future workshops to write Jobs-to-be-Done to identify why users choose Stack Overflow and what might frustrate them. This has laid the groundwork for a lot of our upcoming research. We are excited to dig into these hypotheses and learn even more about how we can help community members.

Finding a solution to your problem takes time and effort. Over the last year, we’ve honed in on the cycle of how an engineer or technologist unblocks themselves. You may already understand from your experience, but it is not a quick process. As we continue to explore this journey of finding a solution to your problem, we want to understand the nuances of the process and which pieces are most important and most difficult for our users. We then hope to explore whether we can help alleviate the pain points we uncover.

Jobs-to-be-Done being researched: “Find a solution when I am at work/school so that I can get unstuck without spending a large amount of time on my problem.”

We’ve recently announced we are bringing back Staging Ground! Yay! Staging Ground attempts to solve a variety of problems. We hope to teach new askers community norms while unblocking them more quickly through asking more successful questions and ensuring that questions can be reused for future users with the same problem. That’s a lot! Especially when you think about the fact that the number of users who are “experts” is small compared to the size of the audience looking to get unblocked. So how can we scale such a useful tool? We will be testing ideas with our users over the coming months to see what’s useful and what isn’t.

Thank you to the folks who have already provided our team with great ideas on how we can improve the feature as it moves out of beta. Beta reviewers have also shared feedback in their private space on Stack Overflow for Teams, and we are excited to prioritize those issues and ideate on new solutions, particularly to address the biggest bottlenecks and burnout points in partnership with the community. You may see some of these concepts in research sessions. We’ll also be engaging with community members on Meta and using other avenues such as surveys.

Jobs-to-be-Done being researched: “Promote healthy community behavior proactively so that I can encourage others to engage more frequently and safely without bias”; “Find people that need help that aligns with the help I can provide when there’s an opportunity to help and I have time so that I can feel the enjoyment of teaching others without wasting time searching for opportunities to apply my knowledge and help others”

To round out our user journey, we want to learn more about the answering experience and how we can improve it. We believe that speeding up the answering process will create value across the community. We aim to learn about how our users find questions to answer, how they answer, what they need to answer (e.g., sources, code editors, translator devices), and if there are opportunities to make this process easier. This means figuring out what makes answering difficult or lowers answer quality on the site, while also recognizing situations where friction may be helpful and should be kept. We are looking to document a clear set of user problems related to answer creation on Stack Overflow and then work alongside the community to solve these problems while remaining open to both AI/ML and non-AI/ML opportunities.

We are excited to use this as an opportunity to test the concept of participatory design. Participatory design (or co-design) is a method where users contribute to the design process, allowing them to provide feedback, suggest ideas, and participate in decision making. Our goal is to build a solution that is not only amicable but a feature our community will celebrate! We hope to have our first session within the next six to eight weeks.

Jobs-to-be-Done being researched: “Share my insights and knowledge in the tech community when I am looking to showcase my knowledge so that I can feel acknowledged for my contributions without spending too much time”

In the interest of gathering more data, I am happy to say that we are now using a research tool, Ballpark, that will give the ability to conduct research to a wider number of Stack Overflow employees outside the User Experience team. To encourage learning, we have set a challenge for our User Experience professionals, Product Managers, and Product Marketing Managers to run two to five studies this quarter using the new tool. This tool specializes in “unmoderated” research which is a form of qualitative research where users complete predetermined activities while answering questions out loud. This type of research is good for more tactical research like identifying the best place in a UI to place a button. While we don’t know all of the research that will happen this quarter using this new tool, we are excited about the learning potential.

Some ideas that are already being discussed:

• Feedback on the Stack Overflow for Teams homepage
• Updated styles and resulting usability for sidebar widgets
• Accessibility feature prioritization to help us identify what would be more impactful for the community

The best way to get involved is to sign up for research invitations in your account settings! We need community members like you to participate in our research to engage in deep conversations around key areas of interest. So please, if you want to help us learn and even help create alongside us (e.g., participatory design sessions)—sign up!

As User Experience professionals and Stack Overflow employees, we are deeply committed to learning as much as we can to better support developers and community members. We are incredibly grateful for your time and passion and, as always, we look forward to working and learning together.