A conversation with Josh Clark (part 2)

Read part 1 of the interview here.

Tangible: Ideating concepts around these new technologies and executing projects that leverage them are two completely different ball games. What are the biggest challenges you see in order to design and deliver products that make these technologies work in the real world?

Josh Clark: Technologists are often smitten by their technologies, and we often want to use them when they’re still fragile, or in ways that are outsized to their actual capability, or in ways that don’t really fit the lives of our users.

As consumers of these technologies, we feel the mismatch of expectation and capability in new technologies all the time. Siri and Alexa and Google Assistant, for example, all set the highest possible expectation, and they routinely fail to meet it: “ask me anything.” As remarkably sophisticated as those services are, they can’t meet this impossible expectation that they set. Sometimes that’s a problem with the underlying technology, but more often I think it’s a problem of design, of setting an expectation that the technology is more capable than it actually is.

Our work is to channel behavior and set expectations that match a system’s capability. That’s always the work of a designer, but it becomes even more urgent the more sophisticated and nuanced the system becomes. In the realm of machine learning and algorithmic interfaces, for example, there’s a lot of magical thinking on the part of both technologists and consumers about what these services do. And as a result, they often fail us.

A big focus of the workshop will be exploring practical perspective and real-world techniques for setting user expectations of a service, within the interface itself. And from there, how might we channel behavior in ways that let these services respond in ways that are actually useful and meet those expectations?

I mentioned before that it took us a while with mobile to figure out how the technology wanted to be used. This is always a struggle with new technologies, new kinds of expression. It takes time to let the new model emerge and breathe, to escape the routine patterns of the technology that came before. But there are methods that designers can bring to an emerging technology to get needed perspective, to understand the opportunities and uses implicit in that technology’s strengths and weaknesses. What are its limits? How does a technology best bend to our lives, instead of the reverse?

In the workshop, we’ll look at the strengths and weaknesses of emerging technologies as a way to understand each of them as a material. Like physical materials, these new design materials have strengths and weaknesses. Like woodworkers who work with the grain of their material, we need to understand the “grain” of each of our own new digital materials.

The best way to learn the grain of a new material—how it wants to be used—is to actually work with it. As you mentioned, there’s a big difference between ideas for product applications and actual products. In the workshop, we’ll explore how to incorporate experimentation, prototypes, and play into our everyday practice. How can we test our ideas, learn from our experiments, and fashion those into products? Too often organizations go directly from idea to product—the Kickstarter phenomenon—without actually researching wither that idea will make anyone’s life better.

One of the fun portions of the workshop, for example, will be spending some hands-on time with some of the new machine-learning services that are available—services that designers can use right now, today—to understand the texture of machine learning as a design material. What is it good and it, and where does it fall down? How do we understand its role in both a product and in an interface?

Tangible: Speaking of machine learning and AI, we designers often think about automating tasks or, even better, anticipating the user’s needs and delighting the user. What are the opportunities and the risks raised by automating decisions on behalf of the user?

Josh: I’m not sure that “automating decisions” should be the goal, although I think you’re right that’s how these systems are often applied. Instead, I think the partnership between human and machine is most powerful when the machine amplifies human judgment and agency instead of replacing it.

As we explore the grain of machine learning, one of the things that we’ll uncover is that machine learning is much better at providing signals than providing conclusions. The machines don’t see the world in black and white, they don’t see facts. Instead, machine learning is all probability, the statistical likelihood of different outcomes being true. In that sense, they make much better advisors than they do decision makers.

The world is a messy place, and like humans, the machines come to sometimes unpleasant conclusions. In fact, these systems are full of human bias and error—because they’re built on human data, crafted around human decisions of what to optimize for. They have human values deeply embedded in them. That’s not necessarily a bad thing, but what is bad is that we pretend it’s not true, that they’re somehow more objective or reliable than people. When we let the machines chug away on their own, the robots often come to weird, alarming, or downright dangerous conclusions. Rather than letting them make decisions on their own, it’s far more useful to let them surface the patterns and signals they see to inform human decisions.

In the workshop, we’ll look at new design patterns for algorithmic interfaces to highlight those signals, to engage attention and judgment at critical moments, instead of burying those signals in pursuit of presenting “one true answer.” Together we’ll find alternatives that would likely make familiar interfaces like Google’s search results, Facebook’s news feed, or Instagram’s photo stream much more insightful.

Tangible: These new technologies are changing the paradigms not only from the user’s point of view but from the business’ and organization’s ones too. In your opinion, why and how should companies introduce, adapt and embrace these technologies? What are their main struggles and how can we designers help them choose their path?

Josh: When you look at emerging technologies, designers are often slow to be involved. With products featuring machine learning, for example, the key players have typically been data scientists and algorithm engineers. And don’t get me wrong, those groups have shown us what’s possible. But you can feel the designer’s influence missing here.

The biggest misfires around new technologies emerge from an overconfidence in the technology itself, a poorly understood definition of the problem to be solved, and a mismatch of technology to user context. And all of these things just make the new product feel broken, and that disappointment leads to a tendency to avoid that risk in the future—by both the business and the customer.

But those are exactly the problems that designers are trained to handle. Here are some of the questions that design is uniquely positioned to help answer when it comes to the application of new technologies:

• What is the problem to be solved?
• What are the moments in the customer journey where this technology can offer a useful intervention?
• What are the activities, environments, interactions, objects, and people who are related to those moments, and how might the technology work with them?
• How can this technology create insight or amplify human agency, rather than replace it?
• How do we set proper expectations for what the technology can do and how it works?
• How do we channel behavior in a way that matches those system capabilities?
• How do we design for uncertainty and failure in the new system?
• How might this technology interfere with the customer’s life, as well as help it?
• What is the worst thing that could happen, and how do we design to make sure that doesn’t happen?

Those are the framing issues we’ll dig into in the workshop. When you step back and look at those questions, that’s all about design and research and user experience. Those are human questions, not technology questions. And whether you’re talking about new or established technology, those human questions are what design and designers are all about.

—

We really wish to thank you Josh Clark for this Q&A session and for touching on these themes.

If you are as eager as we are to dig deeper into the future of design and technology and spend a full day with Josh to hone your skills and fuel your curiosity, don't miss the chance to register to his workshop in Bologna, next October 9th.