Beyond Chat: Why User Experience Design is key to Unlocking AI's Full Potential
Does AI need a UI?
A few weeks ago I was listening to a podcast discussing the latest developments of DALL-E and ChatGPT and noted how the current state-of-the-art interface for generative AIs is text-based chat. This led me to ponder whether chat is the best UI for AI. Perhaps given the infancy of generative AI today, chat is the most flexible means to experiment and work with these AIs. However, In my years of experience working with and mentoring designers, I believe the standard chat interface leaves much to be desired from a user experience perspective. User experience design for AI apps will require the same focus on design principles in use today to create the most intuitive and delightful AI experiences for tomorrow.
One of the core problems I see with the chat interface is the “blank page” effect. Without visual cues or guardrails, users struggle to come up with the right prompts to get the AI to do what they want. I often see Instagram and TikTok posts where users share the exact wording needed to get helpful results from ChatGPT. This points to a usability gap, clever prompting shouldn’t be required to use AI effectively.
Legendary UX expert Jakob Nielsen recently outlined two core issues with chat interfaces for AI:
Most people are better at reading than writing. Crafting text prompts requires skills that not all users possess
Generative AI prompts are specifications of outcomes rather than steps. To iterate, users must start from scratch, tweaking and guessing at words that will get the AI to produce the desired results
I agree with Nielsen when he speculates that “successful AI user interfaces will be hybrid and combine elements of intent-based outcome specification and some aspects of the graphical user interface”. Apps with deliberate visual language and interactions that blend in will ultimately provide more satisfying and productive experiences for getting real work done with AI.
For example, look at Adobe’s integration of its Firefly AI-based image generation into Photoshop. Rather than just a chatbot sidebar, the AI features become a tool accessible within the familiar graphical UI. Users make precise selections before triggering the AI. This is because pointing, even with a mouse, will always be much more efficient than typing a description of what you want.
Adobe Photoshop with integrated Adobe Firefly capabilities
The results that Firefly generates depend on what is provided in the chat box. It can automatically fill a section without prompting or if a prompt is given can specifically generate an image to replace the selection. This simple UX behavior cannot be done with any of the current text-based image generation bots such as Midjourney or DALL-E.
Unprompted generation - selecting the road under the bicycle to erase the skid marks without any prompts
Prompted generation - selecting the center of the road and generating yellow center dividing marks
Under the covers, Photoshop is likely feeding the selected portions of the image to Firefly and using canned prompts to tell Firefly how to perform these behaviors consistently, based on user interactions with the toolbar. These behaviors are the results of Photoshop’s PMs and designers understanding the generative capabilities of Firefly and creating a simple but powerful tool into Photoshop to blend in AI capabilities. This approach balances both human and computer strength. Human-based visual design excels at clarifying complexity. Meanwhile, AI contributes content learned from data at scale.
These behaviors are the results of Photoshop’s PMs and designers understanding the generative capabilities of Firefly and creating a simple but powerful tool into Photoshop to blend in AI capabilities. This approach balances both human and computer strengths. Human-based visual design excels at clarifying complexity. Meanwhile, AI contributes content learned from data at scale.
Chat is a great starting point to explore the capabilities of AI, but it will not be the ultimate UI. Truly human-centered AI products will still require talented PMs and designers who understand the needs of the user to combine the magic of algorithms with the clarity of intuitive visual interfaces.