Grocery Store App AI Assistant
Voice Buddy
Voice Buddy
Grocery Store App AI Assistant
Voice Buddy
AmazonNEXT X CodePath Design Challenge
A voice-powered AI assistant that makes the grocery shopping experience feel more confident and independent for non-English speaking individuals.
Grocery Store App AI Assistant
Problem
How might we improve the grocery shopping experience for individuals who don’t speak or understand English in a predominantly English-speaking environment?
Problem
How might we improve the grocery shopping experience for individuals who don’t speak or understand English in a predominantly English-speaking environment?
Outcome
I designed a white label mobile application feature that utilizes voice-to-text AI to improve users confidence and independence in their grocery shopping experience.
Outcome
I designed a white label mobile application feature that utilizes voice-to-text AI to improve users confidence and independence in their grocery shopping experience.
Outcome
I designed a white label mobile application feature that utilizes voice-to-text AI to improve users confidence and independence in their grocery shopping experience.
For this design challenge, my team was tasked with designing or redesigning an app that addresses one or more food access challenges or strengthens cultural connection through food. My team decided to explore ways that we could incorporate AI to improve the grocery shopping experience for people with varying abilities. Together we conducted research and designed a white label mobile application feature for grocery apps.
For this design challenge, my team was tasked with designing or redesigning an app that addresses one or more food access challenges or strengthens cultural connection through food. My team decided to explore ways that we could incorporate AI to improve the grocery shopping experience for people with varying abilities. Together we conducted research and designed a white label mobile application feature for grocery apps.
AmazonNEXT X CodePath Design Challenge
A voice-powered AI assistant that makes the grocery shopping experience feel more confident and independent for non-English speaking individuals.
AmazonNEXT X CodePath Design Challenge
A voice-powered AI assistant that makes the grocery shopping experience feel more confident and independent for non-English speaking individuals.
Role
UX/UI Designer
Tools
Figma, Adobe Illustrator, Notion, Google Forms
Timeline
4 weeks
Team
2 UX researchers
1 UX/UI designer
Role
UX/UI Designer
Tools
Figma, Adobe Illustrator, Notion, Google Forms
Timeline
4 weeks
Team
2 UX researchers
1 UX/UI designer
For this design challenge, my team was tasked with designing or redesigning an app that addresses one or more food access challenges or strengthens cultural connection through food. My team decided to explore ways that we could incorporate AI to improve the grocery shopping experience for people with varying abilities. Together we conducted research and designed a white label mobile application feature for grocery apps.
Week 1
Week 2
Week 3
Week 4
User research
Established a solution
Pivoted, established a new solution and designed it
Tested the product and sent in our submission
User research
Established a solution
Pivoted, established a new solution and designed it
Tested the product and sent in our submission
Let’s get into the timeline…
For the extreme time constraint that we were left with after pivoting our research focus and design idea, this design project went very well and taught me a lot about collaboration in UX Design.
Let’s get into the timeline…
For the extreme time constraint that we were left with after pivoting our research focus and design idea, this design project went very well and taught me a lot about collaboration in UX Design.
Let’s get into the timeline…
For the extreme time constraint that we were left with after pivoting our research focus and design idea, this design project went very well and taught me a lot about collaboration in UX Design.
Week 1
Week 2
Week 3
Week 4
User research
Established a solution
Pivoted, established a new solution and designed it
Tested the product and sent in our submission
User research
We conducted an initial survey with 10 individuals that have either immigrated from a non-English-speaking country or have immediate family that have.
User research
We conducted an initial survey with 10 individuals that have either immigrated from a non-English-speaking country or have immediate family that have.
User research
We conducted an initial survey with 10 individuals that have either immigrated from a non-English-speaking country or have immediate family that have.
We found that:
60% find that maintaining cultural connections through food is important
50% do their grocery shopping in mainstream grocery stores
30% do their grocery shopping in culturally relevant grocery stores
We found that:
60% find that maintaining cultural connections through food is important
50% do their grocery shopping in mainstream grocery stores
30% do their grocery shopping in culturally relevant grocery stores
We found that:
60% find that maintaining cultural connections through food is important
50% do their grocery shopping in mainstream grocery stores
30% do their grocery shopping in culturally relevant grocery stores
Diving deeper into this information, we conducted follow up interviews with 8 individuals that have immigrated here with little-to-no English knowledge.
Diving deeper into this information, we conducted follow up interviews with 8 individuals that have immigrated here with little-to-no English knowledge.
Diving deeper into this information, we conducted follow up interviews with 8 individuals that have immigrated here with little-to-no English knowledge.
We asked:
Would you say there are any language barriers (for whoever immigrated [to the US]) when grocery shopping at mainstream grocery stores?
Did it make it hard to navigate grocery shopping?
We asked:
Would you say there are any language barriers (for whoever immigrated [to the US]) when grocery shopping at mainstream grocery stores?
Did it make it hard to navigate grocery shopping?
We asked:
Would you say there are any language barriers (for whoever immigrated [to the US]) when grocery shopping at mainstream grocery stores?
Did it make it hard to navigate grocery shopping?
We found that:
25% of them had no issues because they learned English early on
The remaining 75% struggled with grocery shopping when they first moved to the US.
33% of this group are still completely reliant on their English-speaking relatives to navigate grocery shopping
33% are somewhat reliant on the English-speaking relatives
33% figured out how to navigate grocery shopping after the language barrier was resolved.
We found that:
25% of them had no issues because they learned English early on
The remaining 75% struggled with grocery shopping when they first moved to the US.
33% of this group are still completely reliant on their English-speaking relatives to navigate grocery shopping
33% are somewhat reliant on the English-speaking relatives
33% figured out how to navigate grocery shopping after the language barrier was resolved.
We found that:
25% of them had no issues because they learned English early on
The remaining 75% struggled with grocery shopping when they first moved to the US.
33% of this group are still completely reliant on their English-speaking relatives to navigate grocery shopping
33% are somewhat reliant on the English-speaking relatives
33% figured out how to navigate grocery shopping after the language barrier was resolved.
This quote from a user really stuck out to me:
“I’ve relied on my children to translate or avoided asking for help altogether due to fear of embarrassment or being misunderstood.”
This quote from a user really stuck out to me:
“I’ve relied on my children to translate or avoided asking for help altogether due to fear of embarrassment or being misunderstood.”
And the problem was clear:
How might we improve the confidence and independence of grocery shopping individuals who don’t speak or understand English in a predominantly English-speaking environment?
And the problem was clear:
How might we improve the confidence and independence of grocery shopping individuals who don’t speak or understand English in a predominantly English-speaking environment?
Voice Buddy: A chat-bot experience
For the user to communicate with to navigate the grocery store.
I went into this idea with the assumption that the user’s phone is set to their language, so the apps they use should be in the set language as well. The chatbot would be automatically set to their phone’s language.
Voice Buddy: A chat-bot experience
For the user to communicate with to navigate the grocery store.
I went into this idea with the assumption that the user’s phone is set to their language, so the apps they use should be in the set language as well. The chatbot would be automatically set to their phone’s language.
Voice Buddy: A chat-bot experience
For the user to communicate with to navigate the grocery store.
I went into this idea with the assumption that the user’s phone is set to their language, so the apps they use should be in the set language as well. The chatbot would be automatically set to their phone’s language.
Ideation
Ideation
Ideation
Voice-powered AI
Meets stakeholder needs, “Your solution might incorporate AI or technology to…”
Research shows that AI/NLP-based voice bots are increasingly needed to support free-form speech, adapt to accents/dialects, and enable multilingual access (Wibawa 2024; Tackling 2024).
Wibawa, Antonius. “Advancements in Natural Language Processing.” Artificial Intelligence Review, vol. 4, no. 2, 2024, pp. 85–97. ScienceDirect, https://www.sciencedirect.com/science/article/pii/S2772503024000598.
Voice-powered AI
Meets stakeholder needs, “Your solution might incorporate AI or technology to…”
Research shows that AI/NLP-based voice bots are increasingly needed to support free-form speech, adapt to accents/dialects, and enable multilingual access (Wibawa 2024; Tackling 2024).
Wibawa, Antonius. “Advancements in Natural Language Processing.” Artificial Intelligence Review, vol. 4, no. 2, 2024, pp. 85–97. ScienceDirect, https://www.sciencedirect.com/science/article/pii/S2772503024000598.
Voice-powered AI
Meets stakeholder needs, “Your solution might incorporate AI or technology to…”
Research shows that AI/NLP-based voice bots are increasingly needed to support free-form speech, adapt to accents/dialects, and enable multilingual access (Wibawa 2024; Tackling 2024).
Wibawa, Antonius. “Advancements in Natural Language Processing.” Artificial Intelligence Review, vol. 4, no. 2, 2024, pp. 85–97. ScienceDirect, https://www.sciencedirect.com/science/article/pii/S2772503024000598.
Viewing the store directory
Giving users the option to view grocery store maps to navigate the store after hearing where it is.
Viewing the store directory
Giving users the option to view grocery store maps to navigate the store after hearing where it is.
Viewing the store directory
Giving users the option to view grocery store maps to navigate the store after hearing where it is.
More menu
For the user to access other features of the product
More menu
For the user to access other features of the product
More menu
For the user to access other features of the product
Accessing the feature
After sketching potential ways to display the feature, the most intuitive way was placing the chat bot in the bottom right corner.
But what if grocery app users don’t need nor do they want that feature there?
Accessing the feature
After sketching potential ways to display the feature, the most intuitive way was placing the chat bot in the bottom right corner.
But what if grocery app users don’t need nor do they want that feature there?
Accessing the feature
After sketching potential ways to display the feature, the most intuitive way was placing the chat bot in the bottom right corner.
But what if grocery app users don’t need nor do they want that feature there?
Initial use
The feature is stored in the “More” screen on the grocery app for the user to choose to enable.
It gives an introduction to what “Voice Buddy” is before the user enables it.
Initial use
The feature is stored in the “More” screen on the grocery app for the user to choose to enable.
It gives an introduction to what “Voice Buddy” is before the user enables it.
Initial use
The feature is stored in the “More” screen on the grocery app for the user to choose to enable.
It gives an introduction to what “Voice Buddy” is before the user enables it.
Introduction
Meeting Voice Buddy, a first-time usage onboarding experience
Introduction
Meeting Voice Buddy, a first-time usage onboarding experience
Introduction
Meeting Voice Buddy, a first-time usage onboarding experience
Returning use
The user can access the feature at the bottom right corner of their screen.
Returning use
The user can access the feature at the bottom right corner of their screen.
Returning use
The user can access the feature at the bottom right corner of their screen.
Branding:
For the UI and branding, I used Kroger Publix, Walmart, and Sam’s Club to demonstrate scalability for the app feature as a white label product*
white label product idea proposed by a mentor from Amazon’s Design team during our brainstorming and feedback session.
Branding:
For the UI and branding, I used Kroger Publix, Walmart, and Sam’s Club to demonstrate scalability for the app feature as a white label product*
white label product idea proposed by a mentor from Amazon’s Design team during our brainstorming and feedback session.
Branding:
For the UI and branding, I used Kroger Publix, Walmart, and Sam’s Club to demonstrate scalability for the app feature as a white label product*
white label product idea proposed by a mentor from Amazon’s Design team during our brainstorming and feedback session.
Usability testing
Due to our time constraints, our usability tests were very quick and limited to one round with 4 individuals.
Usability testing
Due to our time constraints, our usability tests were very quick and limited to one round with 4 individuals.
Usability testing
Due to our time constraints, our usability tests were very quick and limited to one round with 4 individuals.
This is what we heard from users, and would implement if we had more time:
Users would want a voice selection feature
When beginning to use it, a user was initially concerned that the chatbot wouldn’t understand different dialects
This user proposed the idea of dialect detection, which gave another green light for the use of AI to detect differences in language. Hypothetically, if this were a programmed functioning feature, this would be solved.
A user proposed adding highlights to the store directory when the chatbot directs them to an aisle.
A user proposed the idea of an information icon to view the item and if the chatbot is directing them to the right thing.
This is what we heard from users, and would implement if we had more time:
Users would want a voice selection feature
When beginning to use it, a user was initially concerned that the chatbot wouldn’t understand different dialects
This user proposed the idea of dialect detection, which gave another green light for the use of AI to detect differences in language. Hypothetically, if this were a programmed functioning feature, this would be solved.
A user proposed adding highlights to the store directory when the chatbot directs them to an aisle.
A user proposed the idea of an information icon to view the item and if the chatbot is directing them to the right thing.
This is what we heard from users, and would implement if we had more time:
Users would want a voice selection feature
When beginning to use it, a user was initially concerned that the chatbot wouldn’t understand different dialects
This user proposed the idea of dialect detection, which gave another green light for the use of AI to detect differences in language. Hypothetically, if this were a programmed functioning feature, this would be solved.
A user proposed adding highlights to the store directory when the chatbot directs them to an aisle.
A user proposed the idea of an information icon to view the item and if the chatbot is directing them to the right thing.
Final solution
We submitted our final design solution to the Spring 2025 AmazonNEXT x CodePath Design Challenge!
Final solution
We submitted our final design solution to the Spring 2025 AmazonNEXT x CodePath Design Challenge!
Our mentors from Amazon’s Design team were impressed with our idea because of the creativity in the use of AI and its scalability.
The users that tested the product found it helpful, with an average rating of 4.25 out of 5
All of the users felt more independent and confident using the tool.
Our mentors from Amazon’s Design team were impressed with our idea because of the creativity in the use of AI and its scalability.
The users that tested the product found it helpful, with an average rating of 4.25 out of 5
All of the users felt more independent and confident using the tool.
Our mentors from Amazon’s Design team were impressed with our idea because of the creativity in the use of AI and its scalability.
The users that tested the product found it helpful, with an average rating of 4.25 out of 5
All of the users felt more independent and confident using the tool.
Final solution
We submitted our final design solution to the Spring 2025 AmazonNEXT x CodePath Design Challenge!
My honest reflection
Through this project, I
learned that testing wireframes ensures that the product is usable before the final product is released.
learned that for future projects, I would conduct more intentional usability tests and probably video or in person interviews for better feedback and research.
was introduced to designing with Adobe Illustrator
learned the importance of documenting process work
collaborated with a team of marketing and engineering minds, teaching me to understand different teammate perspectives, something that is crucial to work on any design team
got to experience short timelines after pivoting design solutions with 10 days left to spare
got to build a really cool product with really cool people
felt that feeling when you know you’ve made a significant impact on someone’s life with your work
My honest reflection
Through this project, I
learned that testing wireframes ensures that the product is usable before the final product is released.
learned that for future projects, I would conduct more intentional usability tests and probably video or in person interviews for better feedback and research.
was introduced to designing with Adobe Illustrator
learned the importance of documenting process work
collaborated with a team of marketing and engineering minds, teaching me to understand different teammate perspectives, something that is crucial to work on any design team
got to experience short timelines after pivoting design solutions with 10 days left to spare
got to build a really cool product with really cool people
felt that feeling when you know you’ve made a significant impact on someone’s life with your work
This project was nothing short of amazing, and I’m grateful to God for another opportunity and learning experience <3
Thanks for checking it out,
This project was nothing short of amazing, and I’m grateful to God for another opportunity and learning experience <3
Thanks for checking it out,
This project was nothing short of amazing, and I’m grateful to God for another opportunity and learning experience <3
Thanks for checking it out,
My honest reflection
Through this project, I
learned that testing wireframes ensures that the product is usable before the final product is released.
learned that for future projects, I would conduct more intentional usability tests and probably video or in person interviews for better feedback and research.
was introduced to designing with Adobe Illustrator
learned the importance of documenting process work
collaborated with a team of marketing and engineering minds, teaching me to understand different teammate perspectives, something that is crucial to work on any design team
got to experience short timelines after pivoting design solutions with 10 days left to spare
got to build a really cool product with really cool people
felt that feeling when you know you’ve made a significant impact on someone’s life with your work
