How Flipkart Leverages Generative AI for 600 Million Users

Catering customer support for 600 million users is a feat in itself. Between sessions at this year’s Arize:Observe, Flipkart’s Anusua (Anu) Trivedi talked to Aparna Dhinakaran about the company’s challenges and generative AI use cases.

Among other things, Anu talked about how tricky measuring success can be. Trivedi explains that “you have to define it not only for your models but also for your products. There are LLM metrics, but also product metrics. How do you combine the two to see where things are failing? That’s where Arize has been a fabulous partner for us to figure out and create that traceability.”

With Arize, Flipkart gained the capability to stitch it together and create the right storyline to improve their product.

Watch It

Q+A with Anu Trivedi, Head of Applied AI at Flipkart

Aparna Dhinakaran: Hi everyone, I have a very special guest today with us. Anu, do you mind going ahead and introducing yourself?

Anu Trivedi: Yes, thank you for having me today, Aparna. I am Anusua Trivedi, I go by Anu, and I am the U. S. Center Head for Flipkart. We cater to the volumetric population of India, the Tier 2, Tier 3, cities mostly.

It has been a fabulous journey, to embark upon using generative AI for most of our use cases, and paving the path for conversational commerce in our company.

Aparna Dhinakaran: I have to ask because I feel like most people don’t realize first of all, the scale and complexity of what you do, but can you explain what are your gen AI use cases?

Anu Trivedi: Yeah. First and foremost, Flipkart is a huge platform. We have close to about 600 million users registered on the platform. As you can imagine, catering customer support for such a big user scale is in itself a feat, right? So unless we are investing in the right technologies to create opportunities for self-serve for customers, it’s very hard for us to scale the business.

And that is where generative AI is just not an experiment, but a dire need for us to make the right necessary impacts in our business.

Aparna Dhinakaran: Can you share for folks who are on their journey of getting generative AI in–what are some things that have gone well? What do you guys think of as challenges? Any best practices?

Anu Trivedi: Yes, of course. I mean, it has been really exciting to see the hype from OpenAI to figuring out the actual use cases.

Failing in some cases. For example, we tried out customer support on a use case for highly anxious users. We said hey, that is not the place to basically, you know, invest in. Highly anxious users still want that human component in their conversation. So we gave it a twist. We said, hey, if we cannot help our customers, can we help our agents? So we created the same solution.

We created AgentServe and we started empowering our agents so that instead of searching SOPs, they can just get assistance for generative AI agents to help them answer the question more effectively to our volumetric, anxious consumers, right? And in that way, by reducing some of the call timing, we are making a huge ROI impact in our incremental GMV. So we are still making an impact, but just a little bit of twist in the product solution. So it’s all about understanding a customer base and finding the right route.

Aparna Dhinakaran: External versus bringing that solution internal. Got it. I know we talk a lot about evaluations and observability, but, what are the things you’ve learned on that journey?

Anu Trivedi:  So it has been a learning process. Metrics are very tricky, right? You have to define it not only for your models, but also for your products.

So if I am to break it up into two parts, there’s of course LLM metrics or a data science metrics side of it. And there’s a product metrics side of it. How do you connect the two to find where things are failing?

And that is where I think Arize has been a fabulous partner for us to figure out and create that traceability. We had some offline metrics to connect the data science metrics and the product metrics, but Arize gave us the capability to stitch it together and create the right storyline to basically bring the right knobs to twist on the modeling side or on the policy side to improve the product.