ShareChat is a rapidly-growing social media unicorn headquartered in Bengaluru, India. In all, over 400 million monthly active users rely on ShareChat along with its popular video apps Moj and MX TakaTak to share and consume videos, audio, photos and text in over 15 languages.
For ShareChat, fairness – particularly for low-resource languages and communities with low connectivity – is core to why the company was founded. With the goal of creating an “inclusive community that encourages and empowers each individual to share their unique journey and valuable experiences with confidence,” ShareChat’s social mission extends to its pioneering use of machine learning in areas like multimodal and multilingual abuse detection. This commitment is also attractive to a growing array of advertisers.
To help in the critical task of better monitoring and troubleshooting its models at scale across both structured and unstructured data, ShareChat’s monetization AI team selected Arize as its ML observability partner earlier this year.
At ShareChat, machine learning models touch nearly every part of the business, including:
Delivering consistent results on these use cases across 400 million monthly active users speaking and writing in over 15 languages is challenging. To help, the team at ShareChat has invested in a core ML stack to enable model training and serving and consistent service at scale.
ShareChat faces problems that are common at many companies growing an ML practice. Several pain points stand out.
Troubleshooting ML Is Distracting and Time-Consuming
“Even though we had multiple dashboards in place to catch issues, the process of troubleshooting was exhausting and not the best use of everyone’s time,” recalls Sourav Maitra. “Turnaround was often slowed by needing to go through a lot of tables and then do a bunch of queries. To really get to the bottom of an issue, you also need expertise – often, the person who created the model.”
Building More Robust Model Monitoring In-House Is Costly
Despite having a sizable ML platform team well-equipped for such a task, every initiative comes with tradeoffs. “If we had created ML observability as part of our ML platform team that cut across the feeds, ads, and everything it would likely take at least three or four people along with a product person to understand the problem and do the orchestration,” Sourav Maitra notes. “No matter how good an internal solution we stand up, it’s likely Arize would outrun us because it’s totally focused on ML observability,” he adds.
Lack of Tooling for Monitoring Unstructured Data In Production
Despite a decade of industry investment in deep learning, to date there has not been a great way in the industry to monitor computer vision and NLP models as performance shifts in production. Performance degradation caused by new patterns that models did not encounter in training are difficult to catch with (often costly) human labeling teams.
Analytics Complexity
A platform’s architecture matters, particularly when supporting analytics at enterprise scale; without it, ML teams might be delayed or lose insights at the worst possible time. With over 400 million monthly active users and billions of inferences a month, any potential ML monitoring solution would need to be able to expand query semantics to petabytes of data.
Arize AI processes hundreds of billions of predictions monthly at scale on behalf of clients applying ML across a wide array of use cases. By connecting offline training and validation datasets to online production data in a central inference store, Arize’s ML observability platform helps ML teams streamline model performance management, drift detection, data quality checks, and model validation. Arize also enables users to log models with both structured and unstructured data to the platform for monitoring.
ShareChat first selected Arize in early 2022 after a competitive review of model monitoring platforms. Key factors influencing the decision include Arize’s robust capabilities to surface model and feature drift and performance issues at a cohort level and broad alignment on values and product roadmap.
“Arize enables us to focus on our core rather than building something from scratch,” says Sourav Maitra. “If we’re building a recommendation system, that can be our true focus. That’s one reason why we thought third-party options would be a good fit. There were several other potential partners, but after doing our due diligence it was clear that Arize was the winner – open to feedback, with good alignment on what we need to provide long-term value.”
With robust ML observability from Arize now part of its ML stack, ShareChat’s monetization AI team is able to focus on building new models and accelerating time-to-value for product teams. The organization sees a few key benefits from the Arize platform.
Model Performance Issues Are Now Caught Immediately Instead of Taking 24 Hours or More
Lagged dashboards are now supplanted by real-time alerts. When performance falls below a set baseline, a monitor fires and the team can quickly surface the problematic cohort where drift or performance impact score is highest. Compared to a traditional workflow, “Arize is much faster and you get horizontal and vertical cuts along with some unique insights,” observes Sourav Maitra. Ultimately, that translates into significant time savings for the team and better business results.
Proactive Drift Detection and Troubleshooting
Models are not static. Especially in hyper-growth businesses like ShareChat where data is constantly evolving, accounting for drift is important to ensure models stay relevant. With Arize, ShareChat has robust tracking for prediction, data, and concept drift across model dimensions and values as well as the ability to compare across training, validation, and production environments. “Detecting drift was our p-zero use case with Arize and has been really valuable,” continues Sourav Maitra.
A Design Partnership That Helps ShareChat Hit Its Goals
As an early stage startup, Arize is constantly building and iterating on new features and improvements in close partnership with its design partners. To ShareChat, that means gaining access to an agile team 100% dedicated to building an ML observability platform that fits its vision and needs. “There are areas where we have made feature requests and the Arize team has been really accommodating in terms of altering internal timelines to help us accomplish our goals,” says Sourav Maitra. “Generally, Arize is really open to feedback or even constructive criticism if there is a bug and has been proactive in terms of enabling easier onboarding.”
Robust Model Validation
Arize also helps ShareChat validate models and achieve goals around AI fairness. “We’re planning to create toy models and then push them to Arize to see if all the features we are using are proper or not – and, if not, then we can pull some of the features down,” says Sourav Maitra. “That helps us in building, decisioning, and features.”
Since its founding in 2014, ShareChat has made tremendous headway in its mission to help individuals form substantial connections and stay entertained. The company’s innovative machine learning team, now aided by Arize for ML observability, is well positioned to help the company achieve that goal well into the future.
Download a PDF version of this case study.
Sign up for a free account or book a demo to start your ML observability journey.
