Xander Song is Arize AI’s new Developer Advocate. Before joining Arize, Song worked as a machine learning engineer at Test.ai. He is based in Oakland, California.
Can you introduce yourself and share your career background?
I have an interdisciplinary background and took a winding path to my current role at Arize. As a philosophy and mathematics undergraduate, I was captivated by the ambition of both disciplines to uncover fundamental truth through rational argument. I also gravitated toward teaching and mentorship; my first role out of college was teaching English in South Korea, and I’ve worked and volunteered extensively as a mentor in the context of technical education, intercultural exchange, and diversity and inclusion.
On the technology front, I’m deeply interested in early-stage companies at the intersection of AI and ops. Most recently, I was working as a machine learning engineer (MLE) at test.ai, a startup using AI to change the way people test software. The reason I’m drawn to the space of developer tools and ops is because it’s exciting to shape the processes, techniques, and best practices that define a developer’s workflow. Not only can you positively impact individual developers and companies, but you can change the trajectory of entire industries and communities for the better.
Why did you first decide to get into machine learning?
As an undergraduate, I had a romantic notion of research as a means to explore uncharted intellectual terrain. One summer, I basically lived in my university’s math building and, as part of a research group, proved and published a novel result in matrix theory. The experience of overcoming a deep technical challenge was very gratifying, but I found it difficult to motivate and explain the significance of my findings, even to other mathematicians. Machine learning offered me the opportunity to explore mathematics through a highly applied and practical lens. I was motivated by the desire to understand on a deep and comprehensive level how certain algorithms and models worked under the hood. For example, I wrote my senior thesis on spectral graph theory and its application to semi-supervised learning.
I soon learned that innovation in modern AI and ML is happening not only inside the model, but in the systems and best practices that surround and support the model. In one formative experience, I was tasked with tuning hyperparameters for a moderately complex model as part of a research project. I accomplished this by manually running experiments and keeping track of the results in a spreadsheet. This was a time-consuming and painful process because each experiment had to be scheduled on specialized shared hardware and took hours to run. Fast forward a few years and there are a plethora of AutoML tools that automate this process and eliminate this pain. This was an early lesson in the importance of having the right tool for the job that shaped my interest in machine learning engineering and MLOps.
How would you describe your role and responsibilities at Arize?
My role as a Developer Advocate is to be an intermediary between the company and the larger community of machine learning practitioners. My responsibilities will include both evangelism (for example, leading workshops and creating content to increase awareness and understanding of ML observability in general and of Arize in particular) and advocacy (for example, engaging directly with our users and leveraging my own perspective as an MLE to improve our user experience).
Why is ML observability so important?
Machine learning models can “fail” in production in subtle ways that are difficult to detect and understand. ML observability means knowing immediately when your production models are behaving unexpectedly (monitoring) and having the ability to quickly identify the root cause of the issue. Without monitoring, a poorly performing model can hurt your users, customers or your bottom line before you even know. Without root cause analysis, even if you know something is wrong, you can’t fix the problem in a timely manner.
If you deal with machine learning in production, you probably know this pain on a visceral level. One of the most harrowing experiences of my career to date came when my previous startup was in the process of raising our next round of funding and was at risk of losing a crucial customer due to a poorly performing production model. I was tasked with identifying and fixing the issue, and as you might imagine, both the stakes and my blood pressure were high. While I eventually diagnosed and resolved the issue, it took longer than expected and we might have lost the account had my customer-facing counterparts not carefully managed customer expectations and stalled for time. In cases like this, ML observability not only improves quality of life for MLEs and data scientists; it can mean the difference between the success or failure of your enterprise.
Any insights from working as an ML engineer in prior roles that give you a unique perspective at Arize? I’d imagine there are some unique use cases and ways model performance can degrade in the app testing space that might inform your point of view?
I came to appreciate how challenging it is to convince developers to change their existing workflow. As an example, a major project I worked on in a prior role was a no-code tool for creating and maintaining functional UI test automation. There’s an entire ecosystem of tools such as Appium, Selenium and Cypress that allow you to create such automation by writing code and using so-called “selectors” to interact with specific screen elements of your application. Even though selectors are widely acknowledged to be difficult to work with and are prone to breaking as your application changes, it was nevertheless a huge challenge for us to overcome inertia and convince devs and QA engineers to adopt an entirely novel and unfamiliar paradigm for test automation, even if, from our perspective, it conferred significant advantages such as abstracting away selectors.
One exciting aspect of ML observability is that it is a very young vertical within MLOps and there are no deeply entrenched incumbents to displace. We have the freedom of a relatively blank slate and the opportunity to define the shape and trajectory of this vertical for years to come.
What’s something on your Github or blog that you’re proud of that’s worth sharing here?
Check out this post where I visualize high-dimensional genomic data. It includes an in-depth explanation of kernel PCA and some cool interactive graphics at the end.
What’s one thing that has surprised you since joining Arize?
I’ve been impressed by how customer-obsessed the team is as a collective. Our co-founders Aparna Dhinakaran and Jason Lopatecki not only live in customer calls, but ensure that each employee is well-versed in the mission of the company and can articulate the value proposition we provide. People take pride in building a product that is best-in-class and delights the user, and there’s a strong sense that everyone is bought in, on board, and rowing in the same direction.
What is the one app on your phone that you can’t live without (bonus points for pointing out its underlying ML-powered systems)?
The first thing I do when I wake up in the morning is check out the latest articles on The Athletic, a sports news app. I follow NBA basketball closely, especially the Golden State Warriors, and am getting into European soccer. The app curates a personalized feed of articles and content, presumably using a recommendation system that learns from both users’ stated interests and past interactions with the app. Just one of many ways in which I interact with an ML-powered system in my daily life.
What’s your favorite restaurant where you live?
Until recently, I lived in Honolulu and enjoyed Koko Head Cafe, a brunch restaurant with a unique take on local Hawaiian fusion (think ricotta pancakes with macadamia nuts and haupia, a Hawaiian dessert made from coconut milk). I recently moved back to my hometown of Oakland, California. My go-to spot here is Geta Sushi on Piedmont for simple but quality Japanese fare.
Reach out to Xander and get your MLOps and ML observability questions answered in the Arize Community!