SigOpt, a startup incubated by Y Combinator, has a big vision — co-founder and CEO Scott Clark told me that he aims to “optimize anything that has tunable parameters.”
Some of that might sound familiar, especially since there’s a well-known tech company with “optimize” in its name, but Clark said that SigOpt goes beyond A/B testing. Put (relatively) simply, it doesn’t just let you test different variations, but instead examines the data and recommends “what experiments to run next,” so you can continuously make something better.
To use one of Clark’s examples, if you work for a company that wants to test different versions of an ad, you might normally set up tests for each version, then choose the most effective variant at the end of the test. With SigOpt, you can provide the creative assets and guidelines, then SigOpt creates the different versions, tests them, creates new versions based on those tests, and so on, to automatically maximize revenue or clicks.
Even more intriguingly, SigOpt helps you optimize physical experiments. For example, Clark said one of his initial customers is using SigOpt to test different chemical combinations in creating shaving creams. It shouldn’t matter what you’re testing — Clark said SigOpt’s technology just uses your data and “builds up this model of what it thinks the world looks like. … It’s designed to be a very general system.”
Clark has a Ph.D. in applied mathematics from Cornell, and he said SigOpt draws on the subsequent work he did as an engineer on Yelp’s ad-targeting team, specifically the Metric Optimization Engine, or MOE. That’s an open-source optimization tool that he co-developed, and beyond Yelp, it’s actually used by Netflix. (Xavier Amatrain, who was then Netflix’s director of algorithms engineering, mentions MOE a little under 25 minutes into this talk on machine learning.)
While MOE is open source, Clark and his co-founders are working to build products and services around it. He compared SigOpt’s goals for commercializing MOE to what Cloudera has done for Hadoop.
“Netflix has a really world-class data science team, but we want to be able to provide that same level of optimization to everyone,” he said.
Clark gave me a quick demo of the SigOpt interface and, while it clearly requires some technical know-how (plus, it’s a more manual process if you’re entering data from physical experiments), it did seem relatively straightforward, running tests with just a few lines of code.
Nonetheless, using SigOpt is still a bit above my head. Over time, though, Clark said it will become “simpler still” and will bring in more of the company’s “secret sauce.”
By the way, I was curious about how broadly Clark thought this optimize-everything idea could go. Given the buzz around the quantified self movement, could SigOpt ultimately be used to run experiments on yourself? Probably not on an individual level, he said, but it could aggregate, say, health information from a large number of users and provide recommendations based on that data.