Translate to multiple languages

Subscribe to my Email updates
Enjoy what you've read, make sure you subscribe to my Email Updates

Wednesday, August 26, 2015

The Theoretical Underbelly of Optimizing Machine Learning Systems

Photo: Nicole Hemsoth
"This week we will be delving into systems and architectures designed for machine learning, but since it’s Monday and there is quite a bit to come, it seemed worthwhile to take a step back and consider how architects and software developers are thinking about the current landscape." according to Nicole Hemsoth, Co-founder and co-editor Nicole Hemsoth brings insight from the world of high performance computing hardware and software as well as data-intensive systems and frameworks.

Photo: The Platform

With everyone from Intel touting the next generation deep learning and machine learning as a partial basis for their Altera buy, to webscale companies like Microsoft, Google, Baidu and others seeking ways to boost machine learning algorithms with hardware, accelerator, and of course, software approaches, the larger conversations tend to get lost in the mix. For instance, what does it mean to optimize for these codes—and what are the system design choices that seem to be the best fits?

Photo: Joshua Bloom
It may seem a bit odd to take machine learning systems guidance from the point of view of a UC Berkeley professor who specializes in gamma ray bursts and black holes, but when it comes to applied scientific algorithms across massive, shifting datasets, Dr. Joshua Bloom does have a view into the complexity—and the systems required to tackle it. This is true both in terms of understanding the machines required to process cosmological simulations (i.e. large-scale supercomputers) but for Bloom and his group at Berkeley, the equal challenge lies in the tradeoffs of that computation versus the requirements of the models. Sacrificing accuracy for performance, scalability for complexity, memory for model depth—the list goes on. But at the core, Bloom says, is an increased need for machine learning systems and people building them to understand the purpose of optimizations and apply those to both the hardware and software (and by default, the outcomes and implementation)

Machine learning systems are alive, he says, both “influencing and responding to their environment. At best, they’re valuable, resilient, functioning systems composed of many imperfect parts with many weak contracts between them, built by fallible individuals with broken communication channels, all of whom are living a resource constrained world that’s constantly changing, with the results being consumed by exacting and capricious individuals.” This definition, as he told a group at PyData Seattle, which was hosted by Microsoft, indicates what we already know. This is hard stuff.

The difficulty lies in variability—and that variability means that there are never any standard tradeoffs that suit any algorithms, which is especially true since the same models, once applied to different datasets, can change performance-wise dramatically.

Source: The Platform