- Invest in metrics - these are your customers
- The unknowns in data projects are different to those in traditional software engineering projects and so customers and sponsors need to learn how to understand progress and expectations
- Plan for mitigation - the only certain thing about data is that it will contain errors, so design for data mitigation from the start
- A data engineer or scientist needs to be an expert in communicating with data - invest in this skill
- Wallow in data - present results and discuss insights with your peers to make informed and balanced decisions for the team
The book Agile Machine Learning by Eric Carter and Matthew Hurst describes how the guiding principles of the Agile Manifesto have been used by machine learning teams in data projects. It explores how to apply agile practices for dealing with the unknowns of data and inferencing systems, using metrics as the customer.
Photo: JumpStory |
InfoQ readers can download an extract of Agile Machine Learning - Chapter reprinted with permission from Apress (2020) an imprint of Springer Nature.
InfoQ interviewed Matthew Hurst about using agile for a data engineering team, rebuilding the data catalog every day, continuous integration and deployment of data changes, the benefits of rewriting software, doing sprint demo meetings, what can be done to break the pace for teams that are working at a sustainable pace, and technical excellence in data projects.
Read more...
Agile Machine Learning: Effective Machine Learning Inspired by the Agile Manifesto |
Source: InfoQ.com