We hear it every day, whether we are talking with a startup founder about model selection or with an engineering director of a large manufacturing business about machine learning, training data set, variables selection, and more. Product development is risky and feature set validation can help reduce it. But how?
Products often fail to launch. And when they do, financial results often do not meet profit organization expectations. Why are these themes so common? There are some basic learning methods approaches to feature selection that is easy to overlook, but critical to product success.
In the early days of any product development effort, somebody test data and decides that a market opportunity can be addressed by some combination of features effectively delivered to end users in the form of a final model product.
The ways in which companies define feature sets around which products are developed varies widely. Large corporations often use traditional, data-driven quantitative market research, with focus-group validation of the results. Startups may rely on the expertise of a founder for feature validation set definition. Both of these are valid starting points.
Where things fall apart is when development goes too far based on this early feature set definition.
With any combination of features, compromises and trade-off decisions are made. As development progresses, it becomes more and more difficult to re-visit or modify these early trade-offs. We call this “development momentum,” but it is not necessarily a good thing.
The result of unchecked development momentum can be a late stage product design that incorporates the wrong validation dataset. Taken to the extreme, this momentum can lead to investment in production tooling and launch of a product that includes features that the market is not willing to pay for.
One effective solution is an early and ongoing validation of the feature set data points throughout development. This is best achieved with fast iteration cycles, frequent prototyping, predictive modeling and testing with actual end users. A good development plan will include time and budget for these cycles.
Effective development managers will understand that any one of these cycles may uncover new opportunities, or may uncover deal breakers for the product. The culture of some businesses will drive managers to label this as a failure. But in fact, the real failure to be avoided is tooling up and launching a product that is not a commercial success. A pivot or justified stoppage before tooling investment is often a win.
In addition to feature set validation, functional validation and manufacturability validation are key to product success. More on these topics in the future.