Archive for February, 2015

Data-Driven in one image

Wednesday, February 18th, 2015



Photo credit:

Why Windows 10 will succeed where Windows 8 failed –Data Driven Quality

Sunday, February 15th, 2015


You can’t sort of A/B test your way before the product launches, because you don’t have it in users’ hands yet. You need to use your product intuition to make the right choices. You make these choices and people are paying you to make them.

Steven Sinofsky, former Microsoft President Windows

D11 Conference, May 2013



It starts with everyone in an organization having the curiosity, having the questions, trying to test out hypotheses, trying to gain insights, and then taking actions. You need to have a data culture inside of your organization.  …this is perhaps the most paramount thing inside of Microsoft. The thing we need to do better on is to be able learn from our customers and the data they exhaust … and continuously improve our products and services, that is job number one by far

Satya Nadella, Microsoft CEO

Executive Keynotes from SQL Server 2014 Launch, April 2014

(I cannot find the quote recorded anywhere, but I personally wrote it down while viewing his keynote for the SQL Server Launch. Some oblique references here, here and here)

TestOps redux?

Friday, February 13th, 2015

At one point I attempted to coin the phrase TestOps (pivoting off the increasingly popular DevOps)

Quality in the Cloud: The new Role of TestOps – March 2012

But ultimately there were a few things I did not like about that phrase.  Primarily was that TestOps really is already a part of DevOps.  DevOps is woefully mis-named and should be EngOps or something like that since Test, Dev, PM, and other all share the same goal of producing customer-delighting software and similarly all need to leverage Ops (aka production data) to do so.

What I really prefer is the concept of DDQ (Data-Driven Quality), which includes and expands on the TestOps and Test in Production ideas.

But it looks like my friends at SOASTA are up to something with TestOps….

[And here is where was going to embed the tweet from SOASTA CEO Tom Lounibos, but he deleted it! :-) (it was here).]  Apparently he thought I objected to his use of the term, but far from it. I am more than happy to have friends in the industry like SOASTA leverage good ideas like TestOps (but I really prefer DDQ :-)

Those pesky customers

Sunday, February 8th, 2015

steve_jobs3It is funny that whenever I teach (or lecture, or cajole) software engineering teams about the necessity for customer centricity, I almost inevitably get challenged with the example of Steve Jobs’ Apple.   Jobs did not care about focus groups, Jobs did not task his teams to deeply mine customer data… he just told them what was right and they did it.  Sometimes they got an Apple Lisa out of it, but it was from Jobs’ mind that the iPod, iPhone, and Mac Air sprang forth.

The exception that does not really prove the rule

Well I do not know precisely what to do with Mr. Jobs’ legacy in this context.  I could try to argue how much of his innovation came on the shoulders of others.  The original Mac UI and mouse paradigm having started at Xerox Parc and such… but that is not really relevant.  There is no denying that Jobs’ had the ability to tap into latent need of his customers.

thMAJ42OAXLatent need?  Yup, that is when you give customers what they desperately want, but they do not even know they want it.  For example, Henry Ford is famous for saying

If I had asked my customers what they wanted they would have said a faster horse.

Nobody got rich delivering faster horses… or at least not as rich as Mr. Ford got delivering automobiles to the middle and working classes.

I am the decider

So maybe Jobs and Ford are exceptional.   That does not change the need for understanding your customers (with hard data).   Just recently I was talking to a software entrepreneur who had just launched a new app targeted at fitness trainers and the gyms they work at.   He told me he was dismayed to find all these features that were missing that trainers and gym managers wanted and needed to make the app useful to them.   I started to suggest that this was good news and this limited release gave him precious data about his customers.  His response surprised me.  He said that all these ex-Microsoft guys that work for him suggested a beta release to collect data but he HATED that idea.  He told me this was his vision, and it was that vision he wanted to see realized, resulting in great success for the app.


I am thrilled to see ex *Microsoft* guys associated with customer centricity and data-driven quality.  This is a stark change from just a short while ago, and one that I can humbly claim to have been some part of.   And I guess I should not be shocked that every entrepreneur fancies himself the next Jobs with a Jobsian vision, but I also hope that such entrepreneurs can learn that perhaps they are not, and that data about customers, their problems, and what thrilling those customers with the perfect solution looks like, is perhaps the most powerful tool for success.

Culture of data and experimentation

amazon-instant-video-06-535x535In closing consider the case of my current boss, Jeff Bezos.  Bezos certainly had (and has) a vision for Amazon, but he said early on (and later codified) that Amazon will focus what is right for the customer.  Certainly there are many “Jeff projects” that go to production without a focus group and based purely on Jeff’s vision, but there is always metrics that everyone knows up front by which we will assess the idea, use to change it, or scrap it altogether.  I remember in the early days of Amazon Instant Video when we first launched streaming.  Jeff was sure that the free preview should play the first n minutes of the movie, so that is what we implemented.  The data quickly showed that this is not what people wanted, and they wanted the traditional preview/trailer.  Sure we all knew that would be the case… or did we?  Without first trying this idea we truly did not know… perhaps this is what people had been waiting for and it would be Amazon’s advantage?  It wasn’t, and that is fine… this is the culture of experimentation driven by real users and real data that a company needs to embrace to succeed.

Bing’s fault or Bartell Drug’s fault?

Sunday, February 1st, 2015

Not quite Testing in Production…..but amusing nonetheless