The return of A-Z Testing in Production – STPCon Fall 2012

Download the Slide Deck:



The folks at STPCon had asked me to give an encore presentation of my March 2012 talk A to Z Testing in Production: TiP Methodologies, Techniques, and Examples. The new talk has some substantial updates including more focus on the use of Big-Data and a simplified categorization of test in production methodologies.  Reflecting this the new talk title and abstract are:

A to Z Testing in Production:

Industry Leading Techniques to Leverage Big Data for Quality

Testing in production (TiP) is a set of software methodologies that derive quality assessments not from test results run in a lab but from where your services actually run – in production. The big data pipe from real users and production environments can be used in a way that both leverages the diversity of production while mitigating risks to end users. By leveraging this diversity of production we are able to exercise code paths and use cases that we were unable to achieve in our test lab or did not anticipate in our test planning.

This session introduces test managers and architects to TiP and gives these decision makers the tools to develop a TiP strategy for their service. Methodologies like Controlled Test Flights, Synthetic Test in Production, Load/Capacity Test in Production, Data Mining, Destructive Testing and more are illustrated with examples from Microsoft, Netflix, Amazon, and Google. Participants will see how these strategies boost ROI by moving focus to live site operations as their signal for quality.

You can download the slide deck and listen to the audio from the previous (March) talk here

Also Mark Tomlinson interviewed me at STPCon, you can hear that here.


Pingback from Testing the Limits With Microsoft’s Seth Eliot | Software Testing Blog
Time June 28, 2012 at 6:16 am

[…] STPCon has been kind enough to ask me back to give an encore of A-Z Testing in Production for the October 2012 conference. If you missed it the first time come on by. If you already saw […]

Pingback from Alex Podelko » Testing in Production
Time December 20, 2012 at 12:24 pm

[…] found Seth Eliot’s presentation A to Z Testing in Production: Industry Leading Techniques to Leverag… at STP Online Summit was very thought-provoking. Definitely a non-orthodox view on what is going on […]

Pingback from Seth Eliot's Blog » Blog Archive » TiP is misunderstood – perhaps DDQ is Better
Time January 12, 2015 at 6:54 am

[…] spent a long time talking to folks about the merits of a conscientious Testing in Production (TiP) strategy.  But I knew TiP had a bad rap.  I even shared the story of how some would […]

Write a comment