Performance Testing Software Systems
Desigining Performance Tests with UCML
Overview:
Good load generation tools don't make good tests any easier to design (though they sometimes make well designed tests easier to implement.) As compared to functional testing, performance tests generaly take longer to conduct, typically must be conducted one at a time, and are more commonly inconclusive on their own. Additionally, it is nearly impossible to determine what performance tests will provide significant information value until the results from the previous test are analyzed. These facts make good performance test design absolutely critical to successful performance test projects.
This unique workshop builds on the Performance Testing Software Systems heuristic approach that focuses on mitigating risks to the business and satisfying end users in commercially driven software development environments. This approach marries the software testing insights of James Bach, Rob Sabourin, Cem Kaner and many other members of the Context-Driven School of software testing with the performance testing insights of Alberto Savoia, Ross Collard, Roland Stens, and the rest of the WOPR (Workshop On Performance and Reliability) community. The approach has a track record of success with regard to adequately mitigating business risk in time to keep pace with the commercial aspects of the project. The Microsoft patterns & practices book Performance Testing Guidance for Web Applications by J.D. Meier, Scott Barber, Carlos Farre, Prashant Bansode, and Dennis Rea complements the material presented in this workshop.
This version of the PTSS series of workshops is targeted for the performance testers, analysts, architects and managers who most significantly contribute to performance test design. The course focuses on practicle methods to design and document performance tests that are likely to result in significant information value and ensure that performance testing is focused on achieving buisiness objectives, reducing project risk, and avoiding bad press. Further, it lays out a method for managing performance testing that ensures managers have visibility into the performance testing effort and that peformance testers can be held accountable for their work without the performance testing and tuning losing the flexibility to adapt their testing to the results they are receiving.
This 1-day workshop is primarily offered as an on-site course, but sometimes we work with organizations that arrange to make it public. Contact us for more information.
Course Objectives:
In this course, you will learn:
- Concise, universal heuristics and models for rapid application of core principles
- How to determine and apply oracles to software system performance criteria
- How to rapidly design performance tests that will achieve your performance testing objectives
- How to communicate performance test design to non-performance testers
Course Outline:
Introduction
- Overview
- Why Design is Critical for Effective Performance Testing
CORE PRINCIPLE: Context
- Why Do We Test Performance?
- Criteria
- Objectives
- Procedures
- Environments
- Project
- User
- Business
- Systems
- Know The Commanders Intent
CORE PRINCIPLE:Criteria
- Understanding Boundaries
- Goals
- Requirements
- Thresholds
- Constraints
- Determine Testing Objectives
- The Oracle Problem
CORE PRINCIPLE: Design
Begin With The End In Mind:- Investigate or Validate
- End-to-End or Component
- Times or Resources
- Anticipated or Stressful
- Frequent
- Intensive
- Business-Critical
- Legal
- Obvious
- Technically Risky
- Stakeholder Mandated
- Size
- Caching
- Order
- Response Codes
- Number
CORE PRINCIPLE: UCML
- Overview
- Basic UCML
- Extending UCML
Keynotes by: Tim Lister & Cem Kaner
Tutorials by: Fiona Charles, Harry Robinson, & Cem Kaner
"I tend to rotate conferences from year to year. I've found that most have their own personalities that are slow to change. CAST is different. I always find it progressive and enlightening. It is the one conference I look forward to attending every year."
