3.2 Testing Framework & Methodology

Consistent testing methodology ensures performance measurements are comparable across time and between Editions. This page documents the tools, procedures, and metrics used for Velocity Stack performance testing.

Testing Tools

ToolPurposeKey Strengths
PingdomOverall performance gradeSimple scoring, waterfall analysis, historical tracking
GTmetrixDetailed Core Web VitalsLighthouse integration, video playback, comparison
PageSpeed InsightsGoogle’s official metricsReal user data (CrUX), mobile/desktop split
Chrome DevToolsRequest-level debuggingNetwork throttling, cache control, waterfall

Primary vs. Secondary Tools

Primary: Pingdom

Pingdom is the primary tool for Velocity Stack baseline testing because:

  • Consistent test infrastructure across runs
  • Clear single-number performance grade
  • Detailed waterfall for debugging
  • Content breakdown by type and domain
  • Historical tracking for trend analysis

Secondary: GTmetrix & PageSpeed Insights

Used for deeper analysis when Pingdom identifies issues:

  • Core Web Vitals breakdown (LCP, FID, CLS)
  • Specific optimization recommendations
  • Mobile vs. desktop comparison
  • Real user experience data (PageSpeed only)

Test Procedure

Step 1: Pre-Test Preparation

  1. Verify site is accessible (no maintenance mode)
  2. Confirm cache is enabled and warmed
  3. Check CDN Layer status is active
  4. Note any recent changes since last test

Step 2: Cache Warming

  1. Visit the test URL in a browser
  2. Open DevTools → Network tab
  3. Check response headers for cache hit indicator
  4. If “miss”, refresh and verify “hit” on second request

Step 3: Run Pingdom Test

  1. Navigate to tools.pingdom.com
  2. Enter the test URL
  3. Select test location (use consistent location)
  4. Click “Start Test”
  5. Wait for results

Step 4: Record Results

Capture the following metrics:

  • Performance grade (letter and number)
  • Page size (KB)
  • Load time (ms)
  • Request count
  • Content breakdown by type
  • Any failed requests or errors

Step 5: Screenshot & Archive

  1. Screenshot the summary panel
  2. Screenshot the waterfall (if notable)
  3. Screenshot content breakdown
  4. Save with naming convention: [tool]-[date]-[metric].webp

Test Location Selection

Choose a test location based on target audience:

AudienceRecommended LocationNotes
US East CoastNew York, Washington DCGood for US-focused sites
US West CoastSan Francisco, Los AngelesTests cross-country latency
EuropeLondon, FrankfurtTests CDN edge performance
GlobalMultiple locationsRun from 3+ locations for comparison

Important: Use the same test location for all baseline comparisons. Changing locations invalidates trend data.

Metrics Definitions

Core Web Vitals

MetricWhat It MeasuresGoodPoor
Largest Contentful Paint (LCP)Loading — when main content is visible≤ 2.5s> 4.0s
First Input Delay (FID)Interactivity — response to first interaction≤ 100ms> 300ms
Cumulative Layout Shift (CLS)Visual stability — unexpected layout movement≤ 0.1> 0.25

Supporting Metrics

MetricWhat It MeasuresTarget
Time to First Byte (TTFB)Server response time< 600ms
First Contentful Paint (FCP)First visible content rendered< 1.8s
Total Blocking Time (TBT)Main thread blocked by JS< 200ms
Speed IndexHow quickly content is visually displayed< 3.4s
Time to Interactive (TTI)Page fully interactive< 3.8s

Testing Frequency

EventTest RequiredTools
New Edition deployedFull baselinePingdom, GTmetrix, PageSpeed
Template version updateComparison testPingdom
Capability updateSpot checkPingdom
Cache configuration changeBefore/after comparisonPingdom, DevTools
Monthly maintenanceTrend trackingPingdom
Performance issue reportedDiagnosticAll tools

Results Documentation

File Naming Convention

[tool]-[YYYY-MM-DD]-[HH_MM_SS]-[metric].webp

Examples:
pingdom-2026-01-11-01_28_06-performance.webp
pingdom-2026-01-11-01_28_06-file-requests.webp
pingdom-2026-01-11-01_28_06-grouped-content-type.png
gtmetrix-2026-01-15-performance-report.webp

Storage Location

  • Documentation Platform: /Performance-Test-Results/[Edition]/[Date]/
  • This site: Media library with descriptive alt text

Interpreting Results

Green Flags

  • Performance grade A (90+)
  • All requests from single domain
  • Page size under 100 KB
  • Load time under 200ms (cached)
  • Zero layout shift (CLS = 0)
  • Cache hit on repeat requests

Red Flags

Render-blocking resources

Third-party domain requests

Failed requests (4xx, 5xx errors)

Page size over 1 MB

Load time over 3 seconds

High CLS (content jumping)

Cache miss on repeat requests