Performance Testing Overview
Performance testing measures how systems behave under various load conditions, including response times, throughput, resource utilization, and scalability. It helps identify bottlenecks, validate performance requirements, and ensure systems can handle expected loads.
When to Use Validating response time requirements Measuring API throughput and latency Testing database query performance Identifying performance bottlenecks Comparing algorithm efficiency Benchmarking before/after optimizations Validating caching effectiveness Testing concurrent user capacity Types of Performance Tests Load Testing: Normal expected load Stress Testing: Beyond normal capacity (see stress-testing skill) Spike Testing: Sudden load increases Endurance Testing: Sustained load over time Scalability Testing: Performance at different scales Benchmark Testing: Component-level performance Instructions 1. k6 for API Load Testing // load-test.js import http from 'k6/http'; import { check, sleep } from 'k6'; import { Rate, Trend } from 'k6/metrics';
// Custom metrics const errorRate = new Rate('errors'); const orderDuration = new Trend('order_duration');
// Test configuration export const options = { stages: [ { duration: '2m', target: 10 }, // Ramp up to 10 users { duration: '5m', target: 10 }, // Stay at 10 users { duration: '2m', target: 50 }, // Ramp up to 50 users { duration: '5m', target: 50 }, // Stay at 50 users { duration: '2m', target: 0 }, // Ramp down to 0 ], thresholds: { http_req_duration: ['p(95)<500'], // 95% of requests under 500ms http_req_failed: ['rate<0.01'], // Error rate under 1% errors: ['rate<0.1'], // Custom error rate under 10% }, };
// Test data const BASE_URL = 'https://api.example.com'; let authToken;
export function setup() {
// Login once and get auth token
const loginRes = http.post(${BASE_URL}/auth/login, {
email: 'test@example.com',
password: 'password123',
});
return { token: loginRes.json('token') }; }
export default function (data) {
// Test 1: Get products (read-heavy)
const productsRes = http.get(${BASE_URL}/products, {
headers: { Authorization: Bearer ${data.token} },
});
check(productsRes, { 'products status is 200': (r) => r.status === 200, 'products response time < 200ms': (r) => r.timings.duration < 200, 'has products array': (r) => Array.isArray(r.json('products')), }) || errorRate.add(1);
sleep(1);
// Test 2: Create order (write-heavy) const orderPayload = JSON.stringify({ userId: 'user-123', items: [ { productId: 'prod-1', quantity: 2 }, { productId: 'prod-2', quantity: 1 }, ], });
const orderRes = http.post(${BASE_URL}/orders, orderPayload, {
headers: {
'Content-Type': 'application/json',
Authorization: Bearer ${data.token},
},
});
const orderSuccess = check(orderRes, { 'order status is 201': (r) => r.status === 201, 'order has id': (r) => r.json('id') !== undefined, });
if (!orderSuccess) { errorRate.add(1); }
orderDuration.add(orderRes.timings.duration);
sleep(2);
// Test 3: Get order details
if (orderSuccess) {
const orderId = orderRes.json('id');
const orderDetailRes = http.get(${BASE_URL}/orders/${orderId}, {
headers: { Authorization: Bearer ${data.token} },
});
check(orderDetailRes, {
'order detail status is 200': (r) => r.status === 200,
}) || errorRate.add(1);
}
sleep(1); }
export function teardown(data) { // Cleanup if needed }
Run k6 test
k6 run load-test.js
Run with different scenarios
k6 run --vus 100 --duration 30s load-test.js
Output to InfluxDB for visualization
k6 run --out influxdb=http://localhost:8086/k6 load-test.js
- Apache JMeter
<!-- HTTP Request: Get Products -->
<HTTPSamplerProxy testname="GET /products">
<stringProp name="HTTPSampler.domain">api.example.com</stringProp>
<stringProp name="HTTPSampler.path">/products</stringProp>
<stringProp name="HTTPSampler.method">GET</stringProp>
</HTTPSamplerProxy>
<!-- Assertions -->
<ResponseAssertion testname="Response Code 200">
<stringProp name="Assertion.test_field">Assertion.response_code</stringProp>
<stringProp name="Assertion.test_type">8</stringProp>
<stringProp name="Assertion.test_string">200</stringProp>
</ResponseAssertion>
<DurationAssertion testname="Response Time < 500ms">
<stringProp name="DurationAssertion.duration">500</stringProp>
</DurationAssertion>
<!-- Timers -->
<ConstantTimer testname="Think Time">
<stringProp name="ConstantTimer.delay">2000</stringProp>
</ConstantTimer>
</ThreadGroup>
</TestPlan>
Run JMeter test
jmeter -n -t test-plan.jmx -l results.jtl -e -o report/
Generate report from results
jmeter -g results.jtl -o report/
- pytest-benchmark for Python
test_performance.py
import pytest from app.services import DataProcessor, SearchEngine from app.models import User, Product
class TestDataProcessorPerformance: @pytest.fixture def large_dataset(self): """Create large dataset for testing.""" return [{'id': i, 'value': i * 2} for i in range(10000)]
def test_process_data_performance(self, benchmark, large_dataset):
"""Benchmark data processing."""
processor = DataProcessor()
result = benchmark(processor.process, large_dataset)
assert len(result) == len(large_dataset)
# Benchmark will report execution time
def test_filter_data_performance(self, benchmark, large_dataset):
"""Test filtering performance with different conditions."""
processor = DataProcessor()
def filter_operation():
return processor.filter(large_dataset, lambda x: x['value'] > 5000)
result = benchmark(filter_operation)
assert all(item['value'] > 5000 for item in result)
@pytest.mark.parametrize('dataset_size', [100, 1000, 10000])
def test_scalability(self, benchmark, dataset_size):
"""Test performance at different scales."""
processor = DataProcessor()
data = [{'id': i, 'value': i} for i in range(dataset_size)]
benchmark(processor.process, data)
class TestSearchPerformance: @pytest.fixture def search_engine(self): """Setup search engine with indexed data.""" engine = SearchEngine() # Index 10,000 products for i in range(10000): engine.index(Product(id=i, name=f"Product {i}")) return engine
def test_search_performance(self, benchmark, search_engine):
"""Benchmark search query."""
result = benchmark(search_engine.search, "Product 5000")
assert len(result) > 0
def test_search_with_filters(self, benchmark, search_engine):
"""Benchmark search with filters."""
def search_with_filters():
return search_engine.search(
"Product",
filters={'price_min': 10, 'price_max': 100}
)
result = benchmark(search_with_filters)
Run benchmarks
pytest test_performance.py --benchmark-only
pytest test_performance.py --benchmark-compare
- JMH for Java Benchmarking // PerformanceBenchmark.java import org.openjdk.jmh.annotations.; import java.util.; import java.util.concurrent.TimeUnit;
@BenchmarkMode(Mode.AverageTime) @OutputTimeUnit(TimeUnit.MICROSECONDS) @State(Scope.Benchmark) @Fork(value = 1, warmups = 1) @Warmup(iterations = 3, time = 1) @Measurement(iterations = 5, time = 1) public class CollectionPerformanceBenchmark {
@Param({"100", "1000", "10000"})
private int size;
private List<Integer> arrayList;
private List<Integer> linkedList;
private Set<Integer> hashSet;
@Setup
public void setup() {
arrayList = new ArrayList<>();
linkedList = new LinkedList<>();
hashSet = new HashSet<>();
for (int i = 0; i < size; i++) {
arrayList.add(i);
linkedList.add(i);
hashSet.add(i);
}
}
@Benchmark
public void arrayListIteration() {
int sum = 0;
for (Integer num : arrayList) {
sum += num;
}
}
@Benchmark
public void linkedListIteration() {
int sum = 0;
for (Integer num : linkedList) {
sum += num;
}
}
@Benchmark
public void arrayListRandomAccess() {
for (int i = 0; i < size; i++) {
arrayList.get(i);
}
}
@Benchmark
public void linkedListRandomAccess() {
for (int i = 0; i < size; i++) {
linkedList.get(i);
}
}
@Benchmark
public boolean hashSetContains() {
return hashSet.contains(size / 2);
}
public static void main(String[] args) throws Exception {
org.openjdk.jmh.Main.main(args);
}
}
// Run: mvn clean install // java -jar target/benchmarks.jar
- Database Query Performance
test_database_performance.py
import pytest import time from sqlalchemy import create_engine from app.models import User, Order
class TestDatabasePerformance: @pytest.fixture def db_session(self): """Create test database session.""" engine = create_engine('postgresql://localhost/testdb') Session = sessionmaker(bind=engine) return Session()
def test_query_without_index(self, db_session, benchmark):
"""Measure query performance without index."""
def query():
return db_session.query(User).filter(
User.email == 'test@example.com'
).first()
result = benchmark(query)
def test_query_with_index(self, db_session, benchmark):
"""Measure query performance with index."""
# Assume email column is indexed
def query():
return db_session.query(User).filter(
User.email == 'test@example.com'
).first()
result = benchmark(query)
def test_n_plus_one_query(self, db_session):
"""Identify N+1 query problem."""
start = time.time()
# ❌ N+1 queries
users = db_session.query(User).all()
for user in users:
orders = user.orders # Triggers separate query for each user
n_plus_one_time = time.time() - start
# ✅ Eager loading
start = time.time()
users = db_session.query(User).options(
joinedload(User.orders)
).all()
eager_time = time.time() - start
assert eager_time < n_plus_one_time, "Eager loading should be faster"
print(f"N+1: {n_plus_one_time:.3f}s, Eager: {eager_time:.3f}s")
def test_pagination_performance(self, db_session, benchmark):
"""Test pagination efficiency."""
def paginate():
return db_session.query(Product)\
.order_by(Product.id)\
.limit(50)\
.offset(1000)\
.all()
results = benchmark(paginate)
assert len(results) == 50
- Real-Time Monitoring // performance-monitor.js class PerformanceMonitor { constructor() { this.metrics = []; }
async measureEndpoint(name, fn) { const start = performance.now(); const startMemory = process.memoryUsage();
try {
const result = await fn();
const duration = performance.now() - start;
const endMemory = process.memoryUsage();
this.metrics.push({
name,
duration,
memoryDelta: {
heapUsed: endMemory.heapUsed - startMemory.heapUsed,
external: endMemory.external - startMemory.external,
},
timestamp: new Date(),
});
return result;
} catch (error) {
throw error;
}
}
getStats(name) { const measurements = this.metrics.filter(m => m.name === name); const durations = measurements.map(m => m.duration);
return {
count: measurements.length,
mean: durations.reduce((a, b) => a + b, 0) / durations.length,
min: Math.min(...durations),
max: Math.max(...durations),
p50: this.percentile(durations, 50),
p95: this.percentile(durations, 95),
p99: this.percentile(durations, 99),
};
}
percentile(values, p) { const sorted = values.sort((a, b) => a - b); const index = Math.ceil((p / 100) * sorted.length) - 1; return sorted[index]; } }
// Usage in tests const monitor = new PerformanceMonitor();
test('API endpoint performance', async () => { for (let i = 0; i < 100; i++) { await monitor.measureEndpoint('getUsers', async () => { return await fetch('/api/users').then(r => r.json()); }); }
const stats = monitor.getStats('getUsers');
expect(stats.p95).toBeLessThan(500); // 95th percentile under 500ms expect(stats.mean).toBeLessThan(200); // Average under 200ms });
Performance Metrics Response Time Average: Mean response time Median (P50): 50th percentile P95: 95th percentile (SLA target) P99: 99th percentile (worst case) Max: Slowest response Throughput Requests/second: Number of requests processed Transactions/second: Completed transactions Data transferred: MB/s or GB/s Resource Utilization CPU: Percentage usage Memory: Heap, RSS, external Network: Bandwidth usage Disk I/O: Read/write operations Best Practices ✅ DO Define clear performance requirements (SLAs) Test with realistic data volumes Monitor resource utilization Test caching effectiveness Use percentiles (P95, P99) over averages Warm up before measuring Run tests in production-like environment Identify and fix N+1 query problems ❌ DON'T Test only with small datasets Ignore memory leaks Test in unrealistic environments Focus only on average response times Skip database indexing analysis Test only happy paths Ignore network latency Compare without statistical significance Tools Load Testing: k6, JMeter, Gatling, Locust, Artillery Benchmarking: JMH (Java), pytest-benchmark (Python), Benchmark.js (Node.js) Profiling: Chrome DevTools, py-spy, VisualVM Monitoring: Prometheus, Grafana, New Relic, Datadog Database: EXPLAIN ANALYZE, pg_stat_statements Examples
See also: stress-testing, continuous-testing, code-metrics-analysis for comprehensive performance analysis.