SoapUI is one of the most widely used tools for web services and API testing, and for good reason. It handles both SOAP and REST protocols, supports a range of testing types from functional to security to load, and converts functional test cases into performance tests with minimal effort. But once testers move beyond basic usage and start tackling real-world SoapUI performance testing, a predictable set of problems surfaces.
In this blog, we discuss specific, documented challenges that real SoapUI users run into, along with the precise fixes that actually work.
Table of Contents
1. Starting Without a Baseline (And Why That Kills Your Results)
Most teams jump straight into SoapUI load testing with high thread counts and walk away with results they can’t interpret. Without a baseline, there’s no reference point to compare against. You don’t know whether 900ms is normal or a regression.
The fix: Always start with SoapUI’s Simple Strategy at a low thread count. The default of 5 threads with a 1000ms delay is a deliberate starting point. Run this first pass to catch any threading or resource-locking issues and establish what “normal” looks like for your service. Once that baseline is documented, you can meaningfully compare results as you ramp up load.
2. Choosing the Wrong Load Strategy for the Scenario
SoapUI offers four load strategies: Simple, Thread, Variance, and Burst. Testers who default to Simple for every scenario end up with misleading results because each strategy simulates a fundamentally different behavior.
Here’s what actually happens with each:
- Simple Strategy – Runs a fixed number of threads with a configurable delay. Best for baseline testing and long-running soak tests. However, it does not guarantee a fixed number of executions per second.
- Thread Strategy – Linearly increases thread count from a start value to an end value over the test duration. It’s best for identifying at which thread count TPS peaks or errors begin. For example, setting start threads to 1, end threads to 10, and duration to 300 seconds lets SoapUI incrementally scale load in equal intervals.
- Variance Strategy – Varies thread count in a “sawtooth” pattern over a configured interval. Useful for simulating fluctuating real-world traffic.
- Burst Strategy – Sends a high volume of threads (20+) in short controlled bursts, then measures how well the service recovers. This is the right tool for SoapUI stress testing, not Simple.
The fix: Map your test objective to the right strategy before touching any configuration. If you’re stress testing an API, use Burst. If you’re looking for the thread count where error rates start climbing, use Thread. Using Simple for everything is the single most common SoapUI load testing strategy mistake.
Also Read: Cloud Based Performance Testing
3. Misreading TPS Metrics Due to Calculation Mode
This one catches experienced testers off guard. In SoapUI, TPS (Transactions Per Second) can be calculated in two different ways, and the default calculation for the Simple Strategy is misleading.
By default, TPS is calculated as (1000/avg) * threadcount. When you’ve introduced a test delay (say 10,000ms), this formula produces an artificially low avg, which then skews TPS upward — making your service look faster than it actually is.
The more accurate method is calculating TPS based on actual time passed: time-passed / count. This counts how many requests actually completed in the real elapsed time, regardless of delays.
The fix: In the LoadTest Options dialog, switch TPS calculation to “time passed” mode when using delay-heavy strategies. Also, note that when thread count changes mid-test (as it does in Thread and Variance strategies), results from prior thread counts will skew the statistics for the new count. Be aware of when thread transitions happen and interpret the statistics table accordingly.
4. JVM Heap Memory Errors Crashing Long Runs
SoapUI runs on the JVM, and the default heap allocation is often 512MB. For enterprise-grade SoapUI API testing, especially with large WSDLs, deeply nested schemas, or data-driven tests running thousands of requests, this ceiling gets hit fast.
The symptom is a java.lang.OutOfMemoryError: Java heap space error, often mid-execution. What makes it worse is that SoapUI automatically stores every request-response pair in memory so it can be reviewed in the TestCase log. On a data-driven run cycling through 40+ data sets across 20 test cases, memory fills up quickly regardless of how much RAM your machine has.
The fix:
- Navigate to [SoapUI install directory]/bin/SoapUI-x.x.x.vmoptions and increase the -Xmx value. A common working configuration for load-heavy projects is -Xms512m -Xmx2048m (or higher for very large suites).
- In the TestCase Log Options toolbar, limit how many entries are retained in memory. Reduce log retention to a few hundred entries for long-running data-driven tests.
- For CLI-based test runner executions, set JVM options via JAVA_OPTS environment variable rather than modifying the .vmoptions file directly.
- Avoid running tests in GUI mode simultaneously with editing the project. Parallel operations consume additional threads and accelerate memory exhaustion.
5. Not Setting LoadTest Assertions, Then Having Nothing to Enforce
Running a SoapUI load test without assertions is like running a race with no finish line. You generate data, but nothing tells you whether you passed or failed. Many teams collect statistics and then manually scan for problems, which doesn’t scale and misses subtle degradations.
SoapUI supports four LoadTest assertion types, each serving a distinct purpose:
- Step TPS – Fails if a test step doesn’t hit the minimum required transactions per second. Configure “Minimum Requests” to skip the initial warm-up period before the assertion kicks in.
- Step Maximum – Fails if any individual response exceeds a configured time threshold in milliseconds. A common starting point is 1000ms (1 second).
- Step Status – Fails if the underlying test step returns an error status.
- Max Errors – Fails if the total number of assertion violations exceeds an absolute or relative threshold. Setting this to a value like 1% relative errors gives you a pass/fail gate for the entire run.
The fix: Add at least a Step Maximum assertion and a Max Errors assertion to every LoadTest before running. These act as an automated safety net and terminate the test if results breach thresholds.
Also Read: Selenium Test Automation Services
Putting It All Together: A Practical SoapUI Load Testing Strategy
Before running any SoapUI performance test, run through this checklist:
- Define the test objective — baseline, load, stress, or soak — and pick the matching strategy.
- Set JVM heap to at least 2GB for non-trivial projects.
- Run in a production-like environment with SoapUI on a separate client machine.
- Start with a Simple Strategy baseline at low thread count.
- Configure Thread Startup Delay to prevent artificial spikes.
- Add Step Maximum and Max Errors assertions before running.
- Monitor all four metrics in real time, with statistics resolution tuned to your strategy.
- Switch to CLI mode for runs longer than a few minutes.
- Document thread count at which TPS peaks and error rates climb — this is your capacity data.
- Run a recovery baseline immediately after stress tests.
Struggling to get actionable results from your SoapUI performance testing?
Helixbeat helps engineering and QA teams move beyond guesswork. Whether you’re dealing with memory crashes mid-run, misread TPS metrics, or a SoapUI load testing strategy that isn’t scaling, our experts know exactly where the gaps are and how to close them.
From SoapUI API testing setup to full-scale SoapUI stress testing execution, we bring structure, precision, and real performance insight to every engagement.
Stop collecting data that doesn’t tell you anything. Talk to a Helixbeat expert today!
FAQs
1. What is SoapUI performance testing, and how is it different from functional testing?
SoapUI performance testing measures how a web service or API behaves under concurrent load — focusing on response time, throughput, and error rates. Functional testing (also done in SoapUI) validates whether a service returns correct responses. Performance testing runs those same functional test cases under simulated multi-user load to evaluate stability and speed, not just correctness.
2. What are the four load strategies in SoapUI and when should I use each?
SoapUI offers Simple (baseline and soak testing), Thread (linearly scaling concurrency to find capacity thresholds), Variance (sawtooth thread fluctuation for real-world traffic simulation), and Burst (short high-concurrency bursts for stress testing and recovery measurement). Match the strategy to the test objective — not the other way around.
3. How do I fix OutOfMemoryError in SoapUI during long load test runs?
Edit the -Xmx value in [SoapUI install dir]/bin/SoapUI-x.x.x.vmoptions to increase the JVM heap — -Xmx2048m is a common starting point. Also, reduce TestCase log retention via the log options toolbar, and disable response storage for data-driven tests where replaying individual requests isn’t needed.
4. Why does SoapUI TPS look unusually high when I use the Simple Strategy with a delay?
The default TPS calculation formula — (1000/avg) * threadcount — uses the average response time, which is artificially skewed by the test delay you’ve configured. Switch to the “time passed” TPS calculation mode in LoadTest Options for accurate throughput measurement that reflects actual completed requests per second.
5. What is the difference between SoapUI load testing and SoapUI stress testing?
SoapUI load testing validates that a service performs within acceptable thresholds under expected concurrent user volumes. SoapUI stress testing intentionally exceeds that capacity to find the breaking point and measure recovery behavior. They require different load strategies, thread configurations, and assertion thresholds.
