When I speak to QA Managers in my daily routine, I often hear a common scenario, they own LoadRunner as their performance load testing solution, but they have been told by “C” level executives within their organization to replace it with a “free” open source solution like JMeter. This is almost always due to the excessive price of LoadRunner’s annual renewal, coupled with the fact that every year it becomes more and more outdated and less relevant to the complex demands of today’s load testers.
LoadRunner or JMeter?
As LoadRunner’s deficiencies become more apparent the justification for keeping it becomes harder to justify price-conscious IT organizations, and rightly so.
Here are the five major deficiencies LoadRunner customers tell me frustrates them:
Windows only based architecture that lack support Apple and Linux OS’s.
Not DevOps friendly – limited or awkward support DevOps tools like Jenkins, JIRA, Bamboo, CodePipeline etc.
API Support – APIs can only be created manually by complex & time-consuming programming.
Server deployed – LoadRunner’s Lacks real-time test collaboration and unlimited scalability you see with SaaS-based load testing solutions.
Given the high cost and lack of innovation of LoadRunner, it’s no wonder companies decide to make a 180-degree turn to open source tools. However, in many cases, this choice is moving from a difficult situation to a much worse one.
JMeter brings its own set of challenges as a load testing solution:
Like LoadRunner, it too requires users to know complex programming to create scripts.
Open source tools cannot offer tech support, so you need to search the internet for answers which may or may not be helpful and can be time-consuming.
It’s not secure. How many trojan horses have hacker left in the open source code? I don’t know and neither does anyone else. Your security officer will probably not be happy with an open source tool either.
Also like LoadRunner it does not scale well when pushed to today’s large or even mega tests, that many organizations now require.
Free? – There are many costs associated with open source software. The costs associated with the lack of the ongoing maintenance and support as well as the creating news scripts– not to mention the security risk.
As a sales professional, I use Salesforce. I use it because it is simply the best CRM solution on the market in my experience. Yes, it cost my company money, and yes there are many open source tools that I could use. But as a professional, I have decided to give myself the best chance at success. My management is not going to congratulate me if I miss my sales goals but saved the company money by using an open source CRM. That’s not how the world works.
LoadRunner or JMeter is a false choice. There is a third alternative, Apica LoadTest.
Apica LoadTest offers a modern alternative with modern capabilities:
Advanced Scripting Engine – Create realistic and complex load test scripts without programming, plus it runs Selenium, LoadRunner, and JMeter scripts.
OS Support – Windows, Linux, and Apple.
Modern deployment options – SaaS, On-Premise, or Hybrid.
Unmatched scalability – Up to Tens of million concurrent virtual users.
API Test – Chaining together complex API calls without programming.
Agile DevOps support – Jenkins, Bamboo, Git, AWS Team City, CodePipeline and more.
Native APM Integrations – AppDynamics, Dynatrace, New Relic.
Multi-Team Support – Real-time, centralized, better collaboration and superior testing.
24/7/365 GLOBAL SUPPORT – All performed in-house.
Secure – Will exceed your security team’s assessments.
Pricing – All protocols included with unlimited usage – no hidden costs at a fraction of what you are paying for LoadRunner.
If you think you are paying too much for LoadRunner you probably are. Apica LoadTest offers an interesting choice besides LoadRunner or JMeter.