Just before the pandemic changed our lives with the new working from home experience, we had the opportunity to meet with our customer advisory board (CAB). We discussed what is top of mind for them, what has been working and what is next for Apica. This was a great opportunity to gain deeper insights from customers working directly with our platform in the field. This year was a very important CAB meeting as there were a lot of unknowns at the time, and we wanted to see firsthand what was impacting our enterprise customers.
One of the best conversations from the event was a discussion on the five challenges enterprises need to know about testing and monitoring. The conversation included topics like scalability, automation and compliance. One thing that stood out from the conversation was the increasing demand on applications and services, especially as we were anticipating what the pandemic might do to the working environment. Below are the five challenges we discussed regarding testing and monitoring.
Scalability
One topic at the forefront of the attendee’s minds was scalability. They had mentioned that writing a small piece of code to check the availability of a single resource was relatively simple but doing this at scale proved to be very difficult. As we began to dissect the need for scalability, it was interesting to see that several discussion threads appeared including:
- The breadth of availability and performance monitoring across all key workflows, applications and resources, internally and externally.
- How to scale and manage synthetic monitoring across multiple business units for both “self-serve” and effectively managing internal bill back models for a centralized service offering.
- Automation leveraging tools such as ServiceNow, PagerDuty, AppD and more to create automated checks with the click of a button. Allowing stakeholders quick time to value without consuming valuable resources.
- Finally, scalability regarding effectively consuming results from testing. When looking at a potential 10,000+ scripts, each running many times per hour, having an effective way to look at this data in a meaningful and actionable way is a challenge within itself. AIOps is a possible solution that Apica is beginning to develop as organization’s infrastructures continue to grow.
With the new release of Apica 2.0, many of these topics had been discussed with our product team and top customers already. The affirmation from the larger group was reassuring that our investments in scalability are what our customers want.
Automation Means Shifting Left
We have heard and continue to hear from enterprise DevOps teams that they want their testing and monitoring to “Shift Left.” This prevents issues early in the application development process. Additionally, many operational groups would like to be able to leverage functional testing scripts into production, saving hundreds, and potentially 1000’s of hours of rescripting time annually. By integrating with common CI/CD tools, such as Jenkins, Bamboo and TeamCity to run small performance tests with a handful of users, performance becomes an attribute of the release cycle. Organizations can also save time and money by continuing to leverage existing assets like JMeter scripts to streamline the process.
Monitoring Costs
Do more with less has become not just a motto, but a reality of operating in competitive environments with demanding stakeholders. With unlimited funds and resources, monitoring is not overly challenging. In fact, one member of the advisory board mentioned that they have 98 vendors they work with under application monitoring. However, that is not the reality in which most organizations live. That same gentleman even said he is being tasked with bringing that number from 98 to 50 within the next year.
Organizations are getting better at understanding monitoring costs beyond just licensing, including a numbers of soft costs that are rarely considered. A few examples from the meeting were, “monitoring the monitoring”, creating separate workflows for internal web, vs external web applications, vs non-web applications and even system resources. Managing hardware costs for internal monitoring and the appropriate management of that hardware was also discussed.
Interestingly enough, costs and scalability started to merge at least in concept on many of the specifics. There was some excitement on Apica’s concepts on reducing the overall costs of monitoring by utilizing containers for test executions. This would reduce companies’ monitoring footprint, in some cases by upwards of 80%. Additionally, offering a “bring your own cloud” managed services is something we are very excited to offer!
Compliance and Security
With our Advisory Board primarily made up of organizations from the largest Banks, Financial Services and Insurance companies, security and compliance are always a very hot topic. These conversations revolved around 3 common themes:
- Monitoring Applications through secure applications via SSO, MFA or a PIV smart card to ensure these mission-critical applications are up and available.
- Securing required credentials to access applications in either commercially available tools such as CyberArk, or proprietary libraries secured on-prem.
- Ensuring proper access and secured storage of the scripts, endpoints and user access credentials.
While there were similarities between the core themes for each of the organizations, the reality continues to be that each organization will have its own requirements. While our dev team thrives on customizing these solutions based on our customer’s needs, our product team still strives to figure out how to streamline this process. Fortunately, we have all the Lego pieces required for many organizations and will continue to adapt to our customers’ needs.
Better Metrics
We all know that getting the right metrics can be a struggle sometimes, even when monitoring and reporting have been set up correctly. The challenge facing many organizations is whether they have enough visibility into the data to extrapolate actionable intelligence. Compound this with the massive growth of data and complexity, it is critical to deploy a modern testing platform that provides more robust reporting to analyze all that data. Additionally, the reporting platform should allow you to create reports on the fly and deliver them on a daily, weekly or monthly basis as needed.
The meeting with the CAB was an excellent opportunity to talk about the market and how we can help solve the challenges above. We have seen that end users are becoming more and more fickle about accessing applications they need and want instantly. The current use of applications has skyrocketed as we are required to work remotely. The real challenge is that expectations have increased from global consumers, regardless of their location and the underlying infrastructure. It’s no wonder that 75% of online users expect a greater level of application performance than they did just a year ago. As enterprises look to simplify and improve the customer experience, the infrastructure needs and performance becomes far more complicated. See what Apica’s platform can do to take the complexity, cost and stress out of your performance testing services processes by scheduling a demo.
