The Future of Test Automation Must Be Intelligent

A decade of Agile, DevOps and Continuous Testing is behind us. I heard this week in a conference I was presenting the phrase “Agile is Dead” and other claims about testing being inefficient in many ways.

While many tools have evolved over the past years, and test automation practices and lessons learned have been communicated to practitioners, it is IMHO not enough to move the needle on the efficiency of test automation and continuous testing – especially in the demanding and ever changing digital space.

To have a better DevOps pipeline, and to minimize both the waste as well as the level of escaped defects, testing and especially test automation must become smarter.

In this post I would list the opportunities for test automation to be more intelligent.

WHAT?

Test automation intelligence should guide engineers on which test automation scenarios to execute, automate if missing, and continuously maintain. This is one of the most complex debates within each automation team – WHAT TO AUTOMATE?

Answering this question is never easy, requires risk management, tradeoffs, and often guessing. In addition, it does not always align with the sprint time frames, hence, being thrown out of the software iteration.

With more intelligent solutions that are based on production data, business driven decision making, code analysis, teams will be in a better position to make such decisions.

I love the below visual that Ingo Philipp and originally, Michal Bolton drew around testing is always sampling. This shows that test automation usually addresses what we know about risks and the product out of the entire ocean of risks and features that we either don’t know, cannot assess, or things that were addressed in the past and there isn’t sufficient data to advise whether we should continue using.

New tools that are rising such as Launchable, SeaLights and others aim to help answer some of the above questions, and to help form a continuously relevant regression suite that matches the most recent product code changes, history of defects and more.

WHEN?

The second question that needs to be answered in a more intelligent way is – When to automate and execute my test scenarios? Some say – everything , everyday, some say – everything shifts left, and some are breaking the tests based on personas, skills, and naïve objectives.

Scheduling and deciding which tests to run when must be smarter, and needs to be tied to the overall process, the feedback loops, and the ability of the developers to act upon such execution feedback. If testers throws on the developers to late or to early the results, it makes it unrealistic for them to prioritize, resolve and fix.

Intelligent systems, must be able to determine based on the above WHAT section, which test scenarios, types, and cadence, such should be executed. In an ideal world, an intelligent system would be able to split between CI, BAT (build acceptance testing), integration, regression, NFT and production synthetic monitoring the entire test suites.

As the above diagram suggest, the pipeline has a pre-defined set of quality gates, phases, and milestones to manage and mitigate risks. An intelligent testing system should effectively place all the right test scenarios –CONTINUOUSLY– in the right phases. Keep in mind, the right test cases change from one release to the next.

WHICH?

Now that WHAT and WHEN questions are addressed, it is important to have a smarter approach into test automation coverage – on WHICH platforms and environments should my test automation scenarios be running constantly? Which mobile devices, which desktop browsers? which backend configurations are important? which data-driven scenarios should be plugged into the test cases? Which load and KPIs should be applied so I get the proper user experience feedback from my testing? and more.

Determining the WHICH in test automation is an art, and require data analytics, market research, customers validation, production monitoring, AIOps systems and more, but it is an imperative for success.

WHY?

After determining few WHs questions in test automation above, it is time for the analysis part of test automation intelligence – We defined what to automate, when to run, and on which platforms and environments, but once executing the tests, can we efficiently determine WHY tests are failing? What is the root cause per each failure so we lower the time for resolution (MTTR)?

Here, intelligent reporting platforms are critical in minimizing the feedback loop, the future decision making on which tests are of higher value, which platforms are more flaky then others and much more.

Without a solid test automation analysis platform, a great chunk of teams investment in testing can be of waste.

It is not that complicated to realize why tests are constantly failing, and put them into buckets of failure reasons. Perfecto is doing that in an automated fashion through its smart reporting SDK, and allows teams to filter a lot of the test noise post a regression cycle completion.

HOW?

Test creation has always been one of the hardest and most challenging task for SDETs and business testers. It includes great technical skills, meeting short time windows for automation, and requires deep understanding of the product requirements to really automate what’s expected. While many test automation frameworks evolved over the past years, Including Selenium, Appium, Espresso, XCUITest, and Cypress, these are all code based frameworks that even when coming on top of a BDD platform, they are hard to maintain over time, and analyze when they result in a failure.

New and more intelligent test automation frameworks that are codeless or low-code have grown and are becoming a new reality of test engineers. Such tools can really help address few key challenges that are prominent. Time to automate, time to maintain, flakiness and skillset gap. Intelligent test creation that is plugged with a TIA can be the future of smarter test automation, and enable practitioners to create the most relevant test scenarios with minimal maintenance and shorter amount of time. As I am writing this blog, such tools are still growing and have a lot to prove to the world prior to being widely adopted, but this in my mind should be the future of test automation, and once this is in place, connecting these tools to the cloud for parallel and scaled testing will be the ultimate solution for continuous testing.

BOTTOM LINE

A lot of what is written in this blog already exists to some extent, some if being developed these days and will be available in 2021. It is important to start drawing within each DevOps team the future of test automation goals and objectives and learn how such solutions will fit the process from test automation creation, execution, maintenance, analysis, TIA, defect management, quality governance and more.

This year, I launched my 3rd book that focuses on “Accelerating Software Quality: AI and ML in the Age of DevOps“. This book looks at the modern DevOps and explores how using AI and ML and smarter algorithms, teams can maximize their productivity and deliver more quality functionality faster.

There is a lot to get excited for in 2021, and especially around test automation in DevOps – Happy New Year and Good Luck in your Future of Test Automation

The Role of Artificial Intelligence in E-Commerce Industry

A Guest Blog Post by Ravindra Savaram

When we think about artificial intelligence(AI), the first thing that comes to our mind is a self-driving vehicle or a Terminator-like robot. Both robots and AI are not exactly one and the same. Though often utilized together with bots, artificial intelligence particularly refers to the stimulation of human intelligence processes by machines. AI powers many technologies that we utilize on a daily basis.

Whether AI is something that you have been monitoring for a while or it’s something that you have just come across, it is undeniable that AI is beginning to influence many industries. One place where it is really changing things is e-commerce. From creating personal buying assistants to personalizing the shopping experience, artificial intelligence is something that retailers cannot ignore.

Many areas of e-commerce are ripe for innovation driven by artificial intelligence. Every enhancement to logistics efficiency, recommendations, pricing, or marketing provides retailers an edge over the competition. Retail creates and consumes large volumes of data from various channels. In fact, there is so much data that it’s not possible for a human being to analyze it. These are the ideal conditions for machine learning.

For various data analysis methods, machine learning is the overarching name. In these methods, the computers get insights in data without actually being told where to look for the insights. When exposed a large amount of data, machine learning algorithms can extract patterns and utilize them to generate predictions or insights about the future conditions.

When you upload a cat picture to cat Google Photos, it knows that the object in the picture is a cat. The code that identifies the cat is not written by a human but it is developed as a result of exposing the algorithm to a large number of cat photos(also, the photos of things that are not a cat).

Recommendations

The same principle explained above can be put to use in many e-commerce areas. For instance, the retailers have become really good at recommending products that are related, but the people who do online shopping knows that the recommendation engines get it wrong very frequently. The recommendation engines are quite limited as they can have access to only a small set of data and the ways they can reason about that data are restricted. Machine learning helps merchants find much better ways of modeling the behavior of users so they can make close to exact recommendations about what a customer is interested in buying. With machine learning, the AI can make predictions based on past data. The predictions include what customers will buy next, their typical price threshold, their preferred device and channel, and so on.

Pricing

Today, the online retail industry is constantly presenting new challenges to COOs and CMOs when it comes to pricing. There is a fierce competition among the e-commerce brands of all sizes and guises. Even for an online merchant for a 1000 product list, somewhat tweaking in manual price can become a task that is almost impossible to accomplish. The environment is changing constantly – rival prices, logistics, currency conversions, and delivery rates are just a small sample of numbers or circumstances prone to change continuously.

The tweaking of prices in real time can be accomplished with artificial intelligence depending on multiple data sets including stock levels, resource capacity, internal operations, customer demand and behavior, and market conditions.

High-level of Assistance

The personal shopping assistants were a luxury of the rich or famous once upon a time. Artificial Intelligence has shaken up this scenario and in the process, revolutionized e-commerce. This conversational and intelligent technology has extended to customer service as well. The chatbots and personal digital shopping assistants can suggest the best available products to new visitors in a manner similar to humans, recommend new deals to your returning customers, answer the queries of a customer and provide suggestions, and alert customers when products they may prefer to purchase come into stock or change in price.

Conclusion

By merging intelligent neural networks with massive data sets, the applications of artificial intelligence will help e-commerce companies to build unparalleled competitiveness in the market. The impact of Personalized Merchandising supported by artificial intelligence on the e-commerce industry will continue to rise in the coming years. They not only optimize or automate current processes but also help retailers to avoid common pitfalls of manual approaches, giving customers an enriched experience to maximize profits.

About the Author:

Savaram Ravindra was born and raised in Hyderabad, popularly known as the ‘City of Pearls’. He is presently working as a Content Contributor at Mindmajix.comHis previous professional experience includes Programmer Analyst at Cognizant Technology Solutions. He holds a Masters degree in Nanotechnology from VIT University. He can be contacted atsavaramravindra4@gmail.com. Connect with him also on LinkedIn and Twitter.

Create a website or blog at WordPress.com

Up ↑