AIFutureTestingeGuideTechWell 0 PDF

F U T URE O
H E F
T TE
D S
N
TI
A
NG
AI
eGUIDE
3
AI AND THE FUTURE OF TESTING
eGuide
5 Things That Will Impact At today’s rapid pace of software delivery, manual testing alone is simply inadequate. Artificial intelligence will soon be a necessity for
the Future of Software testers to keep up—and in many ways, that reality is already here. But what do these AI capabilities look like, and how can they augment
Testing and benefit our test activities? This eGuide rounds up a collection of resources that explain what truly qualifies as AI, the different types
of AI, and how quickly AI is advancing. It’s time to embrace the future of testing.
7
Why Testers Should Take
Control of the AI Narrative
In this AI and the Future of Testing eGuide
11
5 Things That Will Impact the Future of Software Testing The AI Testing Singularity
What’s Our Job When the
From the way we look at software, evaluate risks, think about com- Machine learning is rapidly growing more powerful, already some-
Machines Do Testing?
plexity, design our test approach and strategy, and help to release times imitating the actions and judgments of humans better than
a stable product to the customer, technology has had an influence humans. In the near future, even before machines are conscious, they
12 on how we test software. And that influence will only continue as will be able to mimic human software testers. What will be the impact
Leveraging Machine technology advances. On a high level, here are five key things we’re of AI on testing? Jason Arbon has a bunch of ideas.
Learning to Predict Test already seeing that are going to shape the future of software testing.
Coverage 4 Advantages of Applying AI in Software Testing
Why Testers Should Take Control of the AI Narrative: We’re always looking for smarter, faster, better ways of testing. As
14 An Interview with Tariq King and Jason Arbon the popularity of artificial intelligence grows, more and more testers
The AI Testing Singularity In this interview, Tariq King, the senior director and engineering are realizing its capacity to make cumbersome and time-consuming
fellow for quality and performance at Ultimate Software, and Jason tasks simpler. AI is coming, so we should take advantage of it. Here
15 Arbon, the CEO of test.ai, explain the role artificial intelligence plays in
modern testing and why you should establish a foundation right now.
are four benefits to applying AI in testing.
4 Advantages of Applying Testing the Ethics of AI

AI in Software Testing What’s Our Job When the Machines Do Testing? AI is a double-edged sword. When it’s being used in situations involv-
It’s a safe bet that testing jobs won’t be taken over by machines ing sensitive personal data, such as health care, banking and finance,
16 anytime soon. However, those of us in the test industry would be wise and real estate, security is of the utmost importance—and so are
Testing the Ethics of AI to heed cross-industry applications of analytics and machine learning ethical implications. It’s up to testers to mitigate risks and make sure
and begin staking out the proper role of the machine in our testing AI is used responsibly.
17 domain. What could AI mean for testing?
Insight from the Industry
Leveraging Machine Learning to Predict Test Coverage
Test coverage is an important metric within test management, and as Additional Resources
18 technology evolves, we‘re able to leverage new trends to predict cover-
Additional Resources age. Weka, an open source suite of machine learning software, can take
your test management beyond spreadsheets to the latest AI technolo-
gies, letting you predict your test coverage earlier with greater accuracy.
COPYRIGHT 2019 2
3
eGuide
5 Things That Will Impact
the Future of Software
5 Things That Will Impact the Future
of Software Testing
Testing
7
Why Testers Should Take By Raj Subramanian
In the past few decades, software testing has evolved in terms of
11 both the tools used to perform different activities and the mindset
What’s Our Job When the of the people using those tools. There used to be only a handful of
Machines Do Testing? tools used in software testing, but now we have a plethora of tools
to choose from, from proprietary to open source. Likewise, people
12 have started thinking of testers as information brokers instead of
gatekeepers, and there have been a lot of positive developments in
Leveraging Machine
the agile world that have contributed to significant changes in the
Learning to Predict Test
processes that teams follow in their software development lifecycle.
Coverage
Advancements in technology are to thank for these evolutions.
14 From the way we look at software, evaluate risks, think about com-
The AI Testing Singularity plexity, design our test approach and strategy, and help to release
a stable product to the customer, technology has definitely had
15 an influence on how we test software, and that influence will only
4 Advantages of Applying continue as technology advances. On a high level, here are five key tional testing, end-to-end testing, and regression testing. They are
AI in Software Testing things we’re already seeing that are going to shape the future of primarily concentrated in UI-based test automation—the more tests
software testing. the user creates, the smarter the algorithm becomes, which makes
16 the tests more stable.
1. Artificial Intelligence
Testing the Ethics of AI About five years ago, everyone was talking about “mobile first” and Here are some of the benefits we could expect to start seeing in
giving the user a mobile experience using mobile web, native, and testing thanks to AI:
17 hybrid applications. Now, the new buzzword is AI. It is in self-driving • Easier authoring of tests
Insight from the Industry cars, home assistants (people sure do love their Alexa), computer • Lower maintenance work on test scripts
vision, health care, finance, and now in software testing. • Fewer flaky tests
18 • Having nontechnical people start doing automation
Currently, there are very few reliable tools in the market that use • Easier CI/CD integration
Additional Resources
machine learning to help in the authoring and execution of func- • More reusable tests
COPYRIGHT 2019 3
3
eGuide
For example, I built an automation framework with Cucumber, Java, • A need to start automation right at the beginning of the SDLC and
and Appium. Although I had a robust framework and a lot of flexibili- ensure nearly all test cases are automated
Testing ty in writing custom code to perform various actions, I often suf- • All the QA tasks would need to be aligned to ensure a smooth CI/
fered with the common problem of maintenance. When a developer CD cycle
7 changed attributes of an element that were already covered by my

automated tests, the tests started failing. As a result, I spent a lot of
• A high level of collaboration would be needed to ensure there is
continuous monitoring in the production environment
Why Testers Should Take time just maintaining these tests instead of writing new automation • All the QA environments would need to be standardized
Control of the AI Narrative code to cover new functionalities that were implemented. • The testing mindset changes from “completed testing on this
module” to “What are the business risks that have been mitigated
11 This problem can now be solved by using dynamic locators the AI in the release candidate?”
What’s Our Job When the extracts from the Document Object Model (DOM). In real time, the AI
Machines Do Testing? analyzes all the object trees and properties from the DOM and can The key to all the above changes is automation. DevOps and auto-
create a list of different attributes for a particular element. So, when mation go hand in hand—without one, the other won’t work. This is
12 an attribute of an element changes, the AI tries to go to the next where smart people and tools can help in bringing shorter and more
attribute in the list to locate the element and keeps going through dependable release cycles.
Leveraging Machine
the list until the element is located. The tests are more stable, the
authoring and execution of tests are much faster, and the tester has I worked at a company where there was minimum collaboration
Coverage
to spend less time on maintenance. between the development, testing, and operations teams. We were
14 2. DevOps
seeing a lot of gaps in the SDLC cycle in terms of more bugs getting
into production, unstable CI/CD infrastructure, and less visibility into
The AI Testing Singularity DevOps has helped software development teams and operation production monitoring and statistics. Noticing these gaps, the team
teams to better collaborate, thereby ensuring there is constant decided to implement a DevOps practice, and everyone started col-
15 automation and monitoring throughout the software development laborating and contributing in each phase of the SDLC. This started
4 Advantages of Applying lifecycle (SDLC), which includes infrastructure management as well. from requirements gathering and extended all the way to produc-
AI in Software Testing tion release and monitoring.
You may ask, how is this going to influence software testing? The
16 answer: Everything we do as part of testing is going to change. The
changes I foresee include:
This increased culture of collaboration started having positive effects
in team morale, more automation coming into place, and the whole
Testing the Ethics of AI team starting working together as one unit.
17
Insight from the Industry Technology has definitely had an influence on
18 how we test software, and that influence will
only continue as technology advances.
COPYRIGHT 2019 4
3
eGuide
It’s important to change our mindsets about
Testing
how we view systems and to test them
7 accordingly. We can choose to either ignore it
or embrace it. What will you do?
11
Machines Do Testing? 3. QA as a Service In the future, QaaS providers are going to think about more ways to
Just like how we have SaaS (software as a service), IaaS (infrastruc- improve their offerings to stay ahead of their competitors, which will
12 ture as a service) and PaaS (platform as a service), we now have benefit software testers, too.
QAaaS. Over the past few years, this has become a popular way for
Leveraging Machine
companies to meet their software testing needs. 4. The Internet of Things
With the advent of wearables, smart homes, connected cars, and
Coverage
Companies that have QAaaS solutions make different aspects of your other cloud-based technologies, the internet of things (IoT) has
14 software testing process easier by providing:

• Test case management and maintenance solutions
become a big topic of discussion. The amazing thing about these
devices is that there are so many communications and integrations
The AI Testing Singularity • Test automation tools with less need for coding taking place every second.
• Robust test reporting features with logs, video recording, and
15 screenshots Let’s analyze, on a high level, the different communications tak-
4 Advantages of Applying • Easy integration with CI systems ing place with a wearable fitness tracker. First, the mobile app and
AI in Software Testing fitness tracker need to communicate with each other. The data
Resources such as mobile phones, virtual machines, secure net- captured by your mobile app should seamlessly integrate with the
16 works, and human testersIn the past seven years of doing automa-
tion, one big problem I have always run into is having to maintain
desktop, mobile web, and tablet versions of the app, and all these
communications across devices should happen in real time. All the
Testing the Ethics of AI my own server machines to run my automated tests. The server data travels to and from the cloud, the devices, and the apps. People
machines have different problems, like running out of storage space, also can form groups and compete with each other via the app, so
17 a flaky internet connection, slow processing speed for the number these calculations and communications need to happen in real time,
Insight from the Industry of tests that are being run continuously throughout the week, and too. Based on different events triggered, the proper notifications
the need to be frequently updated with the latest OS, build tools, need to be sent to the right user at the right time. All these commu-
18 security patches, IDEs, and so on. These kinds of problems could be nications happen over the internet.
solved with QaaS providers, as they can do all these activities for you,
so team members can concentrate on more critical tasks.
COPYRIGHT 2019 5
3
eGuide
Imagine you are a tester testing this fitness tracker. Where do you there were weird issues: When I minimized the app on the phone,
start? How would you design your test strategy and approach? the Apple Watch went blank, showing only a black screen; but when
Testing The IoT introduces its own level of complexity into software test- I opened the app on the phone again, the black screen disappeared,
ing. It is going to affect the way we think about testing, particularly and the Apple Watch app behaved normally.
7 because more concentration needs to be given tointegration testing

than the old approach of testing each component separately. This is a perfect example of why it is important to do integration
Why Testers Should Take For example, when I was working for a travel booking company, we testing. With more and more devices coming into the market, this is
Control of the AI Narrative came up with a new app for the Apple Watch built using WatchOS going to be critical for organizations and users.
(when it was first introduced by Apple). The application had limited
11 but useful functionalities, like the ability to view notifications and 5. Robots
What’s Our Job When the rewards information, reservations, and locations of hotels, flights, There are now robots that do testing. Some people may think this is
Machines Do Testing? and car rentals. While testing this application, I noticed that when scary in terms of job security, but I still believe that the human mind
the Apple Watch app was connected to the same app on my phone, can never be replaced. There would still need to be humans to mon-
12 itor the robots to ensure they are doing what is expected of them
and to actually program them to do things. How far is this scalable?
Leveraging Machine
Only time will tell.
Coverage
In summary, advancements in technology have already started
14 influencing the way we are doing software testing. It has also caused
companies to rethink their organizational structure: The QA team is
The AI Testing Singularity moving toward being embedded within the development team, and
the whole team will own quality. It also will become important for
15 the research and development group to be in frequent interaction
4 Advantages of Applying with the development group in order to make products smarter and
AI in Software Testing more useful for their customers.
16 There will also be a need to have procedures in place to handle large

amounts of data, as well as appropriate computing power to comb
Testing the Ethics of AI through this data to get useful information and feedback. Finally, to
make all this a reality, companies need to adopt lean processes and
17 be much more transparent to prevent being an obstacle for innova-
Insight from the Industry tion. Lean transformation will be vital for effective growth.
18 It’s important to change our mindsets about how we view systems

and to test them accordingly. We can choose to either ignore it or
embrace it. What will you do?
COPYRIGHT 2019 6
eGuide
Why Testers Should Take Control of
3
the AI Narrative: An Interview with Tariq King and Jason Arbon

Testing
7 By Jennifer Bonine
In this conversation, Tariq King, the senior director and engineering fellow for
11 quality and performance at Ultimate Software, and Jason Arbon, the CEO of
Machines Do Testing? test.ai, talk with Jennifer Bonine about the role artificial intelligence plays in
modern testing and why you should establish a foundation right now.
12
Leveraging Machine
Coverage So, when you think about it, artificial intelligence is really a hot topic these days. Whether you’re in the business of
building software or consuming software, it’s all over. For me, I think that one of the biggest things in terms of the
14 testing community is we’ve seen folks who are using AI for testing. I’ve seen folks who are kind of averse to it, like
I said, but there’s these different pockets of people. And I really wanted to just get everyone familiar with what AI
The AI Testing Singularity
means and what AI brings, and start to get people comfortable with thinking about, what does the future look like?
15 There’s, quite frankly, a lack of innovation when it comes to pushing that boundary, and testing seems to lie behind
4 Advantages of Applying other aspects of software engineering. And I just wanted to try to open up people’s minds to say, hey, look. Let’s not
AI in Software Testing think that these things are impossible and that they’re far away. They’re actually here and they’re coming. And as a
community, we have to get in front of it.
16 Tariq King
And so, my message was really around, let’s stop and think about what we can do to leverage this technology to
Testing the Ethics of AI
make what we do better. But let’s also notice that in terms of, for AI to successful, as with any paradigm, testing has
17 to be a key part of that.
Insight from the Industry And so, the message was really to try to bring those two worlds together: Have people try to get comfortable with
thinking about, “Hey, if our future looks different, it’s okay.” Right? But then also, what ideas can we bring as tes-
18 ters? What things can we bring to this forum to kind of make our impact and make sure that at the end of the day,
Additional Resources AI doesn’t take over? We need testers to kind of help be that ground and that foundation.
COPYRIGHT 2019 7
3
eGuide
5 Things That Will Impact What we found is that some people are visionaries, right? Like, they see something before it comes, and you see it
the Future of Software coming, and having talked to you, I’ve seen this coming as well for a while. But it’s interesting to watch the curve of
Testing other folks starting to get involved, get on board and see that this is, it’s here. It’s here.
Jennifer
7 Bonine
Inevitable. You’ve seen it actually kind of evolve, right?
11
What’s Our Job When the Jason Arbon
12
I have.
Leveraging Machine
Jennifer
Coverage Bonine
14
The AI Testing Singularity Actually, you said “awhile,” it made me a little bit painful for a second because I’m like, I wish I could just code faster.
15 Jason Arbon
4 Advantages of Applying
AI in Software Testing
Yeah. But, it’s two things, I think, right? It’s not even just having the technology and the code, but it’s the adoption,
16 right? Can people—we have a lot of technology that’s ready today, but people aren’t ready for it. Right? So, it’s not
Testing the Ethics of AI even that it’s not we couldn’t have it; there’s lots of stuff out there we could have. But, people aren’t ready for it,
right? To absorb it, to accept it, to embrace it. We have to get them to that stage of being okay with it.
Jennifer
17 Bonine
18 Right. That’s perfectly put. You’re not supposed to upstage the people you’re interviewing. But that’s better said
Additional Resources than either of us had said it this week.
Jason Arbon
COPYRIGHT 2019 8
3
eGuide
I was just talking to one of the other folks, right before you guys, about the fact that technology and change comes
faster than ever before. Things are moving and changing. And if we bury our heads in the sand and pretend stuff
Testing
isn’t here, it’s not coming, or don’t get aware or educated, you get—that gap gets bigger and bigger, and the
7 opportunity gets wider spread. And people get more afraid, I think, sometimes, when they don’t get information. It
progresses, it gets more mature. They haven’t been involved. Jennifer
Why Testers Should Take Bonine
11
What’s Our Job When the The more they adapt, the more angst they have.
Jason Arbon
12
Leveraging Machine
Learning to Predict Test Yeah. And it just creates a bigger angst. And so, as stewards of information, it’s good that you’re creating opportu-
Coverage nities for education and awareness. Because I think a lot of times what people don’t realize is, change creates some
level of fear, and a lot of times it’s fear because they don’t know, right? It’s the unknown. So, get some information,
14 and that can really help a lot with that gap, of just creating information and having, since it is a small community,
people that have some good information, have been involved in it, know it’s here, what it’s doing, to educate others. Jennifer
The AI Testing Singularity Bonine
15
16 The really smart thing that’s gonna happen five to ten years from now is what Tariq was saying, and that is that sys-
Testing the Ethics of AI tems will eventually test themselves from the inside out, right? Rather than from the outside in.
17 And the testing I do is from the outside in, it’s very nontraditional, with swapping the brain out with machine learn-
ing. But his idea and vision is software that actually tests itself. And that’s the stuff that stands between us and the
Insight from the Industry Jason Arbon singularity and the robots taking over. So, I’m really excited to see where he takes that research.
18
COPYRIGHT 2019 9
3
eGuide
the Future of Software Yeah. There’s really a lot of research that started from when I did my Ph.D. That’s kind of the area that we looked
Testing at. But, that’s 2009; the time and the technology has really gone further. And so, for us, like right now we’re trying
to see with all the advances in technology, microservices, artificial intelligence—can we really design systems in the
7 way that they can test themselves?

Because, when you think about software development, yes, we do a lot of testing from the outside. But we also do a
lot of testing from the inside, and if we actually think about designing these systems with these features, then they Tariq King
can be a lot more thorough and actually get to a point where systems may not only be able to detect when they
11 have issues, but they may be able to fix themselves.
12
Leveraging Machine
Coverage AI is just to bring people together. Everyone’s curious about AI for testing. Some are concerned about it, some really
want to dive into it. The idea is to have one place where we can get together and talk about AI. ... Tariq and I were
14 talking about how there’s—we don’t get together very often, there’s very few of us working on the problem today.
We’re basically lonely people, we try to meet on the internet, and there’s a mailing list. We find things, we have pa-
pers, we run into sort of interesting issues—or we survey the community, too, to see what they think about AI. ...
Jason Arbon
15
Because, there’s, I think, a combination of people who are probably super excited about it, a smaller group. A group
16 that just doesn’t know much and really wants to get educated. And then a group that’s probably a little bit fearful
Testing the Ethics of AI of what that means. I’ve even heard some testers saying, “I just hope I retire before it happens.” ... And a lot of that
comes from lack, I think, of awareness and education. So, having places where people can get information, educa-
17 tion, awareness, is always good. Jennifer

Bonine
18
COPYRIGHT 2019 10
3
eGuide
What’s Our Job When the Machines
Do Testing?
Testing
7
Why Testers Should Take By Geoff Meyer
The application of analytics, artificial intelligence (AI), and machine Jason Arbon published a test autonomy maturity model, assigning a
11 learning are transforming jobs in industries once thought to be “safe” measure of an organization’s automation capabilities. L0 (manual test-
from automation. ing) is entirely in the mode of manual testing. L1 (scripted testing) has
a firmly entrenched culture of scripting regression suites. L2 and above
Law firms are using AI-based machines to do the research that used to (exploratory bots, as well as human-directed, generative, and fully au-
12 define the role of first-year lawyers. Trading decisions for our retire-
ment accounts are managed by algorithmic robots crunching through
tonomous testing) are for organizations that have started to move into
the realm of automating their cognitive and complex tasks.
Leveraging Machine massive amounts of historical data and real-time market metrics. The
Learning to Predict Test progress of the autonomous vehicle, underpinned by AI, is on the For an L1 test organization well-versed in a world of automation,
Coverage verge of replacing drivers across the transportation sector. What busi- whether it be scripting of regression tests or the automation of build
nesses in industry after industry have uncovered is this: Processes that and deployment processes, analytics and machine learning represent
14 have large amounts of data, a representative model, and a generally the next generation of automation. And although it’s just another
The AI Testing Singularity understood set of rules are candidates for automation. tool to be considered for our testing toolbox, it’s one that opens up
the possibility of automating tasks in our job that we may never have
15 Geoff Colvin, the author of Humans Are Underrated, points to the first-
year lawyers described earlier as a model for those of us whose job
imagined.
4 Advantages of Applying consists of analysis, subtle interpretation, strategizing, and persuasion. While many of the tasks across our diverse test practices are similar,
AI in Software Testing These jobs are gradually—sometimes entirely—being transformed as each of our jobs has unique challenges, so priority will be determined
increasing sets of tasks are delegated to a “smart assistant.” by the specifics of our organizational context. So your job, when the
16 machines can do testing, is to figure out what tasks you want them to
Testing the Ethics of AI Our job in software testing is also composed of many tasks. From the do for you.
nuances of requirements elaboration, the back-and-forth in establish-
17 ing “expected behavior,” test case analysis, and the other numerous Whether your high-priority pain points include triaging a bounty of
cognitive tasks that we deal with, it’s a safe bet that our jobs won’t automated test failures, avoiding wasted effort researching a defect
be taken over by machines anytime soon. However, for the reasons only to find it is a duplicate, rapidly predicting the root cause of a fail-
18 cited by Colvin, those of us in the test industry would be wise to heed

cross-industry applications of analytics and machine learning and be-
ure, deciding whether to automate or retire a test case, or increasing
regression test coverage in a time-constrained period, there’s a good
Additional Resources gin staking out the proper role of the machine in our testing domain. chance that employing a machine as your smart assistant might be the
next automation approach for you to consider.
COPYRIGHT 2019 11
3
eGuide
Leveraging Machine Learning to
Predict Test Coverage
Testing
7
Why Testers Should Take By Bhavani Ramasubbu
Testers should be involved in the requirements collection phase of the
11 software development lifecycle because it benefits both the QA and
What’s Our Job When the business teams to understand the requirements better. In test man-
Machines Do Testing? agement, we analyze those requirements, prepare test cases, execute
test cases, do bug tracking, and get QA to sign off on test coverage.
12
At my company, a digital commerce agency, we work on e-com-
Leveraging Machine
merce technologies and serve enterprise clients. Test coverage is an
essential metric to measure the quality delivery of our projects. Our
Coverage
test management was initially handled in spreadsheets, but multiple
14 versions of different files ended up underused. Maintaining these

spreadsheets and extracting value became complex, and the data
The AI Testing Singularity was quickly obsolete.
15 Next, we tried a shared Google spreadsheet to work in a collabora-

4 Advantages of Applying tive environment. However, this led to work products being acciden-
AI in Software Testing tally modified or deleted. Preparing reports and presenting projects
to stakeholders consumed more and more of the test leads’ time. were still spending too much time having to manually combine the
16 We decided to explore the world of test management tools. These
predictions made using the reports from these test management
tools with statistics. We had to find another technology to monitor
Testing the Ethics of AI tools have evolved over the years, and they resolved some of our and track the progress of the project’s delivery timeline.
problems with features like a centralized repository of test work
17 products, role-based access, reuse of work products, monitoring We discovered Weka: the Waikato Environment for Knowledge Anal-
Insight from the Industry test activities, and tailor-made reports. They helped us maintain best ysis. Weka is an open source collection of machine learning algo-
practices and helped our test leaders manage more effectively at a rithms for data mining tasks developed by the University of Waikato,
18 higher level. New Zealand. It is available with an easy GUI, and the library is
simple to learn compared to other tools for a beginner in machine
However, they still lacked the ability to predict tasks such as test esti- learning and AI. The algorithms can either be applied directly to a
mation, test coverage of a release, and quality issues. Test managers dataset or called from our own Java code.
COPYRIGHT 2019 12
3
eGuide
There are four main tools in the Weka GUI Chooser:
the Future of Software • Explorer is the primary graphical user interface that
Testing gives you access to most of the functionality
• Knowledge Flow allows you to process, view, and
7 visualize your data

• The Experimenter helps you answer questions, like
Why Testers Should Take whether one classifier is better than another on a
Control of the AI Narrative particular dataset
• The Workbench is the unified UI for Weka
11
What’s Our Job When the Weka has multiple options for generating predictions
Machines Do Testing? of any kind of data. Here’s how we use its libraries to
predict test coverage.
12
Test Coverage Prediction in Weka
Leveraging Machine
The Weka Explorer is an easy-to-use GUI that harness-
es the power of the Weka machine learning software.
Coverage
Each of the major Weka packages—Filters, Classifiers,
14 Clusterers, Associations, and Attribute Selection—is rep-

resented in the Explorer, along with a Visualization tool
The AI Testing Singularity that allows datasets and the predictions of Classifiers
and Clusterers to be visualized in two dimensions. At this point we save this model to reuse, which saves time. When
15 the existing saved model is readily available, we load it and then
4 Advantages of Applying The flow of test coverage prediction in Weka Explorer looks like the upload the .ARFF test set file to be predicted.
AI in Software Testing snapshot on the right.
We can load saved models in Weka and reuse them with new data-
16 We have historical data of test coverage metrics in Excel spread- sets, informing updated predictions. It helps to present the data to
the stakeholders of the project earlier in the schedule, and we also
sheets. For evaluation of the data, we use a .ARFF file as a data source
Testing the Ethics of AI and prepare the .ARFF file with the available existing data of our can make predictions around project estimation and defects with the
projects. After inputting the file, we do the classification. This process same accuracy.
17 is known as training, with two data sets. We analyze the training re-
Insight from the Industry sults and track the accuracy. Once we train the system, then we have If you’re struggling with your test management and spending too
prepare the .ARFF file with the data that needs to be predicted. much time compiling test coverage information, download and
18 explore Weka. Being able to predict your coverage earlier in the
In initial execution, the accuracy was poor—10 percent or so. We had software development lifecycle allows you to better prepare for your
a discussion and analyzed the results. After that, we performed the project delivery.
training with multiple data sets, and it improved the accuracy up to
70 percent in prediction of test coverage results.
COPYRIGHT 2019 13
3
eGuide
Testing By Jason Arbon
7 Artificial intelligence is the next replace humans in testing because machines cannot think, and they
Why Testers Should Take exponential technology trend, and would require human judgment and cognition to operate properly.
Control of the AI Narrative it’s knocking on our front door. Let’s politely say it is far easier to be a skeptic than to actually dive
into the technology and understand its implications.
11 AI is broken into two major cate-
What’s Our Job When the gories of study. Artificial general The reality, though, is that there are several factors that point to the
Machines Do Testing? intelligence is the effort to make AI testing singularity:
machines that are conscious like • The tech itself. Unlike previous technological revolutions (desktop,
12 humans and can reflect on their web, cloud, mobile), AI is designed to learn from humans and rep-
own existence. The other branch licate their judgment and actions. AI is focused on solving human
Leveraging Machine
is narrow AI, also known as ma- input and output functions, and testing is just one of them.
chine learning, which is focused on • The world is spending billions of dollars on AI research. The
Coverage
computer algorithms that can be number of papers published on the topic is growing exponentially.
14 trained with data to mimic human

thinking—without actually thinking.
For a long time our field was dependent on the R&D budget of
open source “charity” projects such as Selenium and Appium and
The AI Testing Singularity low-priority testing efforts of tooling companies and vendors. As
Machine learning is rapidly growing we build AI-based testing approaches, we will ride an unfathom-
15 more powerful, already sometimes able wave of new R&D.
4 Advantages of Applying imitating the actions and judgments of humans better than humans • AI by definition abstracts concepts and feeds on large amounts
AI in Software Testing in many fields. In the near future, even before machines are con- of data and/ computing. AI is itself just data and computing, so AI
scious, they will be able to mimic human testers. What will be the can learn from the very data it generates in a reinforcing feedback
16 impact of AI on testing? loop, and executing this loop can only takes minutes to improve
itself. This means AI will get smarter far faster than humans can
Testing the Ethics of AI The AI testing singularity is the moment software can test itself get smarter.
without human intervention. This moment will come, and we all
17 have questions before that happens: How will machines learn to test The AI testing singularity will not happen overnight. But it is inevita-
Insight from the Industry better than humans? Which testing activities will succumb to AI first? ble; the question is just how soon, and how it will unfold. It’s import-
Which will be last? ant that you know how and when to bring AI into your own testing
18 efforts, as well as where to invest in your personal learning to stay
There are those who say AI is just slightly smarter software and will ahead of the technology curve.
be unable to ever behave like a human tester. Some say AI can never
COPYRIGHT 2019 14
eGuide
4 Advantages of Applying AI in
3
Software Testing
Testing
7 By Ester Brierley
Control of the AI Narrative Since the effectiveness and efficiency of your testing process are states, being able to quickly determine whether or not the program
crucial to the success of your product, there’s no surprise that we’re al- works as intended. AI-powered test automation allows for executing
11 ways looking for smarter, faster, better ways of testing. As the popular- over a thousand test cases in one test run, which is impossible through
What’s Our Job When the ity of artificial intelligence grows, more and more testers are realizing manual testing.
Machines Do Testing? its capacity to make cumbersome and time-consuming tasks simpler.
12 Artificial intelligence is coming, so we should take advantage of its abil- Even though AI still cannot
Leveraging Machine
ities. Here are four benefits to applying AI in software testing.
perform software testing with no
Coverage
1. Unfailing accuracy help from humans, it is already
Even the most experienced testers sometimes make mistakes, espe-
14 cially when dealing with repetitive tasks. This is the main reason au- capable of improving the testing
process significantly.
tomation became so popular. Unlike humans, AI always performs the
The AI Testing Singularity necessary tasks exactly as intended, completing the same repetitive
tasks successfully, time after time. While AI works on repetitive tasks,
15 testers are able to focus on creating effective automation solutions 4. Visual validation
4 Advantages of Applying and on exploratory activities that only humans can perform. Pattern recognition and image recognition enable AI to detect visual
AI in Software Testing bugs by performing visual testing of applications and making sure that
2. Improved flexibility all the visual elements look and function properly. AI can recognize
16 Even the simplest changes in an application can lead to test failures
in automation tools because traditional testing scenarios consider
dynamic UI controls regardless of their size and shape, analyzing them
on a pixel level.
Testing the Ethics of AI a singular selector or path. Therefore, such testing approaches are
somewhat rigid. Machine learning and AI allow for a more flexible Even though AI still cannot perform software testing with no help from
17 testing process, learning relationships between various segments of humans, it is already capable of improving the testing process signifi-
Insight from the Industry documentation. Such systems can automatically adapt to any changes cantly. The main advantage of AI is that it takes automation to a new
in real time, being both flexible and reliable. level so that testers don’t need to deal with repetitive tasks anymore,
18 but it also improves the flexibility and accuracy of software tests and,
3. Increased test coverage overall through pattern recognition and machine learning, allows computers
AI allows you to increase the scope and depth of tests significantly. It to perform tasks that used to require human work. That leaves us
can check the file contents, memory, data tables, and internal program more time to do the creative, exploratory aspects of testing.
COPYRIGHT 2019 15
eGuide
Testing the Ethics of AI
3
Testing By Rajini Padmanaban
7 Artifical intelligence is becoming more and more widespread in the

Why Testers Should Take software world. The potential is mind-boggling: AI is now where
Control of the AI Narrative digitization was a decade ago, touching and impacting every domain.
The technology that started in niche areas, chiefly entertainment, has
11 extended by leaps and bounds.
Machines Do Testing? But before you think this is an ideal scenario with endless possibili-
ties for reach and benefit, consider that AI is a double-edged sword. Testers have a critical role here in vetting solutions, even from the ide-
12 When AI is being used in situations involving sensitive personal data, ation stages. Areas that have traditionally been reserved for human
such as health care, banking and finance, and real estate, security is intervention due to high cognitive involvement, such as accessibility
Leveraging Machine
of the utmost importance—and so are ethical implications. engineering, are now open to the technology touch. A lot of learn-
ing—and unlearning—is needed for testers to be able to bring new
Coverage
How can testers play a role in mitigating risks? perspectives about how solutions will impact stakeholders and users.
14 Testers understand that automation is only as smart as we design Primarily, a lot of investment in quality data feeds that cover positive,
The AI Testing Singularity it to be. This also holds true for AI and similar practices such as ma- negative, null, and boundary values will make all the difference in the
chine learning, image recognition, and natural language processing. test effort and outcomes. Testers will need to employ live monitoring,
15 Plenty of sci-fi films have examples showcasing what can go wrong connect current solutions and new players, and consider what impli-
4 Advantages of Applying when undue powers are given to robots, and AI is often seen as a cations third-party integrations with an app will have. Ethical negative
AI in Software Testing core piece in humanizing technology. It’s up to testers to make sure tests may also need to be taken up in the right doses to maintain
AI is used responsibly—especially considering we’re already at a checks and balances on the adverse potential of AI systems.
16 point where people are discussing whether AI will become powerful
enough to be worshipped. Interestingly, we can also consider leveraging AI itself. Testers
Testing the Ethics of AI should automate tests as much as possible, making them smart and
Such close monitoring is necessary not just because of the potential lean and enabling automation in areas that haven’t been possible
17 AI holds, but also to keep adverse effects under check. Just as ethical until now, leaving more time for strategic manual tests.
Insight from the Industry hacking has gained popularity, ethical AI solutions are on the rise.
At a simplistic level, these solutions mainly focus on ensuring AI as a As AI continues to be used in more and more cases, everyone in the
18 technology does not support immoral causes from societal, environ- software industry has a collective responsibility to build responsible
mental, or political influences, and on limiting the power given to the and ethical AI systems. Testers have a huge role in this process, so
AI algorithms we engineer. start thinking about the implications.
COPYRIGHT 2019 16
3
eGuide
Testing
7 “Not only are UX variations continuing to pro-
liferate, but release cycles get getting shorter. The future is people working on unique, new
For example, even if you manage to run a 60
percent coverage manual UI test, regression
things. Because same-old, same-old can be
will start to occur almost immediately. This handed over to the machine.
11 does not even account for the error-prone
What’s Our Job When the nature of manual QA on your UX in the first “I really made a call to innovate, and the “When you think about software develop-
Machines Do Testing? place. Artificial intelligence technology that thing is that if we really think about testing ment, yes, we do a lot of testing from the
can mimic the human eye and brain has and how it really requires us to be inde- outside. But we also do a lot of testing from
12 opened the door to automated visual testing pendently thinking creatively, can we think of the inside, and if we actually think about
at scale, and behind that first door is a whole a future where we have systems that actu- designing these systems with these features,
Leveraging Machine
house full of applications and use cases that ally can test themselves? Is that too big of a then they can be a lot more thorough and ac-
will be addressed with visual AI.” dream? We put a man on the moon. We had tually get to a point where systems may not
Coverage
—Gil Sever the Wright brothers fly. Are those things big- only be able to detect when they have issues,
14 “I’ve met with various leaders who think au-

ger dreams or smaller dreams than just kind
of solving this problem? ... [Can we] take
but they may be able to fix themselves.”
—Jason Arbon
The AI Testing Singularity tomation is the magic pill, but it’s not. How testing and what is automated testing from
you automate and how much you automate this small checking and verification into, can “I see this as a huge opportunity. I see plenty
15 is the key. Skills and the team effectiveness is we use machine learning, artificial intelli- of practitioners asking themselves, ‘Is AI
4 Advantages of Applying also a large ingredient in this calculation.” gence? Can we leverage all that research to and machine learning going to take my job
AI in Software Testing —Jennifer Scandariato make testing better?” away?’ Totally no. Actually, that’s the big-
—Tariq King gest opportunity—I wouldn’t say ever, but
16 “That’s what I imagine the future of software
development being. Right now, yeah, there’s “I see AI having a huge, huge impact in agile
biggest opportunity in years—that business
testers, manual testers have to get back
Testing the Ethics of AI a lot of people who are building the same, and DevOps environments, especially, say, on stage and support continuous testing
or kind of variants of the same. ... We keep for example, when we talk about test auto- activities. ... Codeless testing based on AI
17 doing the same, kind of trying it in different mation. Test automation starts right from and machine learning can allow them to gain
Insight from the Industry ways. That’s temporary. That will go away. the requirements phase early in the software back trust and become involved more in test
The future is people working on unique, new development lifecycle, all the way through automation activities. By doing that, they
18 things. Because same-old, same-old can be production and monitoring, right, so that’s also contribute and add more coverage into
handed over to the machine.” kind of the shift-left paradigm. I can see AI the overall test suite.”
—Daria Mehra playing a huge part in that.” —Eran Kinsbruner
—Raj Subramanian
COPYRIGHT 2019 17
eGuide
3
MORE INFORMATION FOR SOFTWARE PROFESSIONALS
Testing
7 StickyMinds is home to thousands of software testing resources, including

informative articles about all aspects of testing and interviews with industry
Why Testers Should Take notables. StickyMinds offers how-to advice and views on the latest ideas and JOIN
Control of the AI Narrative practices from experienced software professionals and thought leaders. Join HERE
the community to gain access to exclusive members-only content such as
11 conference presentations, Q&A discussions, a weekly newsletter, and more.

NARROW YOUR SEARCH TO A SPECIFIC TYPE OF RESOURCE:
StickyMinds Articles Interviews TechWell Insights
12 StickyMinds articles cover a wide range of Each year, TechWell interviews dozens of Find stories about the future of testing, AI,
software testing topics, including artificial software professionals, including well-known agile, DevOps, teams, and more, all written by
Leveraging Machine
intelligence, the future of testing, testing tools, thought leaders, seasoned practitioners, and software industry professionals. New stories
Learning to Predict Test test automation, agile testing, test process respected conference speakers. Click here are added each weekday, so click here to read
Coverage improvement, and much more. Click here to to read, listen to, and watch interviews with the latest—or sign up to receive the weekly
read articles about AI and the future of testing testing experts about the future of testing. newsletter roundup of the newest stories.
14 on StickyMinds.

TechWell Conference Presentations STAR Conferences The TechWell Hub
15 Couldn’t make it to a TechWell conference to

sharpen your testing knowledge? TechWell
The STAR conferences feature keynote
presentations, tutorials, and classes covering
Join the active and engaging TechWell Hub
community on Slack to keep in touch with
4 Advantages of Applying conference presentations are available to test techniques, performance testing, test what’s happening in the industry and ask
AI in Software Testing StickyMinds members for free soon after automation, agile testing, mobile testing, test questions about your day-to-day challenges.
conferences end. Click here to join StickyMinds management, and more. Learn from experts The Hub offers a safe space to get real-time
16 and access conference presentations related to

the future of testing and AI.
in the field and network with your peers to get
the most immersive conference experience
answers from other software professionals in
your shoes, and even experts in the field! Click
Testing the Ethics of AI possible. Click here to learn more. here to join for free.
17 In the modern world, where speed of release time is accentuated and pressure to frequently release value to end-users
Insight from the Industry is high, sound testing practices and practical automation frameworks are necessary to fully integrate quality into the
development and delivery process. Coveros has been a pioneer in testing and QA training, offering a broad spectrum of
18 testing courses for all levels. Our coaches and consultants can also pair with and mentor your staff to accelerate their
Accelerate improvement, now and into the future.
Delivery Software Testing courses | Automation courses | Dev & Testing Tools courses | Agile Testing services
COPYRIGHT 2019 18

AIFutureTestingeGuideTechWell 0 PDF

Uploaded by

Copyright:

Available Formats

AIFutureTestingeGuideTechWell 0 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

AIFutureTestingeGuideTechWell 0 PDF

Uploaded by

Copyright:

Available Formats

F U T URE O

4 Advantages of Applying Testing the Ethics of AI

7 changed attributes of an element that were already covered by my

14 software testing process easier by providing:

7 because more concentration needs to be given tointegration testing

16 There will also be a need to have procedures in place to handle large

18 It’s important to change our mindsets about how we view systems

the AI Narrative: An Interview with Tariq King and Jason Arbon

17 to be a key part of that.

Why Testers Should Take

17 tion, awareness, is always good. Jennifer

18 cited by Colvin, those of us in the test industry would be wise to heed

14 versions of different files ended up underused. Maintaining these

15 Next, we tried a shared Google spreadsheet to work in a collabora-

7 visualize your data

14 Clusterers, Associations, and Attribute Selection—is rep-

14 trained with data to mimic human

7 Artifical intelligence is becoming more and more widespread in the

14 “I’ve met with various leaders who think au-

7 StickyMinds is home to thousands of software testing resources, including

11 conference presentations, Q&A discussions, a weekly newsletter, and more.

What’s Our Job When the

The AI Testing Singularity

15 Couldn’t make it to a TechWell conference to

16 and access conference presentations related to

You might also like

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.