00:00:05
hello and welcome to the first video of
00:00:07
the sqb certified tester Foundation
00:00:09
level preparation course my name is MRI
00:00:12
and I'm a creative manager at exactpro
00:00:15
today we will cover chapter one
00:00:17
fundamentals of testing most people
00:00:19
these days have had an experience with
00:00:21
software that did not work as expected
00:00:24
atay when waiting for a credit card to
00:00:26
process or a website not loading
00:00:28
correctly a common example of that some
00:00:31
of the system failures we encounter are
00:00:33
quite trivial but others can be costly
00:00:35
and damaging and may even result in
00:00:38
injury or death testing is a way to
00:00:40
assess the quality of the software and
00:00:42
to reduce the risk of software failures
00:00:44
in operation the quality is the degree
00:00:47
to which a component of system satisfied
00:00:50
the stated and implied needs of its
00:00:52
various
00:00:54
stakeholders one of the commonest
00:00:55
misconceptions about testing is that
00:00:58
it's just activity during which we we
00:01:00
execute tests but that would limit our
00:01:03
activities to Dynamic testing only
00:01:05
emitting static testing sometimes people
00:01:08
consider testing as verification of how
00:01:10
a system corresponds to the given
00:01:12
requirements but it's wrong because
00:01:14
testing also includes another kind of
00:01:16
check called
00:01:17
validation it's how the system
00:01:20
corresponds to the needs of business and
00:01:22
users please check the glosary for full
00:01:24
definitions of these two terms so what
00:01:27
is testing according to the sqb glossary
00:01:30
it's the process consisting of all life
00:01:33
cycle activities both static and dynamic
00:01:36
concerned with planning preparation and
00:01:38
evaluation of a component or system and
00:01:41
related work products to determine that
00:01:43
they satisfied specified requirements to
00:01:45
demonstrate that they fit for purpose
00:01:48
and to detect
00:01:50
defects let's talk about why we
00:01:53
test there can be many reasons for
00:01:56
testing these reasons are also called
00:01:58
test objectives they vary from Project
00:02:01
to project to reach our test objectives
00:02:04
we may have to number one evaluate
00:02:07
documentation and code base using static
00:02:10
testing techniques number two verify if
00:02:13
a product to be tested also known as a
00:02:15
test object corresponds to this
00:02:18
requirements number three validate that
00:02:21
a test object corresponds to the
00:02:22
business needs and users
00:02:24
expectations number four prevent defects
00:02:27
using different techniques and
00:02:28
approaches five find defects and
00:02:32
failures six provide information about
00:02:35
product conditions to the stakeholders
00:02:37
seven reduce risks related to software
00:02:40
quality eight check if regulatory
00:02:43
requirements and standards have been met
00:02:46
other two definitions that get mixed up
00:02:48
quite often are testing and debugging
00:02:51
but they're different we talked about
00:02:53
testing a little bit earlier executing
00:02:55
tests can show failures that are caused
00:02:58
by defects in the software debugging on
00:03:00
the other hand is the process of finding
00:03:02
analyzing and removing the causes of
00:03:04
failures in a component or a system one
00:03:07
more time they are different let's see
00:03:11
an example of how it
00:03:13
works in some cases testers are
00:03:16
responsible for the initial test and the
00:03:18
final confirmation test while developers
00:03:21
to the debugging Associated component
00:03:23
and component integration testing
00:03:25
however in Agile development and in some
00:03:27
other software development life cycles
00:03:29
testers may be involved in debugging and
00:03:31
component testing here are a couple more
00:03:34
terms that get confused they are testing
00:03:37
and quality assurance while people often
00:03:40
use the phrase quality assurance to
00:03:42
refer to testing these are not the
00:03:45
same we know what testing is let's talk
00:03:48
about quality assurance for a second
00:03:50
quality assurance consists of activities
00:03:52
focused on providing confidence that the
00:03:55
quality requirements will be
00:03:57
fulfilled in other words it is typically
00:04:00
focused on adherence to proper processes
00:04:02
in order to provide confidence that the
00:04:05
appropriate levels of quality will be
00:04:07
achieved when processes are carried out
00:04:10
properly the work products created by
00:04:12
those processes are generally of higher
00:04:15
quality which contributes to defect
00:04:18
prevention in addition the use of root
00:04:20
cause analysis to detect and remove the
00:04:23
causes of defects is important for
00:04:26
Effective quality assurance quality
00:04:28
management which includes all activities
00:04:31
that direct and control an organization
00:04:33
with regard to Quality ties QA and
00:04:36
testing together and then there is
00:04:38
quality control it involves various
00:04:41
activities including test activities
00:04:43
that support the achievement of
00:04:44
appropriate levels of quality terms like
00:04:46
errors defects and failures are usually
00:04:48
synonymous in our everyday speech but
00:04:51
when it comes to software testing it is
00:04:53
necessary to distinguish between them
00:04:55
let's review them as a chain of events a
00:04:58
person can make an error which can lead
00:05:00
to the introduction of a defect also
00:05:02
known as a bug in the software code if a
00:05:05
defect is executed it may cause a
00:05:07
failure of a system so in other words a
00:05:10
human action that produces an incorrect
00:05:13
result or an error can lead to an
00:05:15
imperfection or deficiency in a work
00:05:17
product where it does not meet the
00:05:19
requirements or specifications that's a
00:05:22
defect which may cause an event in which
00:05:24
a component or system does not perform a
00:05:27
required function within the specified
00:05:29
limit
00:05:30
that's a failure so for example you get
00:05:34
distracted and make a mistake while
00:05:36
coding again that's an error as a result
00:05:39
your code calls for a monthly report
00:05:41
where you should be calling for a daily
00:05:44
report that's a defect when the script
00:05:47
is executed it generates a monthly
00:05:49
report which was not what the
00:05:52
specification required that's a
00:05:55
failure the say goes to her is human to
00:05:58
forgive Divine unfortunately there is no
00:06:01
Divine component in software testing it
00:06:03
is quite unforgiving here's an example
00:06:06
of a programming error with catastrophic
00:06:08
consequences the so-called 2010 flash
00:06:11
crash it happened on the 6th of May 2010
00:06:14
lasted for about 36 minutes and during
00:06:17
the collapse stage caused the loss of
00:06:19
trillion dollars for the US Stock
00:06:22
Market the event was caused by a number
00:06:25
of factors and one of them was a faulty
00:06:27
cell algorithm that dis regarded the
00:06:29
prices while only taking the volume
00:06:32
information into
00:06:34
account considering the speed at which
00:06:37
the MoDOT exchanges worked today it took
00:06:39
minutes for this Buy sell cycle to hurt
00:06:42
the world's economy one of the important
00:06:44
questions about software defects we need
00:06:46
to ask ourselves is when do they
00:06:50
arise this picture displays four
00:06:53
possible
00:06:55
scenarios in the first case everything
00:06:57
was done right at every stage AG of the
00:06:59
life cycle this is the best case
00:07:01
scenario but not a very realistic one
00:07:04
besides this scenario contradicts the
00:07:07
testing principles we are going to go
00:07:08
over later in this video the absence of
00:07:11
bugs is deceptive and even though you
00:07:14
don't see them it doesn't mean that the
00:07:16
software is defect free in the second
00:07:19
case the requirement and design stages
00:07:21
went well but some defects were
00:07:23
introduced at the development stage
00:07:26
these bugs need to be discovered through
00:07:28
testing and then get results
00:07:30
this is a more lifelike scenario which
00:07:32
testers encounter on a daily basis third
00:07:35
scenario mistakes are made at the design
00:07:37
stage if they are not found and closed
00:07:40
the product will have design flaws still
00:07:42
correctable but at higher cost finally
00:07:46
in the worst case scenario the mistakes
00:07:48
are made on the stage of writing the
00:07:50
requirements and reproduced from there
00:07:52
on snowballing until the product is
00:07:56
released as a rule such mistakes are the
00:07:59
most most expensive ones the root causes
00:08:01
of defects are the earliest actions or
00:08:04
conditions that contribute to the
00:08:05
creation of the defect in other words
00:08:09
they are the sources of a defect such
00:08:11
that if it is removed the occurrence of
00:08:14
a defect is decreased or removed defects
00:08:18
can be analyzed to identify their root
00:08:20
causes so as to reduce the occurrence of
00:08:22
similar defects in the future by
00:08:25
focusing on the most significant ones we
00:08:27
may reach the process improve M ments
00:08:29
that prevent a significant number of
00:08:31
future defects from being introduced it
00:08:34
might seem a bit illogical that software
00:08:36
has defects after all every party
00:08:38
involved loves quality and hates bugs
00:08:41
nobody wants mistakes in the code and
00:08:44
yet they still come up why the reasons
00:08:46
are too many to mention one of the
00:08:48
biggest causes is a failure to
00:08:49
communicate the lack of clarity between
00:08:52
different project stakeholders budged or
00:08:55
misrepresented requirements and
00:08:56
cognitive biases introduced mistakes
00:08:59
leading to defects which end up causing
00:09:01
failures time constraints are another
00:09:03
crucial Factor project scheduling is
00:09:05
often done based on incorrect
00:09:07
assumptions impending deadlines bring
00:09:10
pressure loss of concentration and
00:09:12
mistakes automation or the lack of
00:09:14
thereof is another
00:09:16
contributor sometimes we don't have the
00:09:18
resources to implement Automation in
00:09:20
order to achieve the required coverage
00:09:22
or it can be the other way around
00:09:25
instead of running a manual test that
00:09:27
only takes 5 minutes you spend 5 hours
00:09:29
automating it and still make a mistake
00:09:32
and the leas goes on so rigorous testing
00:09:35
is necessary to discover the bugs and
00:09:37
increase the quality of the system this
00:09:39
is measured in terms of the number of
00:09:41
defects found the amount of tests run
00:09:45
and the extent to which the system is
00:09:47
covered by tests we can do this for both
00:09:49
the functional software attributes for
00:09:51
example sending and receiving an order
00:09:54
correctly and for the non-functional
00:09:56
ones for example sending and receiving
00:09:58
an order order quickly let's talk about
00:10:01
seven testing principles principle
00:10:03
number one testing can show that defects
00:10:06
are present but cannot prove their
00:10:10
absence testing reduces the amount of
00:10:12
undiscovered defects remaining in the
00:10:14
software but even if no bugs are present
00:10:16
it doesn't prove the aips
00:10:19
perfect the absence of proof is not the
00:10:21
proof of absence the fact that you gaze
00:10:24
into the night sky and see no exoplanets
00:10:27
doesn't mean that they are not there we
00:10:29
have been looking for earthlike planets
00:10:30
for a very long time but it wasn't until
00:10:32
1995 that we found one the same goes for
00:10:36
the search for Planet 9 in our solar
00:10:38
system we haven't found it yet but it
00:10:41
doesn't mean that it's not out
00:10:44
there testing is aimed at proving that
00:10:46
software contains defects rather than
00:10:48
proving that it doesn't confirming that
00:10:51
it doesn't is impossible anyway
00:10:54
principle number two exhaustive testing
00:10:57
is
00:10:58
impossible it means testing all
00:11:01
combinations of inputs is not feasible
00:11:03
except for trivial cases instead of
00:11:05
exhaustive testing it is recommended to
00:11:07
focus the efforts on risks and
00:11:10
priorities even with the simplest
00:11:12
program that requires entering a number
00:11:13
into a field the amount of all possible
00:11:15
test case combinations can become
00:11:17
infinite and the test case is not just
00:11:20
one variable it's a set of preconditions
00:11:22
inputs actions expected results and post
00:11:25
conditions developed based on test
00:11:28
conditions now imagine that we're
00:11:30
testing a complex exchange platform at
00:11:33
the limitation applied by time and
00:11:35
budget and you will see why the best
00:11:38
practices suggest targeting the most
00:11:40
important test cases principle three
00:11:43
early testing testing activities should
00:11:46
start as early as possible in the
00:11:48
software development life cycle and
00:11:49
should be concentrated on predefined
00:11:52
objectives the reason for that is very
00:11:54
simple and we cannot stress its
00:11:57
importance enough the early we find a
00:11:59
defect the cheaper it is to fix it
00:12:02
besides a defect discovered at the
00:12:04
requirement stage is much easier to fix
00:12:06
than the one found during the acceptance
00:12:09
testing when each component may affect
00:12:11
the other ones in an unpredictable way
00:12:14
as you go from stage to Stage the cost
00:12:16
of fixing a bug grows exponentially
00:12:19
principle four defect clustering a small
00:12:23
number of modules contain most of the
00:12:25
defects found according to the 8020 par
00:12:29
principles 80% of defects are usually
00:12:32
found in 20% of the source
00:12:35
code the explanation for this effect can
00:12:37
lie in the fact that some parts of the
00:12:40
system are more complex than the others
00:12:42
or reading according to the laow Quality
00:12:46
specifications what it means to a tester
00:12:48
is that if a bug is found in a certain
00:12:51
part of the system then chances are
00:12:53
there will be more of them hiding nearby
00:12:56
principle five the pesti side
00:12:59
Paradox if the same tests are repeated
00:13:02
over and over again then sooner or later
00:13:05
they will no longer find any new bugs
00:13:08
let's just say the bugs that survive the
00:13:10
most intensive testing routine become in
00:13:13
some way immune to this test set it is
00:13:15
similar to bacteria becoming resistant
00:13:17
to certain type of antibiotics if they
00:13:20
used
00:13:21
repeatedly testing an order with the
00:13:24
same parameters may be successful at
00:13:26
first but repeating it all over and over
00:13:29
again will soon stop bringing any
00:13:31
results and give us a false sense of
00:13:33
absence of defects but try changing one
00:13:36
parameter and it will probably uncover
00:13:39
new bugs to overcome the pesticide
00:13:41
Paradox the test cases need to be
00:13:43
regularly revised and new ones need to
00:13:46
be written principle six testing depends
00:13:49
on the
00:13:50
context testing is diverse and context
00:13:53
dependent for example safety critical
00:13:55
software is tested differently from an
00:13:56
e-commerce site take Google for example
00:14:00
they can afford to release an under
00:14:01
tested product and let the users do
00:14:03
their work for them but what could be
00:14:06
acceptable for Google is hardly
00:14:08
acceptable for software powering medical
00:14:09
equipment or
00:14:11
spaceships this is a completely
00:14:13
different level of accountability the
00:14:15
stock exchange software lies in between
00:14:18
it will not kill anyone if not tested
00:14:20
thoroughly enough but it can hurt
00:14:22
Investments jobs and economic stability
00:14:24
principle seven absence of Errors is
00:14:27
deceptive finding and fixing defects
00:14:30
does not help if the system we've built
00:14:32
is unusable and does not fulfill the
00:14:34
needs and
00:14:35
expectations if the system does not do
00:14:38
what the user wants from it the absence
00:14:40
of defects is pointless now let's talk
00:14:44
about the test process which is the set
00:14:46
of interrelated activities consisting of
00:14:48
test planning test monitoring and
00:14:50
control test analysis test design test
00:14:53
implementation test execution and test
00:14:57
completion one thing you have have to
00:14:59
understand is that there is no Universal
00:15:00
way to organize it but there are several
00:15:03
sets of testing activities which can
00:15:05
help you to achieve objectives better
00:15:07
keep in mind that in every situation
00:15:10
context is important contextual factors
00:15:13
that influence the test process for an
00:15:15
organization include but not limited to
00:15:18
one software development life cycle
00:15:21
model and project
00:15:22
methodologies two test levels and test
00:15:25
types three product and project risks
00:15:29
four business domain five operational
00:15:32
aspects for example budget and deadlines
00:15:36
six organizational policies and
00:15:38
practices and seven required internal
00:15:40
and external standards now that we know
00:15:43
what affects the test process let's look
00:15:44
at all the stages that it consists of
00:15:47
let's do it one more time so these
00:15:49
stages are test planning test monitoring
00:15:53
and control test analysis test design
00:15:57
test implementation test test execution
00:15:59
and test
00:16:01
completion test planning involves the
00:16:03
activities that Define the objectives of
00:16:04
testing and the approach for meeting
00:16:07
test objectives within the constraints
00:16:09
imposed by the context test plans may be
00:16:12
Revisited based on the feedback from
00:16:14
monitoring and control activities it is
00:16:16
worth mentioning that it would be very
00:16:18
useful if the test basis has measurable
00:16:21
coverage criteria defined the coverage
00:16:23
criteria can act effectively as key
00:16:25
performance indicators kpis to drive the
00:16:28
activities that demonstrate achievements
00:16:31
of software test objectives test
00:16:33
monitoring involves the ongoing
00:16:35
comparison of actual progress against
00:16:37
the plan progress using any test
00:16:39
monitoring metrics defined in a test
00:16:42
plan test control involves taking
00:16:44
actions necessary to meet the test plan
00:16:46
objectives during test analysis the test
00:16:49
basis is analyzed to identify testable
00:16:52
features and Define Associated test
00:16:54
conditions in other words test analysis
00:16:57
determines what to test during test
00:17:00
design the test conditions are
00:17:02
transformed into high level test cases
00:17:04
sets of highlevel test cases and other
00:17:06
testware so test design answers the
00:17:08
question of how to test test data serves
00:17:12
to assign concrete values to the inputs
00:17:14
and expected results of test cases such
00:17:16
concrete values together with explicit
00:17:19
directions about the use of the concrete
00:17:21
values turn high level test cases into
00:17:23
executable low-level test cases the same
00:17:26
high level test case may use different
00:17:28
dat
00:17:28
when executed on different releases of
00:17:30
the test object the concrete expected
00:17:33
results which are associated with
00:17:35
concrete test data are identified by
00:17:37
using a test Oracle during test
00:17:40
implementation the testware necessary
00:17:42
for test execution is created and or
00:17:45
completed including sequencing the test
00:17:48
cases into test procedures so test
00:17:50
implementation answers the question of
00:17:53
do we now have everything in place to
00:17:56
run the tests during test execution test
00:17:59
Suites are run in accordance with the
00:18:00
test execution schedule test completion
00:18:03
activities collect data from completed
00:18:05
test activities to consolidate
00:18:07
experience testware and any other
00:18:09
relevant information test work products
00:18:12
are created as the part of test process
00:18:14
just as there is significant variation
00:18:16
in the way that organizations Implement
00:18:18
test processes there is also significant
00:18:21
variation in the types of work products
00:18:23
created during the process in the ways
00:18:25
those work products are organized and
00:18:27
managed and in the names used for these
00:18:30
work products let's look at those test
00:18:32
work products by stage during test
00:18:34
planning stage there are test plan
00:18:36
including test basis and entry and exit
00:18:40
criteria during test monitoring and
00:18:42
control stage there test summary and
00:18:44
progress
00:18:45
reports during the test analysis stage
00:18:48
their test
00:18:49
conditions during test design they high
00:18:52
level test cases test data environment
00:18:55
configurations during the test
00:18:57
implementation they procedures Suites
00:18:59
and test execution
00:19:01
schedules during the test execution
00:19:03
stage there are TC statuses and defect
00:19:07
reports during the test completion stage
00:19:09
there test summaries reports action
00:19:11
items CRS finalized testware one more
00:19:14
time test work products and their names
00:19:17
vary significantly but regardless of
00:19:19
these variations in order to implement
00:19:21
effective test monitoring and control it
00:19:23
is important to establish and maintain
00:19:26
the degree to which a relationship can
00:19:28
be established between two or more work
00:19:31
products which is called traceability in
00:19:34
addition to evaluation of test coverage
00:19:37
good tracebility
00:19:38
supports analyzing the impact of
00:19:41
changes making testing
00:19:44
auditable meeting it governance criteria
00:19:47
improving the understandability of test
00:19:49
progress reports and test summary
00:19:51
reports to include the status of
00:19:53
elements of the test basis for example
00:19:55
requirements that pass their tests
00:19:57
requirement ments that fail their tests
00:19:59
and requirements that have pending tests
00:20:02
relating the technical aspects of
00:20:03
testing to stakeholders in terms that
00:20:06
they can understand providing
00:20:08
information to assess product
00:20:10
quality process capability and project
00:20:14
progress against business goals some
00:20:16
test management tools provide test work
00:20:18
product models that match part or all of
00:20:21
the test work products outlined in this
00:20:24
section some organizations build their
00:20:26
own Management Systems to organize the
00:20:28
work products and provide the
00:20:30
information traceability they require
00:20:33
the mindset we want to adapt while
00:20:35
testing is different from the one we use
00:20:37
in the
00:20:38
development if our goal is to create
00:20:41
something we tend to work positively and
00:20:43
constructively to solve problems and to
00:20:45
deliver a product that meets specific
00:20:48
need however when we test this product
00:20:51
we'll look for its flaws our attitude is
00:20:54
critical and
00:20:56
destructive in this case our goal is to
00:20:58
prove that the product is short of
00:21:01
expectations Carl Sean stated in his
00:21:03
rules of skeptical thinking that we
00:21:05
should not cling to the hypothesis just
00:21:07
because it's ours and we should always
00:21:10
get an independent
00:21:11
confirmation this is a common practice
00:21:14
in scientific Community suppose you are
00:21:17
the head of the movie production team
00:21:19
you pick your actors and locations you
00:21:22
work on the story line and you frame
00:21:24
each shot carefully you spend weeks
00:21:27
pacing The Cutting Room floor you at the
00:21:30
soundtrack the release date is here your
00:21:33
movie is ready but all of this doesn't
00:21:35
mean that you have made a good one for
00:21:38
every dark night there is Batman and
00:21:40
Robin for every Matrix there is Matrix
00:21:43
resurrections the critics and the
00:21:45
viewers have a different mentality that
00:21:47
the production team and they are not
00:21:50
going to overlook the plot holes bad
00:21:52
acting and poor taste same goes for
00:21:55
software testing building software and
00:21:57
testing software differ in the
00:21:58
underlying logic we do not mean that a
00:22:01
tester can be a programmer or that a
00:22:03
programmer cannot be a tester although
00:22:05
these are often separate roles sometimes
00:22:07
a good director will be as objective
00:22:09
about his own work as a critics there
00:22:11
are countless stories of movies going
00:22:13
through extensive reshoots to get some
00:22:15
of the finer points just right so with
00:22:19
the right mindset programmers can test
00:22:21
their own code and they can resolve many
00:22:24
issues before anyone else sees the
00:22:26
Prototype how ever we all know it is
00:22:29
difficult to find your own
00:22:31
mistakes so all of them often rely on
00:22:35
others to help to test their work this
00:22:39
other person might be a fellow analyst
00:22:41
designer or developer we can identify
00:22:44
several degrees of separation between
00:22:46
the person who wrote the code and the
00:22:48
one who checks it we organize them in a
00:22:51
simplified
00:22:52
progression tests can be run by the
00:22:55
person who wrote the item on the test
00:22:58
another person within the same team such
00:23:00
as another programmer a person from a
00:23:02
different division of the same company
00:23:04
and finally test can be designed by a
00:23:06
person from a different organization
00:23:08
such a specialized technology vendor
00:23:11
early in the life cycle requirement
00:23:13
reviews by someone other than the author
00:23:16
help find defects before coding starts
00:23:18
and help build the right software after
00:23:21
coding is complete the software can be
00:23:23
tested by an external
00:23:25
team this separation avoids auth bias
00:23:28
and often more effective and fighting
00:23:30
defects and failures we should note that
00:23:32
Independence is not necessarily the most
00:23:34
important factor in high quality
00:23:36
testing developers can be self-critical
00:23:39
and benefit from knowing their work some
00:23:41
software development methodologies
00:23:43
insist on developers designing tests
00:23:45
before they start coding and executing
00:23:47
those tests continuously as they
00:23:49
progress this approach encourages early
00:23:52
testing which is cost effective this
00:23:55
practice is one of the key testing
00:23:57
principles from promoted by the sqb
00:23:59
foundation level syllabus testers and
00:24:01
test managers need to have good
00:24:02
interpersonal skills to be able to
00:24:05
communicate effectively about defects in
00:24:07
order to build positive relationships
00:24:09
with their colleagues here are some ways
00:24:11
to mitigate possible conflicts number
00:24:14
one start with collaboration rather than
00:24:16
battles number two emphasize the
00:24:18
benefits of testing number three
00:24:21
communicate test results and other
00:24:23
findings in a neutal fact focused way
00:24:26
number four try to understand how the
00:24:29
other person feels and why he may react
00:24:32
negatively and number five confirm that
00:24:35
the other person has understood what has
00:24:37
been said and vice versa this concludes
00:24:40
the first chapter of the sqb foundation
00:24:42
level preparation course good luck with
00:24:44
your studies and I'll see you at chapter
00:24:49
2
