Testing Software

The importance of testing by Lezley Davidson

We are now entering the real of software testing, which is something that has been mentioned a lot of my previous blog posts but now we are going to enter completely on the topic.

Before starting, we are going to define testing as validating that a piece of software meets its business and technical requirements.

Testing Process

The process of software test depends a lot of the company or individual, because depends on the workflow that is used, but I found this blog post by Url Eriksson that explains a general overview in 4 key elements:

#1 Test Strategy and Test Plan

I mentioned about testing plan in my Software Verification and Validation blog post, this plan comes at the start of our project while we are setting up the requirements and we improve or make changes when we have a deeper view of the system during the detailed design. Some of the elements that we need to take into account are the following:

  • Features and functions that are the focus of the project
  • Non-functional requirements
  • Key processes to follow – for defects resolution, defects triage
  • Tools—for logging defects, for test case scripting, for traceability
  • Documentation to refer, and to produce as output
  • Test environment requirements and setup
  • Risks, dependencies and contingencies

#2 Test Design

One important thing that we do not take much into consideration while we code is our test cases, this can backfire while we are creating our unit tests and do not have a proper way to test the functionality, one thing that we can do to avoid this situation is to design and plan our test accordingly, or even sometimes you can have a Test Driven Development approach.

Agile Testing Life Cycle

#3 Test Execution

Actual photo of me trying to test my code. Photo on Pickryl by U.S. Navy.

The way the test are executed depends a lot of the methodology of the life cycle that you are following to develop your code, can be a single waterfall integration test, or multiple tests every Agile sprint, or so on, at the end, your end goal is having enough tests to make sure that the system is relatively bug-free and changes do not break key components.

One of the focuses during the execution is “What is the environment that we are running these tests on?”, one example that is really common is on Mobile Development, we have a lot of set of screens and devices that go from low to high end, and need to make sure that an application work on these. On my experience, while I had done web development work at Google, we had tests that make sure that our application runs correctly on different browsers.

#4 Test Closure

To finish up our test we need to make sure that we have a green light for the test, for this, metrics are established in the company or team, some examples are as follows:

  • 100% requirements coverage.
  • Minimum percentage of tests passed.
  • Critical defects to be fixed.
  • Percentage of lines of code covered by test cases.

It all depends on what the company and the stakeholder’s establish as the requirement, as a rule of thumb, the minimum percentage of tests passed is always above 90%.

Types and levels of testing

Types

There are many types of test that we can do to make sure that changes do not break our software, Sten Pittet has a great article where he talks about some types, we are going to talk about them here:

  • Unit Testing: Tests individual functions or methods.
  • Integration Testing: Verify that services or different modules together behave correctly.
  • Functional Testing: Verify that the output of an action gives the correct result, doesn’t matter what happened before.
  • End-to-end Testing: Replicates an user behavior in a complete software environment.
  • Acceptance Testing: Verifies that the system meets business requirements. Mostly done to end users.
  • Performance Testing: Checks the system when it’s under heavy stress to see how it behaves.
  • Smoke Testing: Quick tests that check basic functionality. Normally done after you deploy an application in a new environment.

Levels

We can find four levels of testing, that ToolsQA describes on this blog post as follows:

Testing levels by ToolsQA.
  • Unit Testing: Checks if the software components are fulfilling it’s duty. (Level 1)
  • Integration Testing: Checks the data flow from different components of the system. (Level 2)
  • System Testing: Evaluates all the functional and non-functional needs of the system. (Level 3)
  • Acceptance Test: Checks the requirements of the stakeholder’s are met. (Level 4)

Activities and Roles of Testing

QA image by Colin But.

In every software company, there is a some people that is in charge of the testing. In a blog post by Test Institute. I found 2 of the most important roles that you can find in teams at professional environments.

Test Lead / Manager

As the name suggest, this person functions as the team leader of the testing environment, his/her key activities are:

  • Defining the testing activities for testers.
  • All responsibilities of test planning.
  • Allocate resources for the testing.
  • Prepare the status report of testing activities.
  • Interacts with customers and stackholders.
  • Updating project manager regularly about the progress of testing activities.

Test Engineers / QA testers

These are the subordinates of the test lead, the key activities are:

  • Read all the documents and understand what needs to be tested.
  • Based on the information procured in the above step decide how it is to be tested.
  • Inform the test lead about what all resources will be required for software testing.
  • Develop test cases and prioritize testing activities.
  • Execute all the test case and report defects, define severity and priority for each defect.
  • Carry out regression testing every time when changes are made to the code to fix defects.

Something important to keep in mind is that on some companies, software developers also work as test engineers sometimes, if there is no dedicated team for it.

Testing environments

Types of test environments by Temdotcom.

What are testing environments exactly? Well, remember that previously we talk about all these types of tests? Now, we need to know where to run them, and also, in large production environments, they are mostly automated, for these, you need to have an environment where they can run, and that’s what we are talking about today. There are a lot different tests that can be done, and the types of test environment that we can have depends a lot on the author, so we are going to focus on the ones on this blog post by Temdotcom.

Integration Testing Environment

We talk about integration tests before, to set them up, we need to have a complete environment where we can run it, since we need to run multiple components at the time, and then make them communicate to make sure everything is working correctly.

Normally, a team will have a set of integration tests to run, and it all comes down to having the hardware, software and right configuration to be able to run the tests.

Performance Testing Environment

This environment is to determine how the system behaves in some situations, and verify their response time, stability and other factors that need to be determined based on the requirements.

These tests are quite variable and sometimes complex to determine, some of the things that need to be setup during these tests are:

  • Number of CPUs.
  • Number of concurrent users.
  • Size of RAM.
  • Data volume.

Security Testing Environment

When working with software where security is critical, security teams want to make sure that the software does not have potential vulnerabilities or flaws in different areas. Normally these are setup by security experts.

First of all, they set a base case for normal vulnerabilities that can happen on the environment of the software, for example, SQL Injection. Once this tests are done, the security experts focus on more in depth test cases that require knowledge of how this particular system works.

Chaos Testing Environment

Chaos Engineering is a whole other aspect to learn about, the main goal that we are trying to focus is to test our system when the worst possible scenario can happen, for example, let’s say that Google authentication is gone and your application relies on that, what would happen? One of the most crucial scenarios is a web application with microservice architecture.

Test Case Design techniques

We talked before on this blog post about designing our tests, in which we can use a set of useful techniques that allow is to plan and think about our test cases.

Black-box techniques

Black box testing by Guru 99.
  • Boundary Value Analysis: Explore errors on input domain. Catches any input that might break the system.
  • Equivalence Partitioning: The input data is set into multiple different classes and then testing all of that data, reducing the number of test cases.
  • Decision Table Testing: The test are based on decision tables (duh), and compared with the different combinations.
  • State Transition Diagrams: This tests are based on State Machines with a finite number of states, the transitions are based on rules that depend on the input.
  • Use Case Testing: The test cases are based on business scenarios and end user functionalities. These cover the entire system.

White-box techniques

White Box Testing by Invensis
  • Statement Testing and Coverage: As the name suggest, is testing coverage of the test cases, which means the percentage of all available methods/functions that the source code contains that are covered by tests. The acceptable percentage is given by the client or per business preferred metric.
  • Decision Testing Coverage: This testing technique covers all the possible combination of cases that will test all the branches that are part of the code.
  • Condition Testing: It covers a 100% of the code, and all expression expect true or false. The testing outcome is designed to be fast.
  • Multiple Condition Testing: Same as the condition testing but with more than two possible conditions. More scripts are required, hence more effort.
  • All Path Testing: The source of the program is leveraged to find every executable path. This can help determine faults on the code.

Managing Bugs

Once we have our application running, doesn’t matter the amount of testing that we do, as the application gets bigger, the user will always find a way to break the program in some way, and part of our job it’s to fix it, the problem is, how do we know about that bug?

One of the main ways to do this is through bug reports, which is a detailed way to log and notify developers about the defects that are present on the artifacts, while we encounter these they need to have meaningful information that helps us solve the problem. Guru99 has a great blog post that contains the following fields:

  • Defect ID: Unique identification number.
  • Defect Description: Detailed description of the defect including information about the module in which was found.
  • Version: Version of the application in which was found.
  • Steps: Detailed steps along with screenshots with which the developer can reproduce the defects.
  • Date Raised: Date when the defect is raised
  • Reference–  Provide reference of documents such as requirements, design, architecture or maybe even screenshots of the error to help understand the defect
  • Detected By – Name/ID of the tester/user who raised the defect
  • Status – Current status of the defect.
  • Fixed by – Name/ID of the developer who fixed it
  • Date Closed – Date when the defect is closed
  • Severity which describes the impact of the defect on the application
  • Priority which is related to defect fixing urgency. Severity Priority could be High/Medium/Low based on the impact urgency at which the defect should be fixed respectively

All these information can be filled by either users or testers, depending on the way it is sent. Often a feature in the application is built to be able to log the information, for example, a simple form in a web application.

Final Thoughts

Cool meme by Buggylina

The world of software testing is huge, with a lot of places to focus on and a lot of different techniques that we can use. One of the main things I have learnt during my professional experience is that testing takes more time than what you take to code a feature, since there are a lot of factors that you need to take care of on large production environments.

One of the most important things I can recommend is to NOT write many tests for something that is not necessary, sometimes you might think that over engineering your code might be great, but sometimes can backfire and overshadow bugs that can happen during product. The best advice, be smart about your tests.

Why verifying and validating your software is important?

Photo by ThisIsEngineering on Pexels.com

Hi everyone! Today we are going to learn about an exciting and important topic for software development, Verification and Validation of Software (also known as V&V), the first thing that we need to understand it’s those two keywords, I have mentioned about them in a previous blog post, but will refer to them in here:

“A test of a system to prove that it meets all its specified requirements at a particular stage of its development.”

Definition of Verification by IEEE-STD-610

“An activity that ensures that an end product stakeholder’s true needs and expectations are met.”

Definition of Validation by IEEE-STD-610

As we can see in the definitions, the key difference it’s that verification can happen at any particular stage of the development, this to verify that the system meets the requirements at that stage, and we finally validate our end product to see if it satisfies the users/stakeholder’s expectations. Although, validating can be at the end of each stage too.

V&V on Life Cycles

V&V sounds good, but every time you are working on a software project on a company, it is natural to follow a life cycle model (you can see more about them in my post), and the question will be: How can I verify and validate at every phase?

Mohamed Sami has a great blog post, that I recommend reading, where he talks about V&V and how it’s implemented in each of the Software Development Life Cycle (SDLC) phases:

PhaseVerificationValidation
RequirementsThe requirements need to be unique and server a purpose for each one of them. The requirements need to fulfill the vision of the client/stakeholder, eliminating every ambiguity.
DesignVerify that the designs meet the requirements that are specified by the previous phase.The design has prototypes, either of UI, wireframing, or architecture design. Could even be a design document.
ImplementationThis is the place where we verify the code by using unit and integration testing, and code reviews.The requirements are well defined and implemented, make changes accordingly if necessary.
TestingVerify that all the test cases have passed, and the software works properly.Validate that the software is fit for client and user acceptance, meeting the requirements previously defined.
DeploymentVerify installation and configuration of environment, running tests.Make sure that everything works when is installed.
Implementation of V&V in SDLC phases.

International Standards

Photo by Breakingpic on Pexels.com

Now we arrive to the standards part! Which we have a lot of those regarding software practices, we are going to dive into these really briefly, keep in mind that if you want to deeply learn about these, it is recommended to go to the links I will provide in each one of them. We are going to focus on only two of them ISO and CMMI, keep in mind that there are others too. I recommend reading this blog post where it details more high level information about V&V.

ISO

The first International Standard of V&V that we are analyzing is ISO/IEC 12207:2008, which includes a series of activities of verification and validation that are part of the software development process.

VerificationValidation
The requirements need to be reviewed before going forward.Elaborate test requirements, such as the test cases based on those requirements.
The software design need to have reviews on both high and low level design documents.Evaluate testing cases based on the designs.
During implementation Code Reviews are necessary.Test functionalities and other metrics needed.
It’s necessary to verify that the user manuals and other documents are correct and updated.Test that the software meets business standards.
V&V in ISO 12207:2008

On my experience, this is the one that I am most familiar with in my past internships at Google, specially with both high and low level design documents and code reviews, which is a fundamental part of the development.

CMMI

The Capability Maturity Model Integration (CMMI), evaluates the process of developing, maintaining and operating software systems. It has with it a few V&V activities that are good practice for large companies:

VerificationValidation
Do peer reviews.Validate that the software is ready are suitable for the production environment.
Establish organizational policies for planning and reviews.Make changes accordingly with the lessons learnt in the reviews.
Verify what are the work of the products built.When providing validation, everything needs to be monitored and controlled.
V&V in ISO 12207:2008

Personally, I am not to familiar with this method, although I have experience pieces of it, such the establishment of organizational polices for planning and reviews. From my perspective, this environment works best when releasing huge products for enormous companies, one example is Oracle DB services for companies. You can read more about this model in here.

Plan V&V

First of all, when we start gathering requirements and designing, we need to have a plan on how we are going to verify and validate our software, this is called the System Verification Plan, and it’s common to see in a “V” like life cycle:

V diagram from Berkeley article.

On this document it shows what are the procedures that are going to be follow to validate the program, and it also shows the requirements of the stakeholder’s. It can be written at the start of the software development life cycle, but sometimes changes are made later after the design once things are starting to get clearer about the whole system structure.

Administrating the plan

While in executing, things need to be always monitored, for this, we have can administrate what is happening, for this we mostly use tools, and nowadays, online tools, there is this document from the Requirements Experts where it shows many tools for administration, and also I have a blog post where I talk about Tools for V&V. For administration the ones that are the most useful are:

  • Verification Requirement Definition Sheet.
  • Task Definition Sheets.
  • Task Performance Sheet.

Final Thoughts

Spaguetti Code from EGU blogs.

Verification and Validation is crucial for a lot of software development companies, it provides a way to correct or identify issues before they appear, and mostly avoiding having “spaguetti code” as the code base starts to get larger. Definitely a thing that is worth reading more about!

Software Review

Software review is the process in which a team, users or customers examine a software product! Types of Reviews: Peer Reviews: Your team or colleagues help check your software! Code Review Pair Programing Inspection Management Reviews: Your BOSSES check your software! Audit Reviews: You pay a third party to check your software to evaluate it according to standards and contractual agreements! IEE 1028 . [Entry evaluation]: The Review Leader uses a standard checklist of entry criteria to ensure that optimum conditions exist for a successful review. 1. Management preparation: Responsible management ensure that the review will be appropriately resourced with staff, time, […]

The post Software Review appeared first on AC Development.

V&V Tools

Version Control! Version control is THE MOST important tool to me, it allows you to collaborate with your team and merge code “easily”. Local Version Contorls Pretty much pasting the project in a stable state to another folder and holding on to it for reference. Centralized Version Controls A server stores the stable versions of the software and different team members check out files from there! My top Version Controls Tools! GIT Widelly Used! Super Popular Super Easy BITBUCKET Not soooo popular Harder to get into More professional

The post V&V Tools appeared first on AC Development.

Software Testing

Testing, testing, testing! Software Testing is a method to check whether the actual software product matches expected requirements and to ensure that software product has no defects! Software Testing is Important because if there are any bugs or errors in the software, it can be identified early and can be solved before delivery of the software product. Benefits of Software Testing: Cost-Effective Ensures Quality Ensures Satisfaction Types of Testing! Functional Testing: Unit Testing Integration Testing Smoke UAT Localization Globalization Non Functional Testing: Performance Endurance Load Volume Scalability Usability Maintenance: Regression Maintenance Unit Testing: This software testing approach is followed by the programmer […]

The post Software Testing appeared first on AC Development.

Calidad mejor que cantidad

Software Quality

Bienvenidos una vez más a este su blog favorito donde hablamos del desarrollo de software. En esta ocasión vamos a tomar enfoque en la calidad y las pruebas de software, algo de suma importancia en el desarrollo de software.

A lo largo de estos blogs hemos podido apreciar como el hacer software no es simplemente hacer código y compilar sino que hay todo un proceso detrás de para facilitarnos las cosas tanto a nosotros como a aquellos que trabajan con nosotros. Pues bueno otro de estos puntos del proceso se encuentra la calidad y las pruebas de software, así que porqué no comenzamos directamente definiendo qué es calidad de software:

Qué es calidad de software

Como siempre recurrimos a nuestra vieja y confiable Wikipedia donde se define calidad de software se puede dividir en dos aspectos importantes:

  • Calidad funcional: se refiere a que tan bien cumple el diseño de acuerdo a los requisitos o especificaciones funcionales, es decir, si nuestro programa hace lo que se dijo que haría.
  • Calidad estructural: se refiere a cómo se cumplen con los requisitos extrafuncionales que respaldan la entrega de los requisitos funcionales, es decir, el que tan bien funciona el software según sea necesario, estar listo para imprevistos.

El proceso

Ya hemos hablado de como todo en esta vida para que se haga de manera adecuada debe tener un proceso, y la calidad del software no es la excepción entonces para ello necesitamos dejar en claro como gestionar los posibles defectos y cuáles serán nuestros atributos de calidad.

Podemos tener defectos tanto en nuestro software pero también en el usuario, siendo uno aquel que se basa más en contar y controlar los defectos pudiendo clasificarlos para poder dar prioridades; mientras que el otro se refiere más a aquellos requerimientos que no se cumplieron y provoca que el usuario no pueda hacer lo que se esperaba, estos pueden ir desde errores en el diseño hasta errores en la misma codificación.

Asegurar la calidad de software

Ahora bien debemos llevar a cabo un proceso y cumplir ciertas actividades para poder hacer una revisión de nuestro software y poder medir la calidad de este, es entonces cuando la Software Quality Assurence aparece para indicarnos cuáles actividades debemos llevar a cabo, dentro de las que se encuentran:

  • Definición de procesos
  • Entrenamiento de procesos
  • Implementación de procesos
  • Auditoría de procesos

Entonces la SQA nos ayuda a definir un ciclo para cada proceso y llevarlo a cabo para obtener la tan ansiada calidad de software, así que ahora veamos cómo podemos tener la tan aclamada calidad en nuestro software.

Los estándares en la Calidad de Software

Para poder llevar a cabo el proceso requerimos entonces de las actividades para poder evaluar nuestra creación de software.

Sabemos que para tener una mejor prueba de calidad el acudir a las normas establecidas en la industria es lo mejor ya que obtenemos un cierto grado de calidad que está respaldado, dentro de estos podemos encontrar algunas normas muy conocidos como lo son ISO e IEC y dentro de estos tenemos algunos estándares que nos ayudan a la calidad de software:

  • ISO/IEC 9126, dentro de esta norma nuestra calidad de software se puede clasificar en seis factores, buscando entonces como nuestro software actúa de manera externa a partir de ciertos atributos internos, calificando entonces:
    • Funcionalidad
    • Fiabilidad
    • Usabilidad
    • Reutilización
    • Eficiencia
  • ISO/IEC 250:10:2011, similar a nuestra norma anterior pero que ahora tiene un enfoque hacia el dominio del software, el tipo de producto y el uso que se espera de este, para esta norma contamos con ocho estándares que debemos cumplir:
    • Idoneidad funcional, se refiere a qué tan bien funciona el sistema y el cómo completar las tareas cuando se encuentra en condiciones específicas.
    • Fiabilidad, se refiere a que tan bueno actúa nuestro software en condiciones especificas durante un periodo de tiempo.
    • Eficiencia de rendimiento, se refiere a que tan bien aprovecha los recursos en ciertas condiciones para tener un buen rendimiento.
    • Compatibilidad, se refiere a que tan bien el sistema puede intercambiar información con otros sistemas si lo requiere.
    • Usabilidad, se refiere al grado en que los usuarios pueden utilizar el software para cumplir sus objetivos con eficacia, eficiencia y satisfacción.
    • Seguridad, se refiere a que tan bien el sistema puede proteger los datos teniendo en cuenta niveles de autorización y evitando compartir información confidencial.
    • Portabilidad, se refiere a que tan bien el sistema puede cambiar y utilizarse en otro hardware, software o en un sistema operativo diferente.

Muy bien ya tenemos los puntos que debemos tomar en cuenta para poder reconocer si nuestro software tiene calidad o simplemente cumple su objetivo pero no de la mejor manera.

Conclusión

La calidad como en cualquier otro ámbito debe ser respaldada por ciertos estándares que no solamente dan seguridad y confianza ante el producto sino que también aportan valor a la marca demostrando así que se puede llegar a tener una buena relación con la empresa y el cliente. El software no es una excepción y creo que no solamente debemos contar con las buenas prácticas y todo aquello que se nos ha indicado en semestres pasados sino que también debemos verlo como un producto más profesional y proporcionar siempre la mejor calidad antes que cantidad.

Tools for V&V

From earlier topics, we now know what V&V is and how can we execute it, but this world is all about making life easier,  and how else can you make life easier for developers other than give them tools to facilitate their job? These are tools that will help greatly in the verification and validation process.

Tools for version control

Version control has been very useful for every developer out there. Version control tools are systems that can record changes made in the code file or other files. These records can be consulted later to perhaps restore changes previously overridden or simply to check what changed from one version of the software to another.

Git

Git is one of the best version control tools that are available. I personally use it all the time when it comes to school and work related projects.

CVS

Another popular choice for version control, CVS works in a different way than Git but accomplishes the same goal. Unlike Git that works with commits, CVS keeps track of each file individually, each one of them having its own version key.

Mercurial

Written in python, mercurial is similar to Git in many ways, the key difference is how mercurial saves each commit. Git represents its commits as snapshots, containing in each one of them all the files in the repository, while mercurial represents them as diffs which means a commit will only store changed files. This is a better way to save space but it means Mercurial will be a little slower.

These tools are not the only tools in the market, there are a lot more but these are the most famous ones. If you want more information about these and more version control tools you can visit here

Tools for testing

Testing automation is very used everywhere in the world. This means that it inspired the creation of tools to make automation and other forms of testing a bit easier.

Selenium

Because it is a pain to test web apps in each separate browser to see if it works or displays correctly, Selenium is used to test web applications across various browsers and platforms. You can test in various programming languages and it is said that it supports some of the largest browser vendors in their projects.

Testing whiz

Another all-in-one useful tool, Testing whiz offers various automated solutions that range from web testing to optimization and automation. Although behind a subscription wall, this tool seems pretty useful to save time in testing procedures.

Like version control tools, there are a great number of tools for testing and you are welcome to experiment with different kinds. For more information about this tools you can visit here

Tools for process administration

There are many tools that can used in process adnministration. The main focus of this type of tools is to manage the administration parts of a project, this means user stories, requirements, releases, etc. We have surely heard about process administration tools before, for example monday.com, but there are tools that are aimed specifically at software developement.

Jira

Jira seems to be the most famous tool specifically defined for software developement management. Inside you can create user stories, manage product releases and view your team’s progress. You can use this tool for free up to 10 registered users, after that it will have a cost.

BaseCamp

Basecamp is also one of the more popular choices when it comes to software development management mainly because it can integrate plug-in services that lead to dropbox or anything you need really. The disadvantage of this tool is that it is behind a paywall.

For more information about process administration tools with advantage in software development you can visit here

These are examples of tools used to facilitate the application of verification and validation to the project.

Software Testing

Software testing is the act of testing… software… yeah it’s pretty obvious but the topic still has its trick. The goal of software testing is to provide enough information about what is being tested in order to meet certain requirements.

These requirements commonly being, but not limited to:

  • Meeting the requirements that guided its design and development
  • Responding correctly to all kinds of inputs
  • Performing its functions within an acceptable time
  • Being sufficiently usable
  • Able to be installed and run in its intended environments

Similarly to other topics in this blog, this topic has a process to follow, and this one has a number of steps that can be seen below.

Test Plan: We have seen this before in verification and validation, a test plan can be a document detailing what needs to be tested, how it will be tested, the objective of the test and what to do if the test fails.

Test Design: This will be defined in the test plan; the test design dictates the nature of the test. Will it be a manual test? Will it be automatic? How does it work?

Test execution: In the execution the test will be executed, important things to consider in this point are how much time you have for it and how much time does it take to complete.

Exit criteria: unexpected problems are common in tests; this is why they are here for. So you define what completes the test or completely halts it.

Test reporting: This step will take place once the test has been completed, how well did it go? Was the result satisfactory?

Manual vs automated tests

Manual tests and automated tests each have their disadvantages and advantages, and with a complex code or system the line that separates these 2 types of tests is more evident.

Manual testing is done by a human, giving inputs and receiving what the software returns. This has the advantage that a person is interacting directly with the product which will yield results that can be very specific right of the bat and it is a way of flexibility in the tests.

Automated testing in the other hand is a more robust and straightforward type of test. These are performed by the machine which was preloaded with the necessary instructions. They might not be as flexible and perhaps not be as visual as a manual test, but it is quicker to execute (since you only have to write it one time in order to test 1000 times), it is 100% safe from human errors and can provide quality results, depending on how well it was written.

With this in mind we can now see the various types of tests there are and how they are ordered:

Levels of testing

Unit Testing

During this first round of testing, the program is submitted to assessments that focus on specific functions, individual programs or components of the software to determine whether each one is fully functional. One of the biggest benefits of this testing phase is that it can be run every time a piece of code is changed, allowing issues to be resolved as quickly as possible. It is common for software developers to perform unit tests before delivering software to testers for formal testing.

Integration Testing

Integration testing combines all of the units within a program (most likely previously tested in unit testing) and test them as a group. This testing level is designed to find interface and comunication defects between the modules/functions. This is particularly beneficial because it determines how efficiently the units are running together.

System Testing

System testing is the first level in which the complete application is tested as a whole. The goal at this level is to evaluate whether the system has accomplished all of the requirements and to see that it meets the quality standards of the project.

System testing is done by testers who did not play a role in developing the program. This testing is performed in an environment that closely resembles production.

Acceptance Testing

The final level, Acceptance testing (or User Acceptance Testing), is conducted to determine whether the system is ready for release. During the Software development life cycle, requirements changes can sometimes be misinterpreted in a fashion that does not meet the intended needs of the users. During this final phase, the user will test the system to find out whether the application meets their business’ needs.

For more types of testing visit here: https://www.softwaretestinghelp.com/types-of-software-testing/

Activities and roles in testing:

We’ve seen till now that testing is quite the important task in order to maintain quality in the software, but another important aspect is the people in charge of the tests. There are quite some roles involved in the process of testing, each functioning differently from one another to cover the most aspects possible of the product.

The types of roles involved in testing are shown below:

QA Leader: The face and manager of the testing team; this position presents the connection between the team and whoever wants to contact it

  • Acts as a point of contact for inside and outside interactions, meaning it represents the entire team when dealing with customer relationships
  • Decides about test budget and test schedule, also manages resources and plans the testing process
  • Delegates the testing activities to the team
  • Makes status reports of the testing activities

Test Lead: Highly intelligent and wise, this position will provide great understanding of technical skills such as data management, test design and test development

  • Has technical expertise regarding test programs and how to approach them
  • Provides support for customer interface and will deliver progress status reports.
  • Validates the quality of testing requirements. (Testability, test design etc.)
  • Implements the test process
  • Ensures documentation is complete

Test engineer: This position’s job is to determine the best way to create a process to test a product in the most complete manner possible. There are different roles a test engineer can have, and its area of expertise will depend in which role it takes.

  • Usability Test engineer: Best suited for designing usability testing scenarios and has great understanding of usability issues.
  • Manual Test engineer: Great understanding of the GUI design and its standards. It is best suited for manually testing and attending test procedure walkthroughs.
  • Automated Test engineer: Great understanding of software testing and GUI design. It is best suited for working with test tools in order to build and execute automated tests.

Network Test engineer: Great understanding of databases, operating systems, programming languages and networking. They are best suited for integration testing since they know a little about everything.

Test library and Configuration Specialist: This position will be in charge of managing changes and version control.

  • Manages changes to the test-script
  • Maintains test-script version control
  • Creates test builds whenever required

Tester: This position has to be able to interact efficiently with the testing team, it will design scenarios that would require testing and will execute the tests as defined in the standards and procedures.

  • Designs the testing scenarios for usability testing
  • Analyzes test results and submits reports to the development team
  • Conducts testing as marked in the standards and procedures

Testing environments

We already know about the process of testing and the roles to be taken to accomplish the task, but we still need to know some stuff that will help testing greatly. This is testing environments.

A test environment is a combination of hardware, software, data and configurations which are required in order to test different test cases. These environments are commonly found in the shape of servers and virtual set ups which can be replicated. A testing environment is a very important tool to have at disposition if you want to have confidence in the test results and jump from saying “Well it works just fine in MY computer” to saying “It works on all operating systems with these minimum specifications”.

There are types of testing environments which focus on changing and configuring different aspects of the software:

Integration testing environment

In this environment different software modules are integrated to form a system, and then the test will proceed to verify its behavior.

Environment setup depends on the type of product that wants to be tested. It usually consists on ensuring availability of the right hardware, the right software version and the right configuration. The trick on this environment is that it should mimic production scenarios as closely as possible. This includes servers, databases or any other service required to be involved. This environment is focused in testing communication and functionality between all the components, which is why every other aspect of the environment is set to work in optimal conditions.

Performance testing environment

In this environment the system’s performance will be tested. This means that performance goals like concurrency, throughput, response time and stability will be put to test. A good performance testing environment place a crucial role in defining the bottlenecks in the system tested. What component is slowing down things or what thing breaks if too much stress is bestowed to it are questions this test environment will focus on answering.

The test will vary in the specifications of the hardware, amount of concurrency in the system and the volume of data managed.

This type of test is time consuming and expensive compared to other  tests, which is why it is recommended to run this tests only in required occasions, for example every major change in the system or once a month.

Security testing environment

Data protection Cyber Security Privacy Business Internet Technology Concept.

This type of testing environment focuses on finding security flaws and vulnerabilities. To create this environment organizations usually contract internal or external security experts that will try to identify any vulnerability in the software.

 A good security testing environment will try to be isolated from external sources and try to use mock data that is not in production.

Chaos testing environment

The chaos environment focuses in trying to make the product fail in any point of the execution and make that failure cascade into making the whole system fail. This is done by understanding the dependencies within the project. These environments are set just the same as a production environment and trying to combust the server into a state of error recovery phase, It  is kind of when schools make a fire drill to see the reaction of the students and their exit time. Because of the nature of chaos testing environment, it is executed as rarely as the performance testing environment. Very often they are tested alongside each other.

Test case design techniques

There are different types of techniques that are used to test the software’s functionality. These techniques can be categorized into 3 types

Specification based technique

Also known as Black-Box technique, this type of technique consists of testing based on defined specifications and test cases, this will provide the tests of this nature with a very wide coverage in test cases. One thing to note is that this technique does not use any information regarding the internal structure of the component or system to be tested.

Structured based technique

Also known as the white box technique, the structured based technique takes in consideration the internal structure of the tested components and it is used to derive test cases deemed necessary.

Experience based techniques

This technique will depend on the experience of the team using it. The knowledge and experience of people are used to derive test cases and it is more useful in identifying special tests not easily captured by formal techniques.

For more information you can visit this link

Process for control and management of defects in artifacts

The only thing left to see about software testing is how we can manage the defects we find with the previous knowledge. This is as simple as following some steps in order to classify the defect and resolving it.

Discovery

In the discovery phase, the project team tries to discover as many defects as possible before anyone else can discover them. A defect is said to be discovered and change to status accepted when it is acknowledged and accepted by the developers.

Categorization

Defect categorization helps the software developers to prioritize their tasks. There are types of categories as seen below and developers will categorize defects based on the effects they have on the product.

Resolution

Resolution will stand for the process of fixing the defects. Defect resolution process starts with assigning discovered defects to developers, then defects are fixed and developers send a report of resolution to the test manager.

Verification

After the development team fixed and reported the defect, the testing team verifies that the defects are actually resolved.

Closure

Once a defect has been resolved and verified, the defect is changed status as closed. If the defect has not been resolved the defect has to be checked again in the resolution step with the new observations.

Reporting

Test managers prepare and send the defect report to the management team for feedback on defect management process and defects’ status.

Software Review

Software Review is, as its name implies, a process in which people of different involvements in a project examine the software to see if it meets the necessary requirements of approval. Software review is also part of the life cycle of software development! So it is important for the project to pass through this phase in order to become stronger.

The reviews are actually done by multiple people each of them checking different things for different purposes, each of them checking different aspects of the product. We have project personnel, managers, users, customers, representatives and more giving feedback and giving signs of approval for the project.

Knowing about this topic will increase efficiency in the process of testing and validating the software’s functionality and behavior and It will improve productivity since potential errors are being detected early.

Types of reviews

Software review can be divided into three main types:

Software peer reviews

It is conducted by the creators of the software in order to evaluate the technical content and quality of the work. Checking quality of the software and finding potential errors and defects are the two main actions that take place in this type of review. Peer reviews actually have some subtypes that serve different purposes!

Code review: This is as simple as it gets, it’s a systematic examination of the computer source code.

Pair programming: In this type of peer review, two or more persons develop code together at the same workstation, and then they will review each other’s code.

Inspection: It is a form of formal peer review in which the person follows a well-defined process to find defects in the code.

Walkthrough: In this type, the creators of the software will lead other members of the team to go through the product; the ones viewing the product will act questions and make comments regarding defects that they may find.

Technical Review: It is a form of review in which qualified personnel other than the creators review the software product and see if they find defects regarding usability and problems with specifications and standards.

Software management reviews

They are conducted by a management team directly responsible for the project in order to evaluate the status of the product (how much has been done, what is NOT done yet) and will then direct a course of action depending on the project’s status and schedule. This type of review can be conducted by stakeholders.

If you are doing this type of review, remember to:

  • Check consistency with deviations from plans.
  • Check the adequacy of the management procedure.
  • Access Project Risks.
  • Evaluate the impact of actions and ways to measure those impacts.
  • Produce a list of action items and issues to be resolved and decisions made.
  • The retrospective is important.

Software audit reviews:

They are conducted by external people in order to evaluate if the project can achieve its imposed set of criteria, that being in terms of standards, specifications and agreements. The results of this review include observations, recommendations, corrective actions and a pass or fail assessment.

Software review steps

  1. Entry evaluation: The Review Leader uses a standard checklist of entry criteria to ensure that optimum conditions exist for a successful review.
  2. Management preparation: Responsible management ensure that the review will be appropriately resourced with staff, time, materials, and tools, and will be conducted according to policies, standards, or other relevant criteria.
  3. Planning the review: The Review Leader identifies or confirms the objectives of the review, organises a team of Reviewers, and ensures that the team is equipped with all necessary resources for conducting the review.
  4. Preparation: The Reviewers individually prepare for group examination of the work under review, by examining it carefully for potential defects, the nature of which will vary with the type of review and its goals.
  5. Examination: The Reviewers meet at a planned time to pool the results of their preparation activity and arrive at a consensus regarding the status of the document (or activity) being reviewed.
  6. Exit evaluation: The Review Leader verifies that all activities necessary for successful review have been accomplished, and that all outputs appropriate to the type of review have been finalised.

Reviewing key products

Reviewing the key work products is done through the different types of peer reviews listed above, of course each review checks different products.

Plans: These are checked in all types of peer reviews, checking omissions or inadequacies must be a part of the review process in any of them.

Requirements: There are checked in the walkthrough review only since the requirements are not imposed solely by developers.

Design: This is checked in the walkthrough and inspection review since the design should be following certain guidelines and should be getting feedback by a group of involved people.

Code: This is surely checked in all types of reviews, but it is only truly checked when the walkthrough review is executed since it is checking for standards and specifications.

For more information you can visit here!

Verification & Validation

Verification and Validation (or V&V in short) is the process of investigating and checking in which degree does software satisfy specifications and standards given because of the necessities of the project, and also makes sure it can successfully complete the tasks it was intended to do.

Verification and Validation are two completely different things

For more information about definitions and diferences you can check this link

Verification (Are we building the product right?): It is the process of making sure the software verified can do the tasks imposed without any bugs. This can be resumed in making sure it has no errors, no weird bugs and functions as it should.

Validation (Are we building the right product?): It is the process of making sure that the software can actually fulfill the requirements imposed to it. It is the process of checking that the software created is actually the software we want or need.

To verify and validate there is a common, pretty general but recommended procedure to follow.  Inside the software life cycle, the Verification has to be followed by the Validation. Meaning we first verify if our software actually works and then we validate that our software is actually useful. If we follow the steps of software creation in the image below we can see where is the verification and the validation taking place.

International standards for V&V of Software

To properly and effectively verify and validate your software there are very helpful global standards you would want to follow in order to reach a global acceptance in your procedures and quality of software. There is a great quantity of standards that will help in the application of V&V to any software (ISO 17029, IEEE 1012 and IEC being the most famous ones) each of them varying in methodologies to standardize the process of V&V.

Planning V&V

Having a chosen standard to have for your verification and verification processes is one thing, but another important aspect of V&V is the planning of implementation you will follow to integrate it to the software.

It is very important to have a plan for V&V at the very start of the development, and so it is recommended that you write down a verification and validation planning document containing the details about what is to be tested and how. This is called a master plan.

 Below there are some steps you can follow to build such document.

  1. Describe and set the objectives: Defining the overall project objectives, milestones and activities for teams to do is a key step in the start of the document.
  2. Define the components you are going to test: Listing the software features you’re going to test in the document will serve as a guide for everyone involved on what needs to be verified and validated. It is also important that you specify tests ranging from a small component to a cluster of them.
  3. Define the components you are NOT going to test: There may be instances of components in which testing is really expensive and not very rewarding, so it may be better to skip extensive testing for the components that seem of this type. Including them in the document will clear misconceptions and set on-point expectations about the process
  4. Define how can the components be verified and validated: Have a clear definition of what makes a test case pass and what makes it fail the verification and validation. What criteria they need to pass or what cases should they be able to manage to handle.
  5. Define procedures in case of failure: Often the components can fail some tests. It is important to settle what happens when we are presented with such condition. Can other tests be executed? Do we need to stop and adjust the component before continuing?

With these steps a solid V&V plan will be created ready to be followed, but executing a plan is always a totally different challenge than creating it. AS the old saying says “No plan survives the contact of with enemy”. Administrating a V&V plan will then be our next step.

Adopting a V&V plan

Adopting V and V on a project requires the development of a team and system to capture project requirements, ensure they are understood, and distributed to all staff. The process must be embedded in design and construction processes. There is little value in establishing a V and V team located separately, producing paperwork that remains unused.  Communication is critical.

The requirements list allows monitoring of progress against specifications and criteria. It provides site engineers with a tool to allow them to check the construction and provides a baseline against which changes are evaluated. Site engineers record the emerging evidence showing compliance with the requirements. This is done through site inspections, test reports, measurements, and photographs.

For more administration tips visit here