Tools for V&V

Verification and validation has existed for a while now, this is good for us because people have created many tools to make our jobs way easier. Testing nowadays is much easier and less exhausting than before and I will tell you of some of the most popular tools used for verification and validation. These tools each have their purpose and beware, they are not the only options out there, I will simply talk about the ones I’m most familiar with.

First, I would like to talk about git. Git a tool used for version control. If you are unfamiliar with this term, version control is a system used to record changes to a file or set of files over time. Git is probably the most popular tool used in V&V because of its usefulness when doing any kind of project. If you ever need to go back to some specific version of your files, you can. Let’s say you accidentally mess up something in your code by accident, with git you can simply go back to the last version where your code worked and not lose all your progress. It also allows people to work simultaneously on one project, letting you put everything together once the separate tasks are done. And if any conflicts are detected, it helps you fix them to ensure that the project works properly. There are other tools that can be used for version control but I would be lying if I said I’ve used them. Some of the most popular ones I found where Mercurial, CVS, and SVN, among others. From my experience, this is an essential tool and taking the time to learn how to use it properly is extremely worth the time, as it will make your life easier and your projects, in a way, quicker to manage.

From what I understand, it is harder to talk about only one testing tool since there are many out there used for very different purposes. We have test management tools, test execution tools, test data preparation tools, and the list goes on. In your projects, you will be required to use several different tools depending on what you want to accomplish. In most cases you are responsible for creating the test cases, these tools are meant to help you have a controlled place with all your tests on a particular part of the project and they can also help you automatize your tests. This is especially useful, it saves you time and once the test are done you know your code will properly do its job, that is if the tests were written correctly of course. Some testing tools can also be used while you are writing your code, rather than when you’re done. A very popular tool and one that I’ve used before is Postman. Postman helps you test your API to make sure that your endpoints are working properly. As I stated before, you must identify the parts of your project that need to be tested before you can decide which tools are required. Fortunately for us, there are tools for any kind of process imaginable when developing software, so it should only be a matter of investigating which appeals the most to your needs.

The last kind of tools I wanted to talk about were tools for process administration of V&V. The only problem is that I’ve never really had much experience with any tools of this sort, at least not that I’m aware of. What I know is that these tools, unlike the other tools that focus on a specific part of your projects functionality, are meant to help you in the process of validating and verifying your project.

Whatever your project may be and its purpose, you will always benefit from using tools to help you verify and validate its functionality. No matter which technologies you use, you should always look for a way in which you can prove that what you did is working as intended, otherwise there might be more problems in the future. Releasing code that you know works, even if it takes more time, is always better than releasing something that you didn’t fully test and having to fix it later.

External links:

Version control

Verification and Validation of Software

It is time to talk about two very important concepst within software quality, and those are Verification and Validation (V&V). But as always, these terms have to be defined first through the software engineering approach. So let’s start with Verification:

“Confirmation by the analysis of objective evidence that proves that a product, service or system fulfills specified requirements.”

So basically, to ensure that a product meets its requirements according to the team for the customer. Now, with Validation:

“Confirmation by the analysis of objective evidence that proves that a product, service or system fulfills specified requirements for an specific use or application.”

In other words, to ensure that a product accomplishes its intended use in its intended environment. There can be a confusion between those two terms, but worded like that, its specific purpose can be differenciated easily.

V&V in the life cycle of software development

Even though V&V is constatly referred to the testing phase of the life cycle, it is essential across all phases:

PhaseVerificationValidation
RequirementsIt should be verified that each requirement is doable, in scope, testable, as well as having a purpose and being unique.Requirements should be validated with the client to understand them better and eliminate ambiguity.
Architecture and DesignThe architect and designer should verify that the product meets the non-functional requirements and the scalability.Everything regarding design and architecture should be validated with the client.
ImplementationUnit and integration tests are done to the codem as well as peer review and static code analysis.Requirements should be checked with the client, and since this is the phase where major changes are done, these changes should be validated too.
TestingVerify that everything is working as expected, all test cases have been passed, and designs and non-functional requirements are met.Ensure that the customer accepts the product. That all requirements are met, it is ready for use, and all validation and error messages are clear.
DeploymentAuditing of installation. Make sure everything works as expected with each release.Make sure that installation and configuration is working, also that integration had no problem with internal and external systems.
MaintainanceVerify that everything is fine within integration.Ensure that everything runs smoothly, and understand errors in order to create new requirements.
International standards for V&V of software

There are many standards for V&V, that includes ISO/TS 17033, ISO 14021, IEC, many from IEEE, and countless others. And each one has very detailed documentation that describes each method with detail. There are some that would be great to discuss a little bit more:

  • ISO/IEC 17029: Provides requirements for validation and verification of ethical processes with a focus on environmental issues. Even the software industry should take these aspects into account. It describes how verifiers can avoid conflict of interest, ensure the competence of the verifiers. It verifies ethical claims, sustainability reports, and validates due dilligence approaches and environmental claims.
  • IEEE 1012-2016: Processes are used to determine if the development products satisfies its intended use and user needs, in every part of the software development cycle. Applied to software, systems, and hardware being developed, maintained, or reused.
Planning V&V

A system validation and verification plan should be stated since the begining of the software development. In order to have a good v&v it is important to plan it taking into account who are going to be the main stakeholders, since everything should be validated with them. As stated in the previous point, every part of the life cycle should take v&v into account, and that requires planning of the needs, goals and objectives.

There are important tools that help the v&v planning:

  • Project Management Plan (PMP): Defines the overall project activities, milestones and events.
  • Systems Engineering Management Plan (SEMP): establishes the overall plan for the technical development of the system.
  • Master Integration Verification and Validation Plan (MIVVP): implements the v&v philosphy of the project.
  • System Requirements Document or Specification (SRD/SRS): contains the system’s requirements.
  • Requirements Verification Matrix (RVM): a matrix that lists the requirement with relevant data such as its verification method, responsible organization, level, phase and requirement rationale, as well as a short description for the success criteria.
  • System Integration, Verification and Validation Plan (SIVVP): implements the project’s MIVVP process for a given component.
Administration of a V&V plan

Although planning is very important, it is not the end of it, the plan should be implemented. And to help administering the v&v plan there are some helpful tools:

  • Verification Requirement Definition Sheet (VRDS): documents the details of the specific verification activities identified in the RVM.
  • Task Definition Sheets (TDS): lists tasks to be performed to accomplish the needed verification activities defined in the VRDS.
  • Task Performance Sheet) (TPS): contains the actual procedure used to perform the verification or validation activities defined in the TDS.
References

Sami, M. (2018). Validation and Verification in SDLC. Mohamed Sami. Retrieved from https://melsatar.blog/2018/08/06/validation-and-verification-in-sdlc/

Verification and Validation. (n.d.). Verification vs Validation. Software Testing. Retrieved from https://softwaretestingfundamentals.com/verification-vs-validation/

Standard for V&V. (n.d.). P1012 – Standard for System, Software, and Hardware Verification and Validation. IEEE. Retrieved from https://standards.ieee.org/project/1012.html#:~:text=Verification%20and%20validation%20(V%26V)%20processes%20are%20used%20to%20determine%20whether,specified%20for%20different%20integrity%20levels.

Illner, C. (2019). VERIFICATION AND VALIDATION: WHY THE NEW STANDARD ISO/IEC 17029 MATTERS. DQS-CFS. Retrieved from https://dqs-cfs.com/2019/10/verification-and-validation-why-the-new-standard-iso-iec-17029-matters/

Wheatcraft, L. (2012). Thinking Ahead to Verification and Validation. Requirements Experts. Retrieved form https://reqexperts.com/wp-content/uploads/2015/07/thinking-ahead-to-verification-and-validation.pdf

V&v plan. (n.d.). System Validation and Verification Plans. UC Berkley. Retrieved from https://connected-corridors.berkeley.edu/developing-system/system-validation-and-verification-plans

Images from https://undraw.co/

Verification and Validation of Software (V&V)

In Software Development, the terms verification and validation are used very often. We use these concepts almost every day while we are working in some project. The thing is, that they have different meanings and many people think that this is a synonym. Let’s define each:

  • Verification: The evaluation of whether or not a product, service, or system complies with a regulation, requirement, specification, or imposed condition. It is often an internal process. Contrast with validation. Simply “Are we building the system right? Concerned with the team for the customer”
  • Validation: The assurance that a product, service, or system meets the needs of the customer and other identified stakeholders. It often involves acceptance and suitability with external customers. Contrast with verification. Simply “Are we building the right system? Concerned with the customer”

How is it used in SDLC phases:

PhaseVerificationValidation
Requirements ElicitationThe analyst should verify that each requirement is unique and it has a purpose and origin, as well he/she verify that the requirements are doable, in scope, and testable.The analyst should validate the requirements with the customer to better understand them through different analysis techniques.
Architecture & DesignThe architect and software designer will verify that the software designs are meeting the non-functional requirements, for example, performance and usability.If the software has a UI, should be validated with the customer and the analyst to ensure meeting the customers’ requirements and their expectations.
Implementation (Coding)Similarly, during implementation and coding, the developers start doing unit and integration test for their code, and they may have peer review and static code analysis.During the implementation, the developers may find part of the requirements is not clear or incomplete, they may need to validate that with the customer.
TestingIn this phase, testing and quality team should test the requirements and ensure that all test cases have been passed, all functionalities are working fine and there is enough and acceptable code coverage.The testing team should ensure that the software meets the business requirements and is fit for use by building the user acceptance test cases and validate it with the customers.
DeploymentAuditing of installation and configuration of the environment, run smoke tests to ensure everything is working fine with each release.Making sure that the software is working after installation and configuration.
Operational SupportVerify that the software is up without downtime, there are no data issues and logs are clear.Validating that the software operation is smooth. There are no logical errors or something is broken, evaluate the changes and new requirements.

International Standard For V&V Regulations

ISO 9001 Quality management systems – Requirements requires verification and validation to be planned and carried out and records kept.

IEEE 1012 – Standard for Software Verification and Validation provides a comprehensive how-to for software V&V planning and execution.

Planning V&V

This action includes the commencement, arrangement of reaction, contract, arranging, execution and control, survey and assessment, and conveyance and finish exercises, honesty levels, and so on.