Tools for V&V

Verification and validation has existed for a while now, this is good for us because people have created many tools to make our jobs way easier. Testing nowadays is much easier and less exhausting than before and I will tell you of some of the most popular tools used for verification and validation. These tools each have their purpose and beware, they are not the only options out there, I will simply talk about the ones I’m most familiar with.

First, I would like to talk about git. Git a tool used for version control. If you are unfamiliar with this term, version control is a system used to record changes to a file or set of files over time. Git is probably the most popular tool used in V&V because of its usefulness when doing any kind of project. If you ever need to go back to some specific version of your files, you can. Let’s say you accidentally mess up something in your code by accident, with git you can simply go back to the last version where your code worked and not lose all your progress. It also allows people to work simultaneously on one project, letting you put everything together once the separate tasks are done. And if any conflicts are detected, it helps you fix them to ensure that the project works properly. There are other tools that can be used for version control but I would be lying if I said I’ve used them. Some of the most popular ones I found where Mercurial, CVS, and SVN, among others. From my experience, this is an essential tool and taking the time to learn how to use it properly is extremely worth the time, as it will make your life easier and your projects, in a way, quicker to manage.

From what I understand, it is harder to talk about only one testing tool since there are many out there used for very different purposes. We have test management tools, test execution tools, test data preparation tools, and the list goes on. In your projects, you will be required to use several different tools depending on what you want to accomplish. In most cases you are responsible for creating the test cases, these tools are meant to help you have a controlled place with all your tests on a particular part of the project and they can also help you automatize your tests. This is especially useful, it saves you time and once the test are done you know your code will properly do its job, that is if the tests were written correctly of course. Some testing tools can also be used while you are writing your code, rather than when you’re done. A very popular tool and one that I’ve used before is Postman. Postman helps you test your API to make sure that your endpoints are working properly. As I stated before, you must identify the parts of your project that need to be tested before you can decide which tools are required. Fortunately for us, there are tools for any kind of process imaginable when developing software, so it should only be a matter of investigating which appeals the most to your needs.

The last kind of tools I wanted to talk about were tools for process administration of V&V. The only problem is that I’ve never really had much experience with any tools of this sort, at least not that I’m aware of. What I know is that these tools, unlike the other tools that focus on a specific part of your projects functionality, are meant to help you in the process of validating and verifying your project.

Whatever your project may be and its purpose, you will always benefit from using tools to help you verify and validate its functionality. No matter which technologies you use, you should always look for a way in which you can prove that what you did is working as intended, otherwise there might be more problems in the future. Releasing code that you know works, even if it takes more time, is always better than releasing something that you didn’t fully test and having to fix it later.

External links:

Version control

Software Testing

The software testing process, although not exactly the same for every methodology, can be generalized to contain the main and most important points that help us have an effective testing experience.

First we need to define the test strategy and test plan. Some of the things you can use to describe your project’s testing scope are:

  • Systems that need to be tested
  • Focused features and functions of the project
  • Non-functional requirements
  • Tools
  • And many more

With no strategy and plan, every project will find it difficult to be productive. This will help you define your entry and exit criteria for testing, which are important as a control for your team. If requirements lack specificity, the will not enter testing. If tested code doesn’t meet the specific quality standards, the code will never move onto the next phase.

Second, you need the test design. In here you will have a collection of test cases necessary to validate the system. Test design relies on test experience, testers’ knowledge of the system being tested and prevailing testing practices. It is focused on uncovering and fixing major bugs, rather than any other superficial aspects or flaws.

The third phase is test execution. There are many ways in which you can execute tests, for this step you will have to adapt your testing to the adequate amount of testing until you are sure that your system is fully functional and mostly bug-free. Understanding your test environment requirements is key to be able to decide your testing strategy.

Finally, we have the test closure. Here is where your exit criteria comes into play because it signals the completion of the test cycle and readiness for your product’s release. To ensure test closure, all your requirements must be met, both technical and business. You need to have passed a minimum percentage pass rate, best practice is targeting 90% of your tests passed. Every critical defect must be fixed. These steps are just a general view of how your testing process should look like.

There are four levels of software testing that you must take into account. These are:

  • Unit testing: Program is submitted to assessments that focus on specific components of the software to see if they are fully functional. The purpose of this round of testing is to determine whether the software works as designed. A unit could refer to a function, an individual program or a procedure.
  • Integration testing: This allows individuals the opportunity to combine every unit in a program and test them together. It is designed to find interface defects between every module or function.
  • System testing: The system testing level is the first in which the application is tested as a whole. Its goal is to evaluate whether the system has complied with all of the requirements and to see if it meets the quality standards.
  • Acceptance testing: This is the final level of testing and is used to determine whether a system is ready for its release or not. Requirements may change during the development cycle so you need to test in order to see if the process meets the business’ needs. After this level is passed, the software can be delivered to production.

There are several roles and activities to be done in testing, each of which have certain responsibilities that need to be done. The main roles are:

  • Test manager: The test manager is usually hired when there are many testing groups. Some of the major roles of a test manager are: preparing test strategy, preparing the test budget, defining the levels and test cycles, developing the strategy for test documentation, metrics and reporting, and many more.
  • Test leader: Every testing group must be led by a test leader. The test leader performs the roles of test manager when they are absent and have their own responsibilities including: preparing the test plan at each test level, defining the objectives, assigning roles and providing schedules to testers, etc.
  • Testers: Testers group can have different levels of testers and roles like performance testers, automation testers, etc. Some of their responsibilities are: gathering the test requirements, set up and verify the test environment, automate the tests, and many more.

If you would like to see a more extensive list of the activities and responsibilities that each has, check the external links below.

Something we need to address too are testing environments. Testing environments are a setup of software and hardware in which the testing teams execute test cases. They support test execution with hardware, software, and a network configured. Test environments are configured as needed by the application under test. It is important to have them properly setup in order to ensure that the software is being tested successfully. There are many things to be taken into account when setting up your test environment and in order to avoid this blog becoming too long, I will leave a link below explaining what you must do in order to assure that your process environment is adequate.

Another important point you need to understand is software testing techniques. Their purpose is to help you design better test cases. They help reduce the number of test cases to be executed while increasing test coverage and helping you identify test conditions. Some of the most important testing techniques are:

  • Boundary Value Analysis (BVA)
  • Equivalence class partitioning
  • Decision table based testing
  • State transition
  • Error guessing

As you can see, software testing is an extensive topic that requires research in several different branches in order for you to successfully do your testing. It is important for you to at least have a general idea of what their purposes are and to follow them if you want your test process to be much more effective and fluent.

External links:

Testing process

Levels of testing

Activities and roles

Testing environments

Testing techniques

Software Review

Software review is an extremely important technique used to assure the quality, functionality, and other features of your software. Unlike many of the other techniques for assuring quality that I have talked about in this blog, software review doesn’t follow an exact formula or a standard. I believe it gives you more freedom and in many cases depends on the person coding and the reviewer. What software reviews enable is the opportunity for people in your team to test your code and tell you of possible errors, improvements, etc.

Some of the most common software review practices include:

  • Code review: Systematic examination of the source code used to fix mistakes and remove vulnerabilities.
  • Pair Programming: In this type of code review, two programmers work together on one workstation and develop code. More people working in one piece of code enables the opportunity to bring up ideas that only one person might not have and gives more than one point of view for solving problems which is always good.
  • Walkthrough: For this review, a developer leads a team of software developers to go through a software product. Questions are asked and necessary comments are made about defects and errors.
  • Technical review: A technical review is more formal, qualified personnel review the software product and examine if its suitable enough for its intended use.
  • Inspection: This is another formal type of peer review in which experienced and qualified individuals examine software for bugs and defects using a defined process.

These were mainly software peer review techniques, there are other types of reviews used in later stages by management representatives. They are mainly used to evaluate work status and decisions regarding downstream activities are taken from there.

One final type of review I will talk about is audit reviews. These are external reviews in which one or more auditors that are not from the development team conduct an independent examination of your software product. Done by managerial level people.

A software product is not only comprised by code but includes everything related to the product itself like plans, requirements, design, etc. What I explained above is mainly focused on code review but there are also techniques you can use to review all the other key work products in your software. What is important to understand is that reviews will always (or at least should) come from people from that specific area that is being reviewed. You can’t ask someone from the design team to review someone’s back-end code. So reviewing these other key components of a software product is pretty much the same. Although I am sure there are other specific types of reviews, standards and techniques used to review them, the basic idea is to have someone else to help you find errors and improvements. Reviews don’t have to be as specific or strict as they are described above. A designer might have no use for “pair programming” but taking the core idea and using it from a design perspective would mean having someone help you design your software and make decisions as you start thinking about solutions for the product. Another example are technical reviews, people focused on the budget area might want to review your budget plan before moving forward. Does the fact that it doesn’t have code make it different from the technical review I talked about earlier? Not really, the core ideas are the same.

In conclusion, software reviews are extremely important when it comes to fixing errors and helping improve your software. They are not limited to code and the basic idea of most if not all the reviews can be translated into other areas to make sure that you are producing software to the best of you and your company’s ability.

External links:

Software review

V&V

In order to understand verification and validation of software we must first define it. According to the IEEE, V&V is the process of determining whether:

  • Requirements for a system are complete and correct.
  • Products of each development phase fulfill the requirements and conditions imposed by the previous phase.
  • Final systems comply with specified requirements.

In other words, V&V is the process of eliminating software defects from every part of the software life cycle, on a stage by stage basis before testing.

As with pretty much everything regarding software quality, there are a lot of perspectives and perceptions as to what V&V is. This is why having a definite international standard is hard, there are a lot of existing definitions and each vary in some way. As stated before, the IEEE has defined a technique for V&V but there are other alternatives like the ISTQB or the HTB one. As for approaches, V&V can be used in different ways: testing, life cycle frameworks, and life cycle methodologies. Testing is pretty self explanatory, life cycle frameworks are the use of techniques to help eliminate software defects in each stage, as stated before. The difference between this and life cycle methodologies is that life cycle methodologies are streamlined to include nothing but the most effective and minimum software defect elimination techniques that are recommended by the life cycle frameworks. This is just a quick overview of these approaches, I hope it is enough to understand the basic difference between them. If not, I encourage you to look further into it as the differences in them are more than what they actually share.

Planning is the first step for a successful use of this technique. There are many things to consider, I found this link with a document describing the outline for V&V planning and the recommended content you should look for in it. As you can see, it is a very simple outline but does a good job explaining how the planning is divided and what you should be doing for each step. Administering

External links:

V&V

Planning

Models and Standards

In a past blog I talked about software quality and how we can define it. Tl;dr quality depends on what the company and/or programmers believe is quality. Everyone will have a different vision of quality, but today we will talk about models and standards. Software quality models were created to solve this ambiguity problem. With them, we can standardize the measuring of software so that we know the quality of our product. Now I will talk about some models and explain what their purpose is.

CMMI: Standing for Capability Maturity Model Integration, CMMI itself is an institute which provides several different software quality models focused on different points. For example, there is a model focused on the development of quality software products, one focused on services, and more. What the CMMI institute claims is that many companies and organizations do not have a process to measure their capabilities against best practices, meaning that they can’t see which capabilities are giving the best performance and which ones should be improved. When you know the capabilities of your organization you can bring many benefits not only to the customers of the products they create, but also to company and the employees themselves.

TSP/PSP: Team Software Process (TSP) and Personal Software Process (PSP) can be used to help organization teams be self-directed and make them better at planning, tracking, establishing goals, and owning their processes and plans. TSP itself is meant to guide engineering teams through the development of software products and it helps the organizations establish good engineering practice. PSP is focused on a personal framework for engineers in a software developing context. This model was created by the Software Engineering Institute at Carnegie Mellon University.

ISO-15504: The ISO-15504 is an international standard for software process improvement capability determination. This is yet another model to help organizations measure their capabilities when developing software products. Similar to the CMMI, its purpose is to set a standard for software process evaluation and let the organization evaluate their current performance. It has a set of requirements to evaluate the processes of an organization which can be generalized into: process evaluation, process improvement, and evaluation of the capacity/maturity of processes.

MOPROSOFT: MOPROSOFT is said to be an easy to understand and adopt model which is more practical than other standards thanks to its shorter length. It is also meant to facilitate the fulfillment of requirements from other models, including the ISO-15504. This model is better for organizations with the structure used in Mexico for software development and maintenance. It also provides a low cost solution for organizations and can be either established from zero or adapted in case the organization already had established processes beforehand.

IDEAL method: The IDEAL method does not refer to a software quality model, but rather a decision making tool to help you look at a scenario from an objective perspective and define the problem and explore solutions. Its purpose is to set emotion aside and make the process of decision making action drive, all while reducing the fear and anxiety of taking a decision, encouraging thinking, and being easy to use for everybody. IDEAL stands for the set of steps that one must follow in order to use this method effectively. First you must Identify the problem, then Define your goals, after that you can start Exploring solutions to later Act on it, and finally, Look back. This method can be used in many contexts, not just the software quality field and will help you improve life decisions when used correctly.

These models have different standards and qualities to help you measure and improve the process and quality of your software developing organization. Choosing one will be up to you based on your needs and goals. This was only a brief summary of each of them so I recommend you investigate more about each one (and any others that weren’t listed here) before deciding which you want to use.

External links:

Software Quality Models

CMMI

TSP/PSP

ISO-15504

MOPROSOFT

IDEAL method

Defining Software Quality

Assuring product quality is crucial for every serious business. The problem with software is that it is harder to define quality than it is with physical products. Let’s take a shirt for example. How can we say that a shirt is of good quality? Well our shirt is comfortable, it’s durable, and it has a good design. Of course we could go into more thorough details like “the fabric was made with only the best of cotton which came from plants grown in the optimal coordinates with the perfect temperature and amounts of water” but we don’t need all of that to say that our shirt is of good quality. So, what makes good quality software? Is it the design of the software? Is it the speed at which the software operates or how reliable it is? And is it possible for software to have all of these qualities?

The first thing we need to understand to answer these questions is that every company, regardless on what product they make, will have a different definition of quality. This doesn’t mean we can’t have common key concepts for defining quality. Among the most popular ones are:

  • Design
  • Functionality
  • Reliability
  • Consistency
  • Durability

But of course, many others exist.

Image taken from asq.org

Let me say this right now, no piece of software will ever be perfect. You can try to excel at all of the qualities of software, but ultimately, it is impossible to not have any flaws. The importance of each will vary greatly depending on what exactly you are trying to provide to your customers and what the purpose of the software is. Focusing on what you want the end product to be could become dangerous and counterproductive. When you focus on the process rather than the result, you get more control out of what you have right now, you get to experiment, and it will put aside things that may distract you from what you are currently doing. Why bother stressing over something you haven’t even gotten to yet when you could be giving all your attention to the process that’s currently going on.

Now that we have accepted the reality and know that no software will be perfect and that we should focus on what we have and what we are doing rather than what we want in the future, we still must answer the question, how can we assure quality in software? As stated before, we must have knowledge of what we want the purpose of our software to be. Is it supposed to help people complete work tasks? Then maybe we should focus on functionality and reliability. Is the software an entertainment mean for the general public? Then design and functionality could take a higher priority than the other qualities. Once you know this, deciding whether or not your software is of good quality becomes much easier.

For those who are still unconvinced or unsure as to what quality in software means, I have good news for you. Thankfully, people have asked themselves these questions for many years, that’s why they decided to create standards. Standards exist to make our lives easier when checking for quality in software. Several standards exist and it is up to you to decide which suits your idea better to use as a guideline for ensuring quality. I will be talking more about these standards and their purpose in another blog post.

External links used:

What is Software Quality?

Focus on process, not outcome

Software Quality Standards