Author: Sebastián Cedeño González

Tools for Verification and Validation

Now that all the concepts are set, it is time to discuss the tools that can be used for the different stages that were mentioned in the previous entries.

Tools for version control

Version control is useful to keep track of the changes made to a code, is great for working in teams. It keeps the history of all the modifications, and can undo a change if necessary. Now, to discuss some of these tools.

1. Git

Probably the most widely used in current times. Is a free, cross-platform and open source distributed control tool. It is great for both small and big projects. It is fast, has a full history tree, that is also available offline, has a pretty good interface. It is robust and easily maintainable, and has a variety of commands, and even comes with its own command line. There are many cons to use git, although not everything is perfect, since the history log can be hard to understand, does not support keyword expansion and timestamp preservation, and is more limited for Windows users.

2. Concurrent Versions System (CVS)

It is another widely used revision control system, and has been the tool of choice for many developers since the 80s. Although it is not as feature rich as other tools (this doesn’t mean this tool is not robust), it is fairly easy to learn how to use it. It makes it pretty simple to keep all to date, can allow multiple developers to work on a project parallely, has a and even has a pretty big community that gives it support, since it is open source. However, it comes with its tolls; poor support for distributed source control, no integrity check for source code, it has expensive branch operations, and has no automatic check-outs and commits.

3. Subversion (SVN)

Free and open source, and a popular alternative to CVS, trying to fix all the problems that were found in it and at the same time still being compatible with it. It prevents corruption the the database using atomic operations, where all or none of the changes made to the source code are applied, but no partial changes will affect the original code. Also, SVN fixed the cost of branch operations. SVN has better support for Windows, and can use good GUI tools, and even has support for empty directories. It also is easy to administer and to integrate with IDE and Agile tools. But, it does not store the modification time of files, does not support signed versions, is not good for filename normalization, and is still slower than git.

4. Mercurial

It is a free, open-source distributed tool, written in python to directly compete with git. It has a high performance, robustness and scalability. It has advanced branching and merging capabilities, it is decentralized, easy to learn, lightweight, and simple, and handles both plain text and binary files robustly. It also has an integrated web interface. But, all the add-ons must be written in python, partial check-outs are not allowed, and can become problematic with additional extensions.

5. Bazaar

It is a free, open-source, and distributed tool (what a surprise), that scales and adapts according to the needs of the user. It is similar to git, and mercurial, and can be centrilized or decentralized. Provides free hosting services, it is easy to use, has a high efficiency, speed, and supports tracking of directories. Although, it doesn’t support partial check-outs, and does not preserve timestamps.

6. Monotone

It is a decentralized peer-to-peer tool that focuses on integrity over performance. It is intended for distributed operations. It employs cryptographic primitives to track file revisions and authentications. It can import CVS projects, and uses netsync, a very efficient and robust custom protocol. It requires very low maintenance, has good documentation, it is easy to learn, works great with branching and merging, and has a stable GUI. Although it suffers in performance for some operations, and cannot commit or check-out from behind a proxy (due to a non-HTTP protocol).

Tools for testing

Testing tools offer many options to be used: tools for automation, for functional testing, for non-functional testing, and even for agile testing. I will not discuss every single one of them, but I will put a link with a list of 50 of them (or you can check out the references for even more examples): https://www.guru99.com/testing-tools.html#7

1. Selenium

The number one testing framework, to perform various web application tests on many platforsm and operating systems. It can help to create tests for different programming languages: Java, PHP, C#, Python, Groovy, Ruby, and Perl. Offers record and playback features without the need of learning Selenium IDE. It is the base for most of the software testing tools. It is powerful,and supports multiple frameworks like Data-Driven, KeyWord Driven, Cucumber, JUNIT, NUNIT, and Test NG. And the best part is that it is free to use.

2. Eggplant

Alson known as Eggplant’s Digital Automation Intelligence, is primarily aimed at application testing and GUI testing. It offers an AI-driven test automation for functional, usability and performance testing. It can provide user and business focused metrics for quality and impact on the end-user. Eggplant uses and image based approach, so, by using a single script, test on multiple platforms can be performed (Windows, Mac, Linux, Solaris, among others). As a plus, it has a very intuitive interface, although it is not free. Also, it seems to work better on MacOS than Windows.

3. Katalon Studio

These test automation tool covers API, Web, Desktop and mobile testing. It has a nicely sorted set of features for recording actions, creating test case, generating and executing test scripts, reporting results, and many others. It runs on Windows, MacOS and Linux. Also can be integrated with other tools such as Jira, qTest, Kobiton, Git, Slack, etc. It is not free, although offers a free version for individual testers. It is very easy to learn, even for non-QA testers. For now, it is very limited on the programming languages it accepts.

4. SauceLabs

Is a selenium cloud based tool that offers automated testing over cross-browsers and multiple platforms. It has support for mobile and desktop apps. It has a reputation of reducing testing time to a huge extent (it is widely used by big companies). However, the cloud service is a bit slow, and outages in the software causes the odd test to fail. It is priced, but there are free licences for open source projects.

5. OWASP

The Open Web Application Security Project (OWASP) is a tool for security testing, to help projects to be reliable and trusted. It helps to verify security, validate all types of inputs, implement authentication controls and all the appropriate access controls. It is an open source framework, and it offers many guides and documentation to assit the execution of tests.

6. LAMBDATEST

A scalable cloud-based browser testing platform to perform manual and automated tests on a widely number of desktop and mobile browsers. It supports parallel testing.

Tools for process administration of V&V

Here is the awkward moment, in which a lot of time was spent on a topic that is not very specific (first I found an article from 2006, which means that is not going to have an updated list). These tools are used not only for administrating processes, since most of them have more features than just keeping track of all the tasks that were, are, or will be performed. But they are very useful, and widely used, also, since the trend right now is agile development, most of these tools have a focus on agile:

Jira

Jira is a great software management tool that is widely used in many companies. It is very robust and offers many features that help the management of tasks and processes, and it can be integrated with other tools.

Notion

A very innovative tool that serves as a all-in-one workspace. It supports many features, for productivity, design, and of course, task management. An alternative to Jira, and great for managing the status of processes. It also supports markdown.

References

Version Control Tools. (2020). Top 5 Free and Open-Source Version Control Tools in 2020. GeeksforGeeks. Retrieved from https://www.geeksforgeeks.org/top-5-free-and-open-source-version-control-tools-in-2020/

Version Control Tools. (2020). 15 BEST Version Control Software (Source Code Management Tools). Software Testing Help. Retrieved from https://www.softwaretestinghelp.com/version-control-software/

Version Control Tools. (2015). Top 5 open source version control tools for system admins. FileCloud. Retrieved from https://www.getfilecloud.com/blog/2015/02/top-5-open-source-version-control-tools-for-system-admins/#.X7xSG2hKiUk

Testing Tools. (2020). Top 50 Software Testing Tools in 2020. Guru99. Retrieved from https://www.guru99.com/testing-tools.html#7

Satasiya, P. (2019). Top 10 Automated Software Testing Tools. DZone. Retrieved from https://dzone.com/articles/top-10-automated-software-testing-tools

Testing Tools. (2020). Top 20 Best Automation Testing Tools In 2020 (Comprehensive List). Software Testing Help. Retrieved from https://www.softwaretestinghelp.com/top-20-automation-testing-tools/

Boog, J. (2020). 10 Latest Software Testing Tools QAs Are Using In 2020. QA Lead. Retrieved from https://theqalead.com/tools/software-testing-tools/

Process Administration Tools. (n.d.). A story of tools and the future of work. Notion. Retrieved from https://www.notion.so/about

Process Administration Tools. (n.d.). Jira Software. Atlassian. Retrieved from https://www.atlassian.com/software/jira?&aceid=&adposition=&adgroup=109687540304&campaign=10332064761&creative=443576046641&device=c&keyword=jira&matchtype=e&network=g&placement=&ds_kids=p55122863591&ds_e=GOOGLE&ds_eid=700000001550060&ds_e1=GOOGLE&gclid=CjwKCAiA2O39BRBjEiwApB2Iku6tiVe8LVTlnp_BSWfHFts3jV44ueKHzvrTlWiNrZi4Fi5RQeV_OxoCw9EQAvD_BwE&gclsrc=aw.ds

Software Testing

Software testing has been described previously, but only by name, and maybe the stages at which it should be applied; however, this post will be dedicated to it (as if it wasn’t implied by the title). Software Testing can be defined as:

“Process to check if software meets requirements and its devoid of defects by executing software components and/or using automated tools”

Software testing, is great to make a product cost-effective, secure, of quality, and to assure that meets the customers expectations.

Software Testing Process

The stages of process testing can be diffetent for each project, but it can be broke down into 5 basic ones:

  • Planning: the scope of the test is determined, impact analysis is performed, all resources are listed, and dates for testing are planned (of course they are estimates).
  • Creation of Scenarios: expected outputs are prepared. And test scenarios are created.
  • Preparation of Test Environment and Data Creation: all preparations are made in order to start the test. Basically, setting all up.
  • Run the Scenarios: the created test scenarios are run.
  • Reporting the Results: Pretty self-explanatory, check the results and compare them to the expected output, to design the next test cycle.

I should be noted that this is a process that should be applied on all stages of the Software Development Life Cycle (SDLC).

Types of Testing

There are three main categories:

  • Functional Testing: testing the components that are essential for the functional requirements. This includes Unit testing, Integration testing, User acceptance testing, among others.
  • Non-Functional Testing: testing the components that are essential for the non-functional requirements. This includes Performance testing, Endurance testing, Load testing and so on.
  • Maintenance Testing: testing the product in order to keep it functional or improve it. This includes Regreesion testing and Maintenance testing.

Although some examples were given, there are hundreds of different types of testing. Here is a list of them: https://www.guru99.com/types-of-software-testing.html

Levels of Testing

Lists are always welcomed, so of course the different levels of testing are going to be listed. But before it should be noted that this is not a strict set of levels that should always be followed, this is more of a list of most relevant levels that provide a comple testing process:

  • Unit Testing (Level 1): Testing by small blocks of code. For this to work these blocks should be able to be tested separately, and in a considerable small amount of time.
  • Integration Testing (Level 2): Test groups of blocks of code to check if their functionality as a whole meets the requirements.
  • System Testing (Level 3): Test the system as a whole.
  • Acceptance Testing (Level 4): Test user scenarios, to confirm that the user accepts the product.
Activities and Roles in Testing

There is a designated testing team, or at least it should be, that is divided in different roles, the two main ones are, the test lead and the testers. But here, 3 roles will be given:

  • Software Tester Role: designs test case scenarios, conducts them, analyses the results and reports the observations. Interacts with clients to better understand the product, and do modifications, and also creates the documentation. Ensures that all tests are carried out accordingly to standards and procedures. There are many different roles for a software tester, thus, why so many responsabilities were listed.
  • Software Test Manager Role: represent the team, interact with customers, recruit software test staff, supervise all testing activities, schedule testing activities, select test tools, improve test processes with the help of metrics, and check the quality of the requirements.
  • Software Test Automator Role: design procedures and test cases for automated software testing, design reusable automated scripts, and ensure that all automated processes are done by the defined standards.
Testing Environment

A test environment enables the creation of identical environments for testing, in order to control the most variables in the testing stage. A controlled test enviroment can provide useful feedback to the behavior of the application. There are some key elements that should be completed to create the test environments:

  1. Implementation of test beds (data created and inserted to the environment)
  2. Setup of the database
  3. Configure the environment
  4. Select hardware and software to run the application
  5. Configure the network
  6. Document all actions

There is a difference between test environment and staging environment, since stanging environment is the exact replica of the production level of the application, so is used to check if it is ready for deployment. Basically, a test environment is used to test specific parts of the producto while staging environment is for the product as a whole.

To manage the test environment there is also a checklist

  1. A repository with all the updated versions of the environments
  2. Create new environments as required
  3. Monitoring the environment
  4. Updating/Deleting outdated environments
  5. Check for errors in the environments
  6. Coordinate to solve all the errors in the environments
Test Case Design Techniques

There are three main types of design techniques:

  • Specification-Based Technique: a black box technique. These tests are done to know how the system behaves. This is more about creating test cases based on the output.
  • Structure-Based Technique: a white box technique. Tests that are done to have a level of coverage of control flows. Done by code analysis.
  • Experience-Based Technique: this is entirely based on the tester’s expertise. The tester designs tests based on previous experiences, guesses, and different known techniques.

From these 3 types, many techniques can be chosen, such as, error guessing, boundary value analysis, equivalence class partitioning, decision table based testing, state transition, and many others.

Defect Management

Defects, or bugs, are very relevant to software testing, since one of the main objectives of software testing is to reduce them. The defect management process goes as:

  • Discovery: the team tries to discover as many bugs as possible.
  • Categorization: all the bugs are classified, in order to decide the priorities of the team. A critical error should be fixed immediately. A high level error is the next priority, since it affects the main features. A medium level error causes damage that is not urgent, but no insignificant. Finally, a low level bug is very minor.
  • Defect resolution: a process to fix the bugs, divided in four stages:
    • Assignment: assing the bug to a developer to solve it
    • Schedule fixing: schedule by when it should be fixed
    • Fixing: fix the bug, while tracking the progress
    • Report resolution: report what the solution was.
  • Verification: verify that the bug is resolved.
  • Closure: Change bug status to close.

During all process, the bug should reported to the management team for feedback. Now, to measure the quality test execution, there are to parameters: Defect Rejection Ratio (Number of defects rejected over total of defects raised) and Defect Leakage Ratio (Number of defects missed over total defects of software). The value of these measures is better the smaller it is.

References

Software Testing. (n.d.). What is Software Testing? Definition, Basics & Types. Guru99. Retrieved from https://www.guru99.com/software-testing-introduction-importance.html#1

Testing Environment. (n.d.). Test Environment for Software Testing. Guru99. Retrieved from https://www.guru99.com/test-environment-software-testing.html#:~:text=A%20testing%20environment%20is%20a,of%20the%20Application%20Under%20Test.

Test Case Design Techniques. (n.d.). Software Testing Techniques with Test Case Design Examples. Guru99. Retrieved from https://www.guru99.com/software-testing-techniques.html

Software Testing. (n.d.). Defect Management Process in Software Testing (Bug Report Template). Guru99. Retrieved from https://www.guru99.com/defect-management-process.html

Öztürk, M. (2020). Software Testing Process and Levels of Testing. The Startup. Retrieved from https://medium.com/swlh/software-testing-process-and-levels-of-testing-4274904ce655

Software Testing Roles. (n.d.). Software Testing Roles and Responsibilities. Test Institute. Retrieved from https://www.test-institute.org/Software_Testing_Roles_And_Responsibilities.php#:~:text=Test%20engineers%2FQA%20testers%2FQC%20testers%20are%20responsible%20for%3A&text=Develop%20test%20cases%20and%20prioritize,the%20code%20to%20fix%20defects.

Test Environment. (2019). What Is a Test Environment? A Guide to Managing Your Testing. testim. Retrieved from https://www.testim.io/blog/test-environment-guide/

Test Case Design Techniques. (2016). Test Case Design and Testing Techniques. RapidValue. Retrieved from https://es.slideshare.net/Rapidvaluesolutions/whitepaper-test-case-design-and-testing-techniques-factors-to-consider

Software Review

Software review is an essential part to maintain software quality. Before, the processes, standards and models for assuring software quality were presented, but now, it is time to delve into the actual tasks for verification and validation. Starting with the title of this blog post. As always, the definiton of the topic will be needed in order to continue. Software Review can be defined as:

“It is a systematic inspection of code in order to find errors, enhance the product, and to validate its vital features and components”

This serves the purpose of checking if the software is meeting its requirements, and to adjust development accordingly. There are many benefits that come with applying software reviews to all the stages of the Software Development Life Cycle (SDLC), including the necessary software quality, reduction in the time of production and making testing time & cost effective.

There are three main types of software review, that have different roles and are applied in different stages of the SDLC.

Software Peer Review

These reviews are done by the creator of the code, along with other developers. This helps to check the quality of the software, to find defects and solve then. It is the most basic model of review, and that is why is the most used one. There many types of software peer review.

  • Code Review: a systematic evaluation of the code structure is done, against software standards and procedures.
  • Pair Programming: two or more developers work together, and review each other’s code.
  • Informal: a review that is conducted without the need of a large group or the documentation.
  • Walkthrough: this is a well structured review conducted by a lead team member, usually from the quality assurance department. It is done to detect errors in the code. There are diffetent roles in this review:
    • Author: the creator of the code, acts as an observer and answers questions
    • Presenter: is the one that presents the product, and usually develops the agenda.
    • Moderator: the one that in charge of making everyone follow the agenda, and encourages participation from reviewers.
    • Reviewers: they evaluate the work, to make sure that the standards and guidelines are followed, the requirements are met, and the product is on the right track.
    • Scribe: in charge of taking notes, record the detected errors, comments, suggestions, and unsolved questions.
  • Technical Review: a team of qualified personnels examine the software.
  • Inspection: a formal type of peer review, where defined processes are used in order to review the code.
Software Management Review

This review is applied at the later stages of the development by the management representatives. This is to check the work status, and it is done in order to decide the next major steps of the project.

Software Audit Reviews

It is an external review , where auditors that are not part of the development team do an examinatio of the code. This is done by managerial level people.

Process of Software Review

The IEEE defined a standard for the steps that are required in order to create a complete software review process:

  • Entry Evaluation: a check-list by entry criteria tha ensures that all conditions are met to have a successful review.
  • Management Preparation: a management representative checks that the software review has all the required resources.
  • Review Planning: an objective is set, and the team of resources is formed based on it.
  • Preparation: the reviewrs prepare the reviewing tasks.
  • Examination nad Exit Evaluation: the evaluation is done, and reviewers share their results, and verify that all review tasks are done.
Review Key Products

There are important parts of the software development process that should be evaluated: plans, requirements, design, and code. For these for key work products exist different standards that can be applied to each one of them. Although eviews can be distributed diffetently for each key product:

Key ProductPeer to Peer ReviewWalkthrough ReviewInspection ReviewManagement Review
Plans✔✔✔✔
Requirements✔✔
Code✔
Design✔✔
References

Software Review. (2020). Software Review. ProfessionalQA. Retrieved from https://professionalqa.com/software-review

Melo, W., Shull, F., Travassos, G. (2001). Software Review Guidelines. Coppe/UFRJ. Retrieved from https://www.cos.ufrj.br/uploadfile/es55601.pdf

LalSingh, A. (2009). Guidelines to review Work Products. Rass Tools Limited. Retrieved from https://es.slideshare.net/AshokKumar1213/guidelines-to-review-work-products

Verification and Validation of Software

It is time to talk about two very important concepst within software quality, and those are Verification and Validation (V&V). But as always, these terms have to be defined first through the software engineering approach. So let’s start with Verification:

“Confirmation by the analysis of objective evidence that proves that a product, service or system fulfills specified requirements.”

So basically, to ensure that a product meets its requirements according to the team for the customer. Now, with Validation:

“Confirmation by the analysis of objective evidence that proves that a product, service or system fulfills specified requirements for an specific use or application.”

In other words, to ensure that a product accomplishes its intended use in its intended environment. There can be a confusion between those two terms, but worded like that, its specific purpose can be differenciated easily.

V&V in the life cycle of software development

Even though V&V is constatly referred to the testing phase of the life cycle, it is essential across all phases:

PhaseVerificationValidation
RequirementsIt should be verified that each requirement is doable, in scope, testable, as well as having a purpose and being unique.Requirements should be validated with the client to understand them better and eliminate ambiguity.
Architecture and DesignThe architect and designer should verify that the product meets the non-functional requirements and the scalability.Everything regarding design and architecture should be validated with the client.
ImplementationUnit and integration tests are done to the codem as well as peer review and static code analysis.Requirements should be checked with the client, and since this is the phase where major changes are done, these changes should be validated too.
TestingVerify that everything is working as expected, all test cases have been passed, and designs and non-functional requirements are met.Ensure that the customer accepts the product. That all requirements are met, it is ready for use, and all validation and error messages are clear.
DeploymentAuditing of installation. Make sure everything works as expected with each release.Make sure that installation and configuration is working, also that integration had no problem with internal and external systems.
MaintainanceVerify that everything is fine within integration.Ensure that everything runs smoothly, and understand errors in order to create new requirements.
International standards for V&V of software

There are many standards for V&V, that includes ISO/TS 17033, ISO 14021, IEC, many from IEEE, and countless others. And each one has very detailed documentation that describes each method with detail. There are some that would be great to discuss a little bit more:

  • ISO/IEC 17029: Provides requirements for validation and verification of ethical processes with a focus on environmental issues. Even the software industry should take these aspects into account. It describes how verifiers can avoid conflict of interest, ensure the competence of the verifiers. It verifies ethical claims, sustainability reports, and validates due dilligence approaches and environmental claims.
  • IEEE 1012-2016: Processes are used to determine if the development products satisfies its intended use and user needs, in every part of the software development cycle. Applied to software, systems, and hardware being developed, maintained, or reused.
Planning V&V

A system validation and verification plan should be stated since the begining of the software development. In order to have a good v&v it is important to plan it taking into account who are going to be the main stakeholders, since everything should be validated with them. As stated in the previous point, every part of the life cycle should take v&v into account, and that requires planning of the needs, goals and objectives.

There are important tools that help the v&v planning:

  • Project Management Plan (PMP): Defines the overall project activities, milestones and events.
  • Systems Engineering Management Plan (SEMP): establishes the overall plan for the technical development of the system.
  • Master Integration Verification and Validation Plan (MIVVP): implements the v&v philosphy of the project.
  • System Requirements Document or Specification (SRD/SRS): contains the system’s requirements.
  • Requirements Verification Matrix (RVM): a matrix that lists the requirement with relevant data such as its verification method, responsible organization, level, phase and requirement rationale, as well as a short description for the success criteria.
  • System Integration, Verification and Validation Plan (SIVVP): implements the project’s MIVVP process for a given component.
Administration of a V&V plan

Although planning is very important, it is not the end of it, the plan should be implemented. And to help administering the v&v plan there are some helpful tools:

  • Verification Requirement Definition Sheet (VRDS): documents the details of the specific verification activities identified in the RVM.
  • Task Definition Sheets (TDS): lists tasks to be performed to accomplish the needed verification activities defined in the VRDS.
  • Task Performance Sheet) (TPS): contains the actual procedure used to perform the verification or validation activities defined in the TDS.
References

Sami, M. (2018). Validation and Verification in SDLC. Mohamed Sami. Retrieved from https://melsatar.blog/2018/08/06/validation-and-verification-in-sdlc/

Verification and Validation. (n.d.). Verification vs Validation. Software Testing. Retrieved from https://softwaretestingfundamentals.com/verification-vs-validation/

Standard for V&V. (n.d.). P1012 – Standard for System, Software, and Hardware Verification and Validation. IEEE. Retrieved from https://standards.ieee.org/project/1012.html#:~:text=Verification%20and%20validation%20(V%26V)%20processes%20are%20used%20to%20determine%20whether,specified%20for%20different%20integrity%20levels.

Illner, C. (2019). VERIFICATION AND VALIDATION: WHY THE NEW STANDARD ISO/IEC 17029 MATTERS. DQS-CFS. Retrieved from https://dqs-cfs.com/2019/10/verification-and-validation-why-the-new-standard-iso-iec-17029-matters/

Wheatcraft, L. (2012). Thinking Ahead to Verification and Validation. Requirements Experts. Retrieved form https://reqexperts.com/wp-content/uploads/2015/07/thinking-ahead-to-verification-and-validation.pdf

V&v plan. (n.d.). System Validation and Verification Plans. UC Berkley. Retrieved from https://connected-corridors.berkeley.edu/developing-system/system-validation-and-verification-plans

Images from https://undraw.co/

Models and Standards for Software Process Improvement

After discussing some basic concepts of Software Quality, it is time to start talking about some of the models and standards that are used in the software industry. These models exist in order to help the improvement of processes and identify inefficiencies. But before jumping into the models themselves, let’s define Software Process Improvement (SPI). Which can be stated as:

“A set of applied theories, methods and techniques oriented towards quality management in order to achieve higher product quality, greater product innovation, faster cycle times, and lower development costs, simultaneosly.”

SPI consists of 4 main cycle steps: Current Situation Evaluation, Improvement Planning, Improvement Implementation, and Improvement Evaluation. SPI can bring so many benefits to a company, such as cost reductions, process consistency, reduction of time to market, customers satisfaction, automation, autonomy. There is many evidence of for the value that SPI can bring. Of course, it has its tolls that make a lot of companies think very carefully about implementing SPI on their projects. It adds a lot of time pressure and it is not recommended on budget constraints, also, on smaller companies it could be very difficult to measure the improvements on the processes.

Now that a basic understanding of SPI was set, it is time to talk about some SPI methods and standards.

CMMI

“The Capability Maturity Model Integration (CMMI) is a set of best practices”

This model was developed by the Software Engineering Institute at Carnegie Mellon University as a process improvement tool for projects, divisions or organizations.

CMMI starts with an appraisal to evaluate three specific areas:

  • Process and service
  • Service establishment and management
  • Product and service acquisition

It is a process and behavioral model. That means it can not only improve performance by developing measurable benchamarks, but can also encourage productive and efficient bahavior throughout the organization.

Even though the model was shaped for software engineering, it has evolved for hardware, software and service development across every industry, aiming to become easier to understand an use, more cost-effective, and easier to integrate and deploy.

In order to measure the CMMI methods, 5 levels were created:

  • Initial (Level 1): The worst stage a business can find itself in. Work gets completed but it’s often delayed and over budget.
  • Managed (Level 2): There are still a lot of issues to address, but there is an improvement on project management.
  • Defined (Level 3): Projects are guided by organization wide standards, and businesses understand what are the goals for improvement.
  • Quantitatively managed (Level 4): The business is ahead of risks, being more data driven.
  • Optimizing (Level 5): Processes are stable and flexible. The environment is more predictible and the organization is in constant improvement.

Once an organization reaches level 5 of maturity the next step is maintainance and continuous improving.

Certifications for CMMI are offered directly by the CMMI Institute, to certify individuals, appraisers, instructors and practitioners.

  • Individual certifications
    • Certified Enterprise Data Management Expert
    • Certified SCAMPI Lead Appraiser (People CMM)
    • Certified Introduction to People CMM Instructor
  • Appraiser certifications
    • Certified SCAMPI B&C Team Leader Certification
    • Certified SCAMPI Lead Appraiser (CMMI for Acquisition)
    • Certified SCAMPI Lead Appraiser (CMMI for Development)
    • Certified SCAMPI Lead Appraiser (CMMI for Services)
    • Certified SCAMPI High Maturity Lead Appraiser
  • Instructor certifications
    • Certified Introduction to CMMI for Development Instructor
    • Certified Introduction to CMMI for Services Instructor
    • Certified CMMI for Acquisition Supplement Instructor
    • Certified CMMI for Services Supplement Instructor
    • Certified CMMI for Development Supplement Instructor
  • Practitioner certifications
    • Certified CMMI Associate
    • Certified CMMI Professional

CMMI is a model that seeks to create predictable environments, to facilitate the efficiency, proactiveness and productiveness of products, services and departments.

TSP/PSP

“Team Software Process (TSP) and Personal Software Process (PSP) allows an organization to create self-directed teams that establish goals, processes and plans, to track their work”

As CMMI, TSP/PSP was developed by the Software Engineering Institute at Carnegie Mellon University. As the self explanatory names state, PSP is more focused on the individual improvement, while TSP is a team improvement. PSP gathers, registers and evaluates data for the improvement of the individual processes of a member to help the whole team. TSP takes the concept of PSP and applies it to the whole team. This model is a nice compliment to the CMMI method.

PSP has different levels

  • PSP 0: Time and defects registration.
  • PSP 0.1: Phase to set standards and measurements.
  • PSP 1: Estimation of the magnitud of the project and testing.
  • PSP 1.1: Task planification.
  • PSP 2: Code and design review
  • PSP 2.1: Design specification and analysis techniques.

A PSP certification can be obtained by the SEI at Carnegie Mellon University through an exam.

In order to have TSP, PSP has to be implemented first. TSP is implemented through cycle processes, that include launching, execution and the end of the process. TSP assings duties to the members of the team:

  • Team leader
  • Customer interface manager
  • Design manager
  • Implementation manager
  • Test manager
  • Planning manager
  • Process manager
  • Quality manager
  • Support manager

TSP can be certified in the same way as PSP.

ISO-15504

“ISO/IEC 15504 is an International standard for Software Improvement Capability Determination, also known as SPICE, to evaluate the processes in software development products”

This standard takes 3 elements into account:

  • A 6 level schema to evaluate the capabilties of processes
  • A set of requirements to define processes
  • A set of requirements to implement consisten evaluations

The 6 levels of process capabilites are:

  • Level 0: Incomplete
  • Level 1: Performed
  • Level 2: Managed
  • Level 3: Established
  • Level 4: Predictable
  • Level 5: Optimizing

These levels are similar to the levels to the CMMI method, this is due to the fact that CMMI is an alternative to ISO 15504. And as such, ISO 15504 is focused on the improvement of the processes.

In order to assing a level to a process there exist classification attributes, whose values serve to that purpose:

  • N: not implemented (0-15%)
  • P: partially implemented (> 15-50%)
  • L: highly implemented (> 50-85%)
  • F: fully implemented (> 85%)

It is more accesible to get an ISO 15504 certification than a CMMI, and cheaper to implement. Also CMMI is not an international standard.

MOPROSOFT

“MOdelo de PROcesos de SOFTware is a software maintainance model in order to help companies improve their work in a competitive environment”

It was created by the Secretaria de Economía in order to be the basis for the Norma Mexicana para la Industria de Desarrollo y Mantenimiento de Software. Moprosoft is structured in 3 categories:

  • Direction: It establishes the guidelines for the management processes, and uses the information that receives in order to get feedback that helps the improvement.
  • Management: It defines processes and gives directions to the operations sector, according to the strategy developed by the Direction category, and reports results to the former.
  • Operations: in charge of generating products with the information fed by the management, and deliver reports.

Its levels of process capability levels are the same as the ones in ISO 15504: Performed, Managed, Established, Predictible, and Optimizing. It offers also different categories to the processes that are grouped in the 3 different structure categories:

  • Direction:
    • Business
  • Management:
    • Proyects
    • Human resources and Work Environment
    • Goods, Services and Infrastructure
  • Operations:
    • Specific Management of Projects
    • Development and Maintainance of Software
IDEAL method

“Initiating, Diagnosing, Establishing, Acting, Learning (IDEAL) method serves as a guide for initiating, planning and implementing actions that help the process improvement in the context of a CMMI product suite.”

Also created by the SEI at Carnegie Mellon University, it was created as a process-improvement and defect reduction methodology. It consists of 5 phases:

  • Initiating: establishes context for change. It decides which level of risk is addecuate for the organization. It also should be decided who will be taking action.
  • Diagnosing: Identify problems within the processes and gives recommendations to get from the current state to a decided state.
  • Establishing: Sets priorities based on many elements, develops a path to obtain the desired state and plans the actions that will be taken.
  • Acting: Creates, tests, refines and installs the solution.
  • Learning: proposes future actions based on the analysis and validation of the solution.
References

Sami, M. (2018). The Software Process Improvement (SPI) – Reward or Risk. Mohamed Sami. Retrieved from https://melsatar.blog/2018/06/26/the-software-process-improvement-spi-reward-or-risk/

Keshta, I. (2019). Approaches to Software Process Improvement: A State-of-the-Art Review. ResearchGate. Retrieved from https://www.researchgate.net/publication/337327204_Approaches_to_Software_Process_Improvement_A_State-of-the-Art_Review

White, S. (2018). What is CMMI? A model for optimizing development processes. CIO. Retrieved from https://www.cio.com/article/2437864/process-improvement-capability-maturity-model-integration-cmmi-definition-and-solutions.html

CMMI. (n.d.). What is CMMI®?. CMMI Institute. Retrieved from https://cmmiinstitute.com/cmmi/intro

TSP/PSP. (n.d.). ¿Qué es PSP y TSP?. Avantare. Retrieved from http://www.avantare.com/0/materiales-de-referencia/que-es-psp-y-tsp/print

TSP/PSP. (n.d.). PERSONAL AND TEAM SOFTWARE PROCESSES (PSP/TSP). European Software Institute. Retrieved from https://www.esicenter.bg/processes/tsp_psp

ISO 15504. (n.d.). Consigue la certificación ISO 15504 de forma sencilla. calidad y gestión. Retrieved from https://www.calidadygestion.com/iso-15504/

IDEAL (n.d.). IDEAL — Initiating, Diagnosing, Establishing, Acting & Learning. Play-In-Business. Retrieved from https://www.plays-in-business.com/ideal-initiating-diagnosing-establishing-acting-learning/

ISO 15504. (n.d.). EL ESTÁNDAR INTERNACIONAL ISO/IEC 15504. Normas ISO. Retrieved from https://www.normas-iso.com/iso-iec-15504-spice/

Humphrey, W. (2000). The Team Software ProcessSM (TSPSM). Carnegie Mellon Software Engineering Institute. Retrieved from https://resources.sei.cmu.edu/asset_files/TechnicalReport/2000_005_001_13754.pdf

Moprosoft. (2011). Moprosoft. Allsoft. Retrieved from https://www.allsoft.mx/recursos/AS-Moprosoft.pdf

Moprosoft. (n.d.). ¿Qué es Moprosoft?. Hospisoft. Retrieved from https://www.hospisoft.mx/moprosoft/

Software Quality

The software industry is an exponentially growing market, demanding more and more, creating a highly competitive environment where deploying code into production as quickly as possible has become a necessity in order to survive. This arises a big concern, since creation without control leads to chaos. If so much software is being developed, there needs to be an entity, a methodology, a guide or something that assures that everything will be under control, even in the worst case scenario (which may be not as common as people believe, but as plausible to haunt everyone that is aware of it.) So of course, there is an answer to this nightmare, a concept that has existed almost since the inception of software development itself, that being “Software Quality”.

Defining Software Quality

Now, defining the term of quality of software is not quite as easy as it may seem. I have found a plethora of definitions, but I will use the one that I like the most:

“Software Quality is about maintaining the functionality of a software and assuring that it fulfills its purpose.”

I like that definition, because it takes into account the non-technical factor of the software doing what is supposed to do. I paraphrased the the definition from this video:

Regarding the more technical aspects of software quality, there are some key facets that should be noted:

  • Good Design: It should be attractive, people will not use it if it is not user friendly and does not have a fitting aesthetic.
  • Reliability: It should always work as expected, with no issues.
  • Consistency: It should behave in a consistent manner on the platforms where it is available.
  • Durability: It should be reliable for as long as possible.
  • Maintainability: After release, any bug fixes and code enhancements must be added without compromising reliability.
  • Valuable: All the work put into the development of the software should feel worth it. If a user pays for it, it should feel like money well spent.
Maintaining software quality

Having established the requirements to measure software quality, let’s go to the next step. Software quality can be achieved in a short period of time; however, it is important to consider that it should be maintained throughout a relatively long period of time. For that, there is “Software Quality Assurance”. But, what is its definition?:

“A process to assure the quality in software engineering processes, methods, activities and work items, and ultimately give confidence that a producto is of good quality.“

There are many stages of the SQA (Software Quality Assurance), which go from the beggining of the project, to its after life. SQA should be involved since the definition of the requirements and architecture. For SQA is involved in all the software development cycle, it is important to note that it involves more than the more technical aspects of the product; it also should check for the quality of management.

SQA’s work is to find weaknesses among the different processes and correct them, in order to be continually improving. And there are many facets to consider (management, testing, design, configuration, documentation, to give some examples). So here comes another question, what needs to be considered in order to have a good SQA? Or in other words, how can the quality of the software be meassured?

Standards

There are very well known software quality standards to which the processes in the software development cycle should be complied against in order to ensure a good SQA. These is very simple in concept, SQA follows some models that have requirements to meet in order to be considered of good quality, and that is it. These requierements cover the processes, activities, and everythinh that involves SQA. There are many examples of standards that are going to be discussed in a later blog. For now, this is the introduction to Software Quality.

References

Software Quality. (2018). What is Software Quality? Try QA. URL: http://tryqa.com/what-is-software-quality/#:~:text=Quality%20software%20is%20reasonably%20bug,satisfy%20stated%20or%20implied%20needs.%E2%80%9D

Software Quality. (2020). What is Software Quality? and How to achieve it?. testbytes. URL: https://www.testbytes.net/blog/what-is-software-quality/

Software Quality. (2020). Software Quality. SOFTWARE TESTING. URL: https://softwaretestingfundamentals.com/software-quality/

Software Quality. (2020). Definition of Software Quality. Xbosoft. URL: https://xbosoft.com/definition-software-quality/

Software Quality Assurance. (2020). Software Quality Assurance. SOFTWARE TESTING. URL: https://softwaretestingfundamentals.com/software-quality-assurance/

Software Quality Assurance (2020). What Is Software Quality Assurance (SQA): A Guide For Beginners. Software Testing Help. URL: https://www.softwaretestinghelp.com/software-quality-assurance/

Images from https://undraw.co/