For this entry we’re making a little throwback to another post, this time I’m talking about V&V, given that there is already an entry here about V&V,  I’m going into the specifics tools that are used in V&V. For this topic, I'll divide the categories in three:

• Tools for version control
• Tools for testing
• Tools for process administration of V&V

Version control

 

For version control tools we need a system that records changes to your code or files, with time in consideration so you can recall previous versions. The one that is most used is GIT. If you’re reading this you probably have worked on git already, but the main takeaway is:

• Compatible with all operating systems
• Distributed Systems that allow users to work on a project from various sources
• Branching, a parallel line to the main sources files so you can work without negatively impacting the final project          
• Fast & open source

Other tools for version control are: Mercurial, AWS CodeCommit, CVS

Testing

For testing tools, a frameworks that allows you to test your product in various environments is needed, one of the most used ones is Selenium, Selenium is a testing tool which allows has the next features:

• Open source
• Provides playback and record feature for tests
• Record actions and export them in a script
• Supports the following languages: C#, Java, Python, PHP, Ruby, Perl, and JavaScript
• It supports the following operating systems: Android, iOS, Windows, Linux, Mac, Solaris.
• And the following browsers: Google Chrome, Mozilla Firefox, Internet Explorer, Edge, Opera, Safari, etc.

Some of the alternatives to selenium are: 

• Robot Framework.  
• Cypress.  
• Katalon Studio.  
• Screenster.  
• CasperJS.  
• Watir.  
• Cucumber.  
• Ghost Inspector.

Process administration

In this category products are used to develop and satisfy management needs, from test cases to requirements. One of the most used in this category is Jira, some of its uses are:

• Scrum boards
• Kanban boards
• Roadmaps
• Agile Reports

And some of its alternatives are:

• ClickUp.
• Binfire.
• Basecamp.
• PivotalTracker.
• Asana.
• Clubhouse.
• Trello.
• ProofHub.

And that’s it, if you made to this point, Thank you for reading, I hope I can write soon, if not, its been a pleasure to write these, Have a good one everybody!

 

This time were talking about software testing, one of the most important processes in software quality assurance, which I might say a lot, but NOW I’m really meaning it (Not that I didn´t meant it before, it just really important this time).  The definition of software quality is “an investigation done with the purpose of informing stakeholders about the software’s quality of the service or product. It pretty much means that the tester checks that the product is behaving as it should, and if they find errors, they solve them. Software testing has a defined process which is divided in 5 steps:

1. Planning

In this step we determine the scope of the test and the risks of it, we also define the materials to be used, the sources the test needs and we give an estimate of the test dates.

2. Creation of scenarios

In this step we define the expected outputs of our test, and the scenarios of the test are written. The scenarios of the test should range between simple and diverse, so that the test itself can achieve a validity.

3. Preparation of test environment and data creation

In this step the preparation of the test are administered, the project should be installed in the test environment, the data to be used in the project should also be provided and prepared with accordance of the project reach.

And the test environment itself is tested so it doesn’t generate a problem during the test itself.

4. Run the Scenarios.

Simplest step, the scenarios previously created are applied and the tests are run. All the scenarios should be applied successfully but time constrains may make this a little too tough.

5 Report the results

A report of the result is created so that the stakeholders concerned are communicated, it should be noted that this report should use a basic language as possible, and it can present some videos or photo to help illustrate the results.

After all this explaining it should be clear what the general process of software testing is, but this process is applied even further as there are different types of software testing:

• Unit Testing
• Integration testing
• System Testing
• Acceptance testing

Unit testing, which is usually done by developers test whether a piece or block of code is doing its purpose. In this case we want to test a piece of code that calls another one and to isolate that piece so that we can verify its functionality.

Integration testing is done by testers, this one checks the combination of various parts in conjunction and their functionality as a whole. For example, if in a user interface test a button isn’t working, all the part that compose that user interface are considered as faulty. So, in integration testing we test the operation of two or more code blocks.

System testing is a test of the system as a whole (Shocking I know). If the previous tests are done in one or a group of code block, this test is done on the whole system, and it’s performed by QA testers. For example, testing an application and making sure that all of the functions are being processed as it was designed, if a bug is found during this test it usually means that the previous test were incomplete.

Acceptance tests are performed by end users, and its purpose is to confirm that the product is considered successful by a group of users.

Now in every software quality test there is usually a team or group of persons behind it, and these people have defined tasks and roles, so let’s do a breakdown of these. A group of quality testers is made up of, usually these roles:

·        Software tester:

o   Responsible for designing testing scenarios for usability testing.

o   Responsible for conducting the testing, thereafter, analyze the results and then submit his observations to the development team.

o   He may have to interact with the clients to better understand the product requirements or in case the design requires any kind of modifications.

o   Software Testers are often responsible for creating test-product documentation and must participate in testing related walk through.

 

·        Software test manager:

o   Responsible for all interdepartmental meetings.

o   Interaction with the customers whenever required.

o   Responsible for recruiting software testing staff.

o   Schedule testing activities create budget for testing and prepare test effort estimations.

o   Carry out continuous test process improvement with the help of metrics.

o   Check the quality of requirements, how well they are defined.

o   Trace test procedures with the help of test traceability matrix.

 

·        Software test Automator:

o   Be able to understand the requirement and design test procedures and test cases for automated software testing.

o   Design automated test scripts that are reusable.

o   Ensure that all automated testing related activities are carried out as per the standards defined by the company.

Another important aspect of software testing is the environment of your tests, while you could have all your tests in one environment it is recommended that for some types of tests your environment is prepared for the specific situation. We have four different types of testing environments, which are:

Integration Testing Environment

For this type of environment you don’t need a specific need of your environment, given that the tests that are applied are integration test, with an environment that can support these and has access to the basics and has the configuration and management of application servers, web servers, databases, and all the infrastructure needs of the application, you’re good.

Performance testing environment

Int this case the tests that is going to be applied is the system test, but you’re probably want to check how your product performs, so in this case you’re going to need multiple environments that has different kinds of configuration and characteristics, like: number of CPU cores, size of RAM,  concurrent users, Volume of data, etc.

Security Testing Environment

For this case of testing we need an environment which is devoid of security flaws (kind of impossible, I know) because these test are the ones with online services integrated, so when you’re running your system test you need to do a security test. Some of the rules for these tests are:

• Have an isolated test environment.
• Have non-disclosure agreements in place.
• Don’t leave the system in a worse state.
• Don’t touch production data

Chaos Testing Environment

 

One of the most important aspects in software quality is reviewing the software itself. Software review is defined by the IEEE as “A process or meeting during which a software product is examined by a project personnel, managers, users, customers, user representatives, or other interested parties for comment or approval”, so to put it simply, when we talk about reviewing software were talking about checking everything related to a software product deliverable, so this includes reviewing a few parts of the project: contracts, plans, budgets, specifications, designs, source code, user documentation, support and maintenance documentation, test plans, test specifications, etc.

Now there are various types of software testing, but well talk about three: Peer reviews, management reviews and audit reviews

Peer review is when one of your coworkers review a project part, this can be formal or informal, as there are protocols and standard practices for these, but it can also be as casual as one could imagine. The most common practices revolve around walkthrough of the product, technical reviews, or inspections of the software. In this types of review there is only two role, the developer and the coworker/buddy, in this case od software review an supervisor or managerial worker cant be involved, because it would turn this type of review into another.

The second type of software review is managerial review is a review process which is conducted by a supervisor and consists of two different roles: management (individual or group) and developer. This type of review can be an informal one, but it is required to follow a standard or structure, the one proposed by the IEEE is the following:

• Evaluate entry
• Management preparation
• Plan the structure of the review
• Overview of review procedures
• Individual Preparation
• Group Examination
• Rework and/or follow-up
• Finish evaluation

And the last type of software review is audit review, this one is the most on depth type of review as this one is conducted by an independent party, which assesses compliance with specifications, standards, etc. Given that this is an examination from an independent organization there are protocols and defined roles to follow in this case, the roles consist of:

• The Initiator, which could be a manager in the audited organization, a customer or user representative of the audited organization, or a third party. They decide upon the need for an audit, establishes its purpose and scope, specifies the evaluation criteria, identifies the audit personnel, decides what follow-up actions will be required, and distributes the audit report.
• The Lead Auditor, which must be someone "free from bias and influence that could reduce his ability to make independent, objective evaluations". They are responsible for administrative tasks such as preparing the audit plan and assembling and managing the audit team, and for ensuring that the audit meets its objectives.
• The Recorder, which documents anomalies, action items, decisions, and recommendations made by the audit team.
• The Auditor(s), also free from bias, they examine products defined in the audit plan, document their observations, and recommend corrective actions.
• The Audited Organization provides a liaison to the auditors, and provides all information requested by the auditors.

And follows these principles.

• Timeliness
• Source openness
• Elaborateness
• The financial context
• Scientific referencing of learning perspectives
• Literature-inclusion
• Inclusion of user manuals & documentation
• Identify references to innovations

Every key product in the software development process (plan, requirements, design, code) can be review with the former types and standards, as it allows for adaptation with all the products that need software review.

We’re almost done with this course, so expect more dump of subject in the following week, till the next one.

 

So, in the software field, you know: software engineering, testing and project management, (V&V) is  one of the most important part in software quality, if not the most important one, this sis the process where we *verify* that our software is up to our specification and we *validate* that it fulfills its purpose, it is also called quality control, so you might have heard about it.

To clarify it a little further, V&V consist of two parts: verification and validation

Verification means: Are building this right?

Validation means: Are we building the right thing?

One of the most asked questions in this topic is: How often should I do V&V during software development life cycle?(I know it’s a long question to be asked often but just roll with it). Well the answer is: always. Software quality is a fragile thing, if we go through a stage in developing our product without the proper testing a validation, we can create unforeseen problems down the line, which could have been avoided with V&V.

Now there isn’t just a way to determine if your V&V is doing good, because we measure how your software fulfills some of its goal, there isn’t a set way of doing this, because every software is different, the is a need of different standards for these variating projects, some of them are:

• ISO/TS 17033
• ISO 14021
• IEC 17029
• IEEE

And even more!!

If you would like to know more check out:

To do V&V you have to have to have a plan in mind to go about it, as I’ve said before, every software is different, so you might have different ways to go about deploying your V&V, but it easy, and ill break it in 5 points:

• Description: pretty much just explaining what you’re going to do and the definitions inside your V&V
•  Component test: Defining which components are you going to be validating and verifying, how are you verifying them
• Functional test: Same as above but for the general system or in bigger chunks of components
• Acceptance rate: Checking how if the previous tests pass their specific acceptance criteria
• Results: Summarizing results

And that’s it, this is the general framework a V&V plan should have

And to end this topic ill talk about administrating the plan which is the easiest part of V&V, for this we can summarize this to some points:

• Document the tests outcomes in a concise way.
• List all components to be tested in an optimal order
• Document the tests
• Make sure your plan accounts for every environment
• Document EVERYTHING, for some reason the documentation for pretty much everything is undervalued, but bad documentations practices takes a heavy toll on the final product.

And that’s it for V&V!!!

 

 

For this entry we’re talking about a more specific topic that I said we were going to talk about more in depth: Models and Standards for Software Process Improvement. As I said in the last entry, there is a wide range of different models and Standards to use during QA of your process, so now we’re going to talk about 5 of them, so get ready.

 

CMMI

CMMI stands for: Capability Maturity Model Integration, this model stands out for being a process and a behavioral model for quality assuring. This model was developed by the Engineering Institute at Carnegie Mellon University as a collaboration from the US government and DoD in 1987 and the first version was released in 1997. As a side fact this model is the one used in US governmental software development.

This model starts by creating an evaluation on three different areas: process development, establishment & management and product acquisition. For the evaluation method the SCAMPI appraisal method is used and is divided in three different evaluation classes A, B, C.

Class A: Is the most rigorous appraisal as it must be performed by a certified appraisal leader. This type of SCAMPI is the only one that results in an official rating and it is used as a benchmark for various business.

Class B: This type of SCAMPI analysis is less formal, it finds a CMMI maturity level and gives a predicted success on the processes used

Class C: This analysis is more flexible and cheaper than the previous ones but riskier, this SCAMPI method evaluates the organization practices and if these align in the CMMI practices.

Now, CMMI method evaluates the organizations that embraces CMMI into 5 levels, level 5 being the highest, the breakdown of these levels is:

• Initial (Level 1): This is the worst level as it shows that processes in the organization aren’t, well, good, this is described as: “work gets completed but it’s often delayed and over budget.”
• Managed (Level 2): This level shows that there are at least some decent processes getting implemented, as this level describes the process as: “planned, performed, measured and controlled”.
• Defined (Level 3): Here the business is described as proactive, there’s usually a set of “organization-wide standards”. Businesses understand their shortcomings, how to address them and what the goal is for improvement.
• Quantitatively managed (Level 4): In this stage the company is pretty much in good term in their process quality. The organization is working off quantitative data to determine predictable processes that align with stakeholder needs.
• Optimizing (Level 5): Here, an organization’s processes are stable and flexible. At this final stage, an organization will be in constant state of improving and responding to changes or other opportunities.

Now for a closing part on CMMI, there has been a constant evolution in their model, as there is now a version 2, which implement more maturity models into its framework, which shows its adaptability to apply in various organizations.

For even more info on CMMI you can check: https://www.cio.com/article/2437864/process-improvement-capability-maturity-model-integration-cmmi-definition-and-solutions.html

TSP/PSP

This model is made from 2 frameworks that are technically one but two as the same time(?) this will get clearer in a bit. TSP is based on PSP. PSP stands for: Personal Software Process and TSP stand for: Team Software Process, kinda similar right? Well, technically, this framework is pretty much one, as TSP is PSP based and it requires that the developers in the team are already educated in PSP. PSP and TSP were created by Watts Humphrey, A.K.A the father of QA, in 2005 and 2006 respectively, he also created the first version of CMM in 1989, so he’s kinda a big deal.

PSP follow and linear improvement approach as it rates the developer in 3 different phases:

PSP0 which introduces the developer to discipline and measurement with concepts such as: planning, development and postmortem.

PSP1 which prepares the developer for estimating how large would a new program be and how to create a test case for it, as well as schedule planning and estimation.

PSP2 which helps the developer to review design and code through quality management and design.

PSP also emphasizes the use of historical data to analyze a process, this data collection is based on four concepts: Scripts, measures, Standards, Forms.

Side note: I once lost a friend’s PSP in a 6x6 mtrs classroom and to this day, the fact that nobody was able to find it, haunts me, all this PSP talk made me want to get a PS Vita to play some console locked games, like Persona 4, but maybe it should be better to emulate it. All this PSP talk is distracting me, I’ll continue.

Now we can talk about TSP, which is textually a lot easier to explain, this is basically a team of experts in PSP maintaining a self-directed team. Applying all PSP lessons a team can, and will, cover the role and tasks of a manager by themselves, for example, a TSP working team can: plan and track their work, manage the quality of their work, develop software in a proactive way to meet their goal, etc.

If you are curious in this topic you can learn more at: https://explainagile.com/agile/team-software-process/

ISO-15504

Ok two out of five so far, let’s get started with the next one!

ISO-15504 also known as SPICE(Software Process Improvement and Capability Determinator, great name by the way, is another framework for quality assessment, this one is helpful when we wish to analyze the capability of the process and services, this framework was initially developed in a draft in 1993 and the first conference of this framework was in Limerick in 2000 called, as you guessed it, SPICE.

As the previous framework CMMI, this framework also works with assessment and evaluations, which assesses(?) the process capability level in 6 levels as follows:

• Incomplete (Level 0): This process is either non-functional or incomplete
• Performed (Level 1): This process is implemented.
• Managed (Level 2): The process is managed, and the results are specified and maintained
• Established (Level 3): The process is standardized and used throughout the organization
• Predictable (Level 4): This process is constant, and the outputs are within the expected limits
• Optimizing (Level 5): This process is constantly improved to maximize optimization withing the organization

The capability of the processes is measured through attributes, which in this case are nine:

Process performance, Performance management, Work product management, Process definition, Process deployment, Process measurement, Process control, Process innovation, Process optimization

And these attributes are assessed with a four-point scale:

• Not achieved (0–15%)
• Partially achieved (>15–50%)
• Largely achieved (>50–85%)
• Fully achieved (>85–100%).

Did I already make a stupid process processing joke? As you can see this methodology help us evaluate the process in so many ways, and this makes it a truly viable framework to help our QA. But technically this framework in not implemented anymore, as it evolved into ISO 33000 in 2015, which encapsulates even more concepts and models from other ISO/EIC models, but, as I’ve said before this method is still viable.

If you would like to read more about at: https://www.turcert.com/en/belgelendirme/sistem-belgelendirme/iso-15504-yazilim-surec-degerlendirme-sistemi

MOPROSOFT

Now we’re talking about another process standard and this is a curveball, this one is called MoProSoft and it was created in Mexico in a collaborative effort by the Mexican Association for Software Engineering Quality and the Universidad Nacional Autónoma de México headed by Dr. Hanna Oktaba (A badass in the software engineering field) in 2005.

This model stands for “Modelo de Procesos para la Industria del Software” or Processes Model for the Software Industry in English. And it was developed with the intention to elevate the capability of Mexican software organization to compete in an international level. This model has a somewhat clear and long list of criteria that explains the elaboration of this model, but the only part that really is important for us (or for this topic at least) is the process categorization, which this model specifies that you must have a set of processes to cover with all of the types of categories, which are the following:

·        Top management Category

o   Business management: It must have a process that takes into consideration the organization’s strategic plans, objectives and goals

·        Management Category

o   Process management

o   Project management

o   Resources management

·        Operative Category

o   Specific project administration

o   Software maintenance and development

With these set of processes, we can open the door to adopt a more rigorous model as ISO 9000 or CMMI

If you want to read a bit more about this topic (and you know/want to learn Spanish) here a good post about MoProSoft: https://www.ana2lp.mx/desarrollo-de-software/moprosoft-modelo-mexicano/

IDEAL Method

Now for our last model we have the IDEAL methods which, as you might have guessed, is another acronym, which stands for: Initiating, Diagnostic, Establishing, Acting & Leaning, this model was published under the Software Engineering Institute at Carnegie Mellon University by Bob McFeeley in 1996 and it consists in a 5 phase cycle.

This model is considered an organizational improvement for CMMI and the SPICE methodologies as it salvages some of the framework infrastructure. The name of the method itself is the 5 phases of the cycle, which it cycles after the fifth step and goes back to the second and so on. Now the specifics for the phases are:

• Initiating: In this step and necessity and a sponsor must be found to start the diagnosis.
• Diagnosing: This is an analysis phase as you want to innovate the current practices in processes, to compare it to other models, it would be like and SCAMP or an appraisal.
• Establishing: Here we create specific solutions from the result of our diagnosis and finish a concrete plan.
• Acting: Act according to the plan.
• Learning: Learn and adapt to improve the next cycle of the model

Now each for the phases, these have a defined set of activities which makes this model even more developed that it would be initially assumed. The activities are:

•  Initiating: Establish context, build sponsorship and charted infrastructure.
• Diagnosing: Characterize current and desired state, Develop recommendations.  
• Establishing: Set priorities, Develop approach, Plan actions.
• Acting: Create solutions, Test/Pilot Solution, Refine Solution, Implement Solution
• Learning: Analyze and validate, Propose future actions.
  

 

While this method seems like it could be a methodical problem solving employed in various sectors of any business it goes to show how much does the actual processes of QA need to be in the spotlight as it is a REALLY important part of QA.

AND THAT’S A WRAP. This one was longer than expected an somehow I kinda want to explain more things in this topic, but it’s already longer than expected so I might make and extra entry just for a model or two of these, and so I can dive deeper in a couple of the most interesting ones. Well I hope you enjoyed this one!

 

 

 

 

 

 

For every software out there, there has been an insufferable amount of tests done to make sure that the quality is up to par (yeah I know shocking) but there is a whole sub field of software engineering dedicated to make sure the process behind these test isn’t just running your product and toying with it

For any software that is developed a certain degree of quality must be attained, but without a reliable process or methodology to follow the average quality of the software developed would be, in one word, wonky.

So for this specific necessity a new field of software engineering was created, and so the field of software quality and testing was born, and to make it even more formal we can define software quality as : “A field of study and practice that describes the desirable attributes of software products.”

 

Now, if we want to get into the specifics of the process of software quality we would be talking about quality assurance, and quality assurance in software testing is defined as “A procedure to ensure the quality of software products or services provided to the customers by an organization.” This is better known as QA testing, and as for the process itself of QA testing we can talk about great number of different methodologies, but for this entry (or for at least this version totalyNotForshadowingAnUpdatedVersionOfThisEntryLaterDownTheLine) we will talk about one, the standard (or at least the standard one according to my research)

This process is defined in just a few steps, which are:

• Review of requirements
• Test planning / writing test cases
• Unit testing
• Integration testing
• System testing
• Performance testing
• Security testing
• Cross-browser testing / cross-platform testing
• Updating test cases
• Regression testing

You still here? Great! Now we are gonna talk about each one in excruciating detail… Or not. I might create a new entry just for this topic or create an update blog post for this one, just because there is so much to talk about here, but for now I’m only going to give an overall idea, but, if you would like to know more about the process in detail you can check it over here: https://kruschecompany.com/quality-assurance-in-projects/#ixzz6YW68XxL8

 

With the aforementioned steps we have in our hands a powerful methodology which we can, and should, use in every software testing project, sadly I doubt this is a standard in some parts of the software development industry.

A couple of paragraphs before we briefly mentioned QA, but we didn’t really get into it, soooooo…

Quality Assurance is the way, believe it or not, we can ensure a quality standard for our software. With QA we can define which set of organizational processes should be paired with the software development standards that suits them the best, and so we can create an specific methodology for each case that we encounter in the industry, funny enough this is also a somewhat controversial topic in the field, as QA has been criticized heavily, stating that “process standardization can sometimes stifle creativity, which leads to poorer rather than better quality software.”

There are even more processes to define which process to process (uhhhh, yeah let’s go with that), One cycle we can use to do QA is the PDCA cycle which stands for: Plan, Do, Check, Act. And for we can define each step as:

• Plan – to plan and establish the process objectives and determine the processes that are required to deliver a high-Quality end product.
• Do - Development and testing of Processes and "do" changes in the processes
• Check - Monitoring of processes, modify the processes, and check whether it meets the predetermined objectives
• Act - Implement actions that are necessary to achieve improvements in the processes

At this point you might be tired of processing how many times I’ve said process, so let’s get going with the last part of this entry: Standards in Software Quality.

There isn’t just a standard for all QA that is realized. Each organization has a different set of goal and workflow, so to accommodate for this there is quite a range of standards in the field. Each standard has a different set of goals, methods and ways to implement, all of this to pretty much identify weaknesses in our process and to give us the tool to correct them. To select one is quite the important task and to do QA without any of these is pretty much useless, so when testing software there must be an previous investigation of which models suit your process the best, as for which models and standards are there, well we might talk about this soon(totally not the next entry)

If you want to read even more about this topic, you can check the material at the sources down below:

QA Process: How the QA Team Tests Your Project: https://kruschecompany.com/quality-assurance-in-projects/ 

What's Quality Assurance(QA)?:https://www.guru99.com/all-about-quality-assurance.html

Software Quality Standards: https://qa-platforms.com/software-quality-standards/