Introduction
For most organizations, quality has become a “number one” priority. The high costs of errors and the recovery from them is driving management teams to focus on achieving flawless results, especially in the documents that send daily to millions of customers. However, to keep up with evolving customer expectations, regulatory changes and competitive pressures, organizations are forced to make significant changes rapidly.
Today, even conservative organizations such as large banks and insurers are migrating mission critical applications to cloud based platforms to leverage their faster deployment capabilities and ensure they can respond to customers immediately. The down side of this high velocity of change is that, indeed, changes happen extremely quickly, often on very tight schedules controlled by the hosting organizations.
In order to bridge this gap, an ultra-efficient quality assurance process is needed and radical changes must be adopted for quality assurance of cloud-based applications. Fully automated testing that streamlines testing processes and identifies problem areas very quickly are required in order to contend with shorter release cycles. Naturally, as these applications being tested reside in the cloud, it makes sense to have a cloud based QA process in place.
Challenges in Achieving Automated Quality Assurance for Customer Communications
Organizations are increasingly challenged in achieving flawless automated quality assurance testing for Customer Communications Management (CCM) software. Consider the processes and toolsets that are required:
- A test environment must be created for each platform that is used to create output If you use multiple document composition software packages and post-composition tools, you’ll need to do this for each one.
- Test suites must be created and the tests must thoroughly cover all conditions that might be encountered in your CCM production deployments.
- Once created, your test suites and individual cases must be run, and the results reviewed and Each time a change is made to your software, templates, configurations and computing environment, these processes much be completed.
- There are utilities for comparing print streams, but you will need to create, modify and maintain scripts to do the testing
- If there are discrepancies or differences, you must determine the cause and the possible fixes – whether it’s a change to a script or configuration file, a program change, or a problem that involves another vendor.
- And then of course, once the test case or issue is fixed, you need to repeat the entire cycle again.
It is a challenge to manage all of the testing required for document creation tools. And as more and more projects move to Agile and cloud methodologies, these challenges grow exponentially. Unfortunately most organizations cut some corners to simplify their QA implementation, and that usually leads to problems.
Why is QA of transactional document software so complex?
Performing Quality Assurance in a high volume transactional document environment is inherently complex. Beyond the obvious challenges in dealing with millions of documents containing highly variable information, you must also deal with a multi-dimensional matrix of:
- Tools that need testing
- Outputs from those tools
- Platforms that the tools run on
- Fixes that are applied to the Tools
- Results of the tests (Error Return codes, file comparisons, and performance tracking).
Building test environments can be a very complicated process, with multiple interdependencies, including the type of software being tested, the desired testing methodology, the people responsible for maintenance of testing environments, and many additional factors.
Significant infrastructure resources must be committed to support an effective Quality Assurance program for CCM operations. In addition to testing servers for each platform, storage, software, automation systems, as well as trained, qualified people are required. All must be maintained and revision levels must be kept current. Investment in the computing resources that are dedicated to this function may go unused for months depending on testing schedules, and many organizations are reluctant to fully fund dedicated Quality Assurance operations due to their concerns regarding underutilization of assets and rapid obsolescence.
In some organizations, QA operations may happen sporadically, such as when document application templates are modified or when a new version of one of their tools is available and needs to be tested. Others create a periodic QA regimen that may happen daily, weekly, several times a year, or maybe only once a year. Unless trained, dedicated QA staff regularly utilize their QA environment, often they cannot remember what they did or the procedures they followed the last time they conducted testing, and there are challenges related to re-learning basic operations each time a particular tool/application needs testing.
Your customers’ Personal Sensitive Information is often incorporated into QA testing, raising many issues related to secure handling and unauthorized access. Your test systems should be protected in a PCI- DSS or HIPAA compliant environment, or even better, personal sensitive information should be redacted. Redacted test files are ideal to utilize internally or to send to a vendor to use in testing. Adding on premises redaction capability to clean your files of personal sensitive information before they are used in QA is a best practice that should be considered.
Effective automated Quality Assurance testing depends on the quality and validity of the tests that are executed. Improperly built test cases can cause myriad problems that spawn less- than- ideal outcomes. For example, if a test case puts the program being tested into a loop, it can run endlessly until someone notices that it has never completed. In a serially organized testing system, the loop will cause all succeeding tests to not be executed. This scenario will typically causes major delays in downstream project schedules due to missed testing deadlines.
These challenges add up to a major headache for most organizations, resulting in serious issues:
- Application errors are missed in QA, causing failures with customer communications in production
- New releases encounter significant delays, slowing organizations’ abilities to meet market needs
- Excessive resources are involved in the QA testing process, often pulled in from other groups in the organizations
- Fines and penalties may be assessed for missing SLAs or for violating regulatory requirements
In order to mitigate the aforementioned issues, some organizations often run multiple versions of software, risking problems with future CCM software upgrades. In fact, we have witnessed some organizations running multiple versions of the same software due to their inability to do proper regression testing.
Moving QA into the Cloud
With organizations being geographically dispersed and production output operations being shared across different departments and external vendors alike (and with much QA work being outsourced), it makes sense to take advantage of cloud technology so that workers can fully contribute, regardless of where they reside physically or virtually.
All industries need effective Quality Assurance programs for their customer facing document workflows and high volume document production applications, especially in highly regulated markets including Financial Services, Insurance, Healthcare, Telecommunications, Utilities, Government, Education, Retail, Transportation and Manufacturing
As organizations in all segments are moving many mission critical functions to cloud-based platforms, a complete rethinking of how to best accomplish QA processes is required.
CrawfordTech’s QA in the Cloud
Over the past 20 years Crawford Technologies has built a very sophisticated Quality Assurance system that we use for regression testing of our own suite of document management software. The comprehensive environment includes a fully automated testing framework, a database managed testing facility and a web browser-based dashboard for managing the testing process. CrawfordTech has also built a high security data center with PCI-DSS, HIPAA and SOC2 certifications to host this facility. It works equally well for testing on-premises software as it does for cloud applications.
QA in the Cloud is CrawfordTech’s secure cloud-hosted QA service which provides users with all the tools needed to fully automate document-related Quality Assurance testing processes. Users can load their own test cases and software — including software from other vendors — onto the system. Benchmark files will then be uploaded (or created) and verified. When you have a new release of software, you can simply load it up and schedule test runs. You do the same thing with configuration files or template changes.
CrawfordTech QA in the Cloud provides emailed reports as well as a browser-based dashboard for monitoring and viewing the results. Both enable visibility at a glance into the impact of any changes that have been made, and point you to only the areas that require you attention. You will then be able to quickly and easily assess whether you can put a new release or version into production, or whether changes will be necessary.
With organizations now spreading out geographically, and a great deal of QA work being outsourced, it makes sense to take advantage of cloud technology so that workers can fully contribute, regardless of where they reside. Organizations in all segments are moving many mission critical functions to cloud-based platforms, requiring a complete rethinking of QA processes. QA in the Cloud is the logical choice to address today’s and tomorrow’s quality assurance needs.
Getting Started with QA in the Cloud
QA in the Cloud is a turn-key solution. Crawford Technologies sets up your desired QA environment and supports as many applications that you need to test. CrawfordTech QA experts build and automate your initial set of test cases. After training, your QA staff will be able to easily create and load additional test cases and software, including software from other vendors, onto the system. Benchmark files will be uploaded or created and verified. When you have a change to test, you simply load the new test case and the automated test runs will take over. QA in the Cloud will automatically generate and email reports to you, and your QA staff will use a browser based dashboard – available from any physical location – to monitor and view testing results. The service can also be utilized by your development team to verify their fixes before committing them, and test runs can be easily scheduled any time your staff needs to do so.
Why QA in the Cloud?
QA in the Cloud is the logical choice to address today’s and tomorrow’s rapidly evolving quality assurance needs. QA in the Cloud can be instrumental in helping organizations achieve their quality and efficiency goals. Typical QA in the Cloud use cases include:
- Regression testing new CCM software releases and patches
- Regression testing application design and setup changes for CCM components
- Regression testing application program changes
- Testing migrations from one system to another
- Verification of outsourcing/insourcing results
- User Acceptance Testing (UAT)
- Rate table change testing
- Initial application testing
QA in the Cloud Benefits
For organizations that want the best in quality control for their CCM solutions and components eliminating errors in customer communications and streamlining change management processes — QA in the Cloud provides a no-compromise comprehensive solution that can be deployed quickly without the investment, hassles, and ongoing costs related to in-house QA operations. QA in the Cloud enables organizations to
- Reduce errors through improved quality control
- Avoid errors causing unforeseen and costly consequences
- Mitigate regulatory non-compliance risks
- Experience faster time-to-market by leveraging quicker release schedules
- Maintain more reliable release cycles can be maintained
- Better utilize quality assurance staff
- Reduce Quality Assurance investments and cost by leveraging CrawfordTech’s infrastructure
- Save money, time and resources recovering from errors in customer communications
- Eliminate risks and costs of data breaches by redacting private sensitive information from test files
The service is hosted on CrawfordTech’s private cloud platform that is PCI-DSS, HIPAA and SOC2 compliant, offering complete security for you and your customers. With CrawfordTech QA in the Cloud you can outsource your document QA infrastructure to experts and focus on what you do best – running your business.
In its current configuration, QA in the Cloud has been used internally for 8 years by Crawford Technologies, and it has been instrumental in the maintaining the high quality of our software. This quality is reflected in our most recent Net Promoter Score® of +68 and our customer support rating of 9.75/10.
