Never neglect regression testing. Do Microsoft One Version updates entail additional costs? Yes, if you do not test them. If you do not run regression testing, updating – or even adding modifications and functionalities – your Dynamics 365 Finance and Operations is at risk of malevolent defects.
How to ensure you minimize that risk and do not incur any unnecessary costs in the process? By regression testing. In the below article, you will find out how you can run regression testing with Microsoft proprietary software – Regression Suite Automation Tool (RSAT) and also learn what are its benefits and drawbacks.
Regression testing with RSAT
Most of the businesses choose to customize Dynamics 365 Finance and Operations and so they need to change its standard features. Such attempts – so as not to impact the business processes – need to be tested for system regression. Once you decide not to give up on regression testing, another question arises – how to implement it effectively in your company?
A big challenge when it comes to regression testing is that when done manually, it devours gigantic amounts of time (not to mention resources and testing budget money). It is because manual regression testing requires preparation of myriad test suites and test cases for each of them. It is not to say that tis cannot be carried out by the testing team. However, the entire process will consume a vast amount of hours that can otherwise be spent on more productive activities.
That is why an automated testing framework – such as Regression Suite Automation Testing tool – is a must. The more complex the system and the more customizations you need, the more sense (and money) it makes to use an automated testing toolset. RSAT is particularly useful in the case of an ERP systems whose application environment needs to be frequently updated – such as Dynamics 365 Finance and Operations (F&O).
RSAT – how does a D365 F&O automated testing tool work?
Regression Suite Automation Tool (RSAT for short) is a Microsoft tool dedicated to automatic regression testing of Dynamics 365 environment. You can benefit it in many ways – with RSAT, you are able to test your Dynamics 365 Finance and Operations environment either during or after your system implementation (or upgrade process).
RSAT works in quite a simple way. Version 2.2. allows better usability and improves productivity. It enables you to create and maintain the test cases and attachments in the tools itself. In its earlier versions, it was not possible – you had to use Microsoft Azure DevOps to maintain the test cases and only then could you switch to RSAT for test execution, which was burdensome.
All the test plans and test suites can still be maintained in Azure DevOps. If you want the changes made in Regression Suite Automation Tool to be automatically uploaded to Azure DevOps, you only need to enable the Enable upload to Azure DevOps option. And this means that the test suites will include the updated test cases that are available to other users. Also, the created test cases can in run in DevOps by a pipeline.
With Dynamics 365 RSAT, you can provide it with an instruction (so to say) which would contain a specific description of the process. Then, RSAT will execute it automatically. You can use the help of built-in Dynamics 365 Task Recorder in order to create such an instruction. RSAT allows you to record almost any business process and then save it as a recording.
RSAT – benefits of Dynamics 365 automated testing tool
Now, let’s take a look at what benefits Regression Suite Automation tool (RSAT) offers.
This automated testing tool, first of all, significantly reduces the time and cost of user acceptance testing (UAT) for Dynamics 365 Finance and Operations applications. UATs are typically required prior to any Microsoft One Version application update, before you apply custom code or configurations to your production environment.
With RSAT, you can record business tasks easily by using the Task Recorder and then convert those recording of processes into a whole suite of automated tests. This means that for testing, you do not need to have the source code written. No-code capability of RSAT is one of its greatest advantages.
RSAT is also fully integrated with Microsoft Azure DevOps. You can benefit from this integration for the purpose of: test execution, reporting, and investigation for any defects found along the way. Test parameters are decoupled from test steps and stored in Microsoft Excel files
Microsoft RSAT helps you document your business processes automatically, which limits the amount of manual testing required to be done by business owners. As with every agile automated testing tool, this allows you to increase ROI following every One Version update.
RSAT – technical aspects of Microsoft automated testing tool
Once Regression Suite Automation Tool is fully configured and the business process has been recorded with Task Recorded, you can start with creating your own tests.
Each of RSAT test case is based on a separate recording. On this basis, RSAT generates a set of files. These include executable files with C# code and an Excel file (a configurable one) that holds the test parameters
During test execution, RSAT – using Selenium framework – starts up a browser automatically, navigates to the system and repeats all the step as included in the given recording.
For each test case (and its successful execution), you need both a recording file and a test automation file (with test parameter files).
There are two option during test execution – the test can be either successful or it can fail. If the reproduced process runs smoothly, it means that the test run has ended successfully. However, if the program fails to execute any of the steps correctly, it will trigger an error message and the given test will fail.
RSAT for Dynamics 365 – automated testing tool implementation
The prerequisite for installing and configuring Regression Suite Automation Tool is having Dynamics 365 Finance and Operations platform of version 15 or later on your computer. Then, you will only need an Azure DevOps Test Manager or Test Plans license (if you have a Visual Studio Enterprise subscription, you already have a license to Test Plans).
RSAT must have access to your test environment via a web browser. RSAT is designed to be installed on any Windows 10 computer and connect remotely via a web browser to an environment.
Also, for generating and editing the test parameters, you need to have Microsoft Excel installed on your machine, too.
RSAT is a free software which means that you can download its latest version from the Microsoft version completely free of charge. The tool can be installed on any machine running Windows 10, Windows 11 or Windows Server 2016 (as you need to connect via a web browser anyway).
RSAT – weaknesses of the tool
If you care about business continuity and understand that regression testing is obligatory, RSAT will definitely be a helping hand. However, as with every automated testing tool, it is important to know its limitations beforehand so that you can compare it with other tools on the market but also to see, in the long run, if it does cater to your business needs.
The biggest problem with RSAT that is discovered during Proof of Concept phase is that the tool does not allow reusability. That means that the test that you create cannot be re-used for different test suites. This seriously impact the test project timeline and also means additional resources are required.
When you look at RSAT capabilities, it is also worth noticing that it does not allow all types of Dynamics 365 F&O testing. The main downside in this respect is that with RSAT you are unable to run performance testing which are equally important to regression tests. Unfortunately, this type of testing is not supported by this tool which means you either skip performance tests (not a recommended option) or use another automated testing tool (which obviously generates additional licensing or subscription costs).
Another important aspect when you compare automated testing tool compatibilities is time savings. When you consider the tool adoption time, for RSAT it takes 3-6 moths to implement the tool whereas, e.g., for Executive Automats, the adoption time is only two weeks. The average time for creating and running a simple automated test is 5-10 minutes with RSAT while other tools offer around 3 minutes for this task. Simultaneous test runs are also quite limited and time-consuming. An average time needed to execute 100 scripts, 3mins each with Regression Suite Automation Tool is 300 minutes as they need to be run one by one. Tools such as Executive Automats enable you to run even 20 sessions in parallel (see the screenshot from the tool below):
