Dynamics 365 is a world-class ERP and CRM. It makes your people more productive and efficient, while driving down business spending.
But like any software platform, it also comes with both direct and indirect costs. These include licenses, consultants’ fees, and maintenance.
And one of the bigger costs associated with the platform is Dynamics 365 testing. Since Microsoft pushes out two major release waves a year – plus several smaller updates – you must constantly check these to ensure they don’t ‘break’ your environment.
This testing costs money. But most organizations I speak to don’t realize how much.
So, in this article, I’ll be explaining what Dynamics 365 testing costs, both directly and indirectly. I’ll also compare automated testing of Dynamics 365 with the manual approach.
The direct and indirect costs of Dynamics 365 testing
Every time Microsoft releases an update to Dynamics 365, it’s important to do a full round of regression testing.
Failing to do these tests could cause a huge amount of disruption. If an update locks people out of forms, returns internal server errors, or stops data from being synced across apps, it could cause serious damage to your business and reputation.
Doing this testing is therefore an essential maintenance task. It should identify any bugs or errors, so your dev team can resolve them before rolling the update out.
However, testing inevitably costs money. Below, I’ve outlined the direct and indirect costs of different testing methods.
Costs of manual Dynamics 365 testing internally
This is the most common approach that I come across at organizations. The IT department asks lines of business users to do a series of tests in the test environment each time an update is released.
In most cases, these testers are subject matter experts drawn from departments like finance, sales, logistics, or operations. They’re the people who know the day-to-day processes best, and are often the “go-to” tech person within their team, but they are not developers.
It’s typically something a fairly experienced employee will need to do. This is because the tester has to know the business processes and all the customizations around it to be able to test them thoroughly and make sure they still work well after the update.
Direct costs of internal Dynamics 365 testing
The obvious direct cost is your employees’ time. You need to multiply the hourly rate of your workers by the number of hours it takes them to do the testing. As I mentioned, testing is usually done by more senior staff, whose hourly rate will be higher.
For a fairly simple Dynamics 365 implementation at a small company (less than 150 users), I’d expect it to take at least 160 hours for the entire company to go through a regression test cycle, with many key users from different departments taking part in the process.. But for more complex implementations at large companies and multinationals, it can easily eat up hundreds of hours. And, since testing is carried out several times a year, the cost estimate is usually much bigger than one would think.
Indirect costs of internal Dynamics 365 testing
The hidden or indirect costs of D365 testing are, in my opinion, just as significant as the direct costs. While it’s not always easy to quantify, they can really add up:
- Opportunity cost: If your experienced business users are spending dozens or even hundreds of hours on testing each time you need to go through a regression test cycle, their labor and productive time is diverted away from more valuable activities. If a finance controller, customer service leader, or regional manager is spending time on testing, they’re not able to do the things that actually contribute to your business’s growth.
- Effect on morale: I’ve never met a business user who actually enjoys Dynamics 365 testing – especially by the time they’ve done multiple cycles in a single year. It’s boring, tedious, and frustrating work. In the worst-case scenario, people might just quit – if you didn’t sign up to be a software tester, you might well start looking for a job somewhere else.
- Reliability problems: The reality of manual Dynamics 365 testing is that people start to take shortcuts. If you have to keep doing the same tests every few months, but can’t see any change, then you might be tempted to tick the box saying a test has been done, even if you haven’t actually completed it. This might not be an issue at first… until something breaks because an employee got bored of the endless D365 testing.
Related: How Dynamics 365 test automation cuts testing time by 80%
Costs of outsourced Dynamics 365 testing
The second approach to D365 testing is to outsource it. This involves finding a testing company and giving them access to your test environment. Then, you show them what tests you want them to do, and how.
Outsourcing is often quite challenging with Dynamics 365 testing, since the testers need to have a good knowledge of your business processes. They will also need to have access to confidential information to complete their job effectively, which raises a lot of privacy and security issues. But it can work for some companies in some industries.
Direct costs of outsourced Dynamics 365 testing
How much you pay for outsourcing depends on what you need and where you’re sourcing the work. In some markets (e.g. India, the Philippines), you can often find regression testing services at fairly affordable prices. If you’re outsourcing to a local company in a higher-income country, it will usually cost much more.
Indirect costs of outsourced Dynamics 365 testing
These indirect costs are, in my opinion, just as significant as the outsourcing company’s fee:
- Time spent specifying: You need to spend a significant amount of time specifying which tests need to be completed, documenting exactly how you want them to be done, and explain how issues should be reported on. This will amount to several days’ worth of upfront work.
- Training, feedback, and quality control: You will also need to spend a fair amount of time explaining how you want the work completed. It’s also a good idea to do regular spot checks to ensure quality is hitting the required standard.
Costs of Dynamics 365 test automation
The last approach is to automate Dynamics 365 testing. Using a solution like Testing by XPLUS, you train the testing tool to perform the tests. Then, once this is done, the system can repeat each test in exactly the same way, as often as you want.
Direct costs of automated testing for Dynamics 365
The direct cost of automated Dynamics 365 testing tools vary depending on the size of your organization and the number of tests that need to be completed. But, once the tests have been automated, you can run a full regression test cycle as often as you want, even for the tiniest change in the system, for no extra cost. Compared to manually doing tests internally or via an outsourcing company, automated testing software is always far cheaper.
To get an estimate of how much you could save when you automate Dynamics 365 testing, use our ROI calculator today.
Indirect costs when you automate Dynamics 365 testing
There are still some indirect costs of Dynamics 365 test automation, but these are lower than with manual testing.
- The first ‘training’ round: Your business users need to train the software to do all their usual tests. With Testing by XPLUS, this is quick and easy – they simply record themselves doing the process. No technical skills or workflow mapping are needed.
- Some remaining manual tests: While most tests can be automated, around 10-20% of Dynamics 365 tests will still need to be checked manually. These will be the ones that can’t be validated automatically, for example, confirming that a report displays the right colours or layout.. So manual testing will still take up time, but nowhere near as much as 100% manual testing.
- Test maintenance: Occasionally, you might need to update your automations (for example, if you change a specific process). Each time you do this, you’ll need to modify your test scripts around the process, though this usually only takes a few minutes.
Read more: Calculating the ROI of Dynamics 365 test automation
Automate Dynamics 365 testing and avoid hidden costs
With so many of our customers, I’ve seen first-hand how automated testing for Dynamics 365 delivers major cost savings. Besides the direct costs of employee time, it’s all the other indirect costs that this approach helps avoid that are, in my view, the most important. From improved morale, to letting people spend more time on productive, fee-earning tasks, automation just makes a lot more sense.
If you’re interested in the potential of our solutions for automated Dynamics 365 testing, contact me today for a demo.
Frequently asked questions about the costs of Dynamics 365 testing
We answer your questions on D365 testing costs.
How much does it cost to automate Dynamics 365 testing?
The cost of automated testing for Dynamics 365 depends on the automation tool you use and the number of tests you need to automate and of course, their complexity. With Testing by XPLUS, you pay a competitive upfront license fee, then you can automate as many tests as you wish. The only other cost is the time needed to ‘train’ the solution. You will need to create test scripts that cover your entire test scope, and that takes time. A competent partner may help you deliver the automation faster, but their services will be an additional cost. Once your test scripts are ready, you can run them whenever needed and as many times as needed, at no additional cost.
We’re a small team and we don’t have that many tests. Would automated Dynamics 365 testing save us money?
Yes, almost any organization can save money when they automate Dynamics 365 tests with Testing by XPLUS. Even if you’re only automating half a day’s work each month for 10 SMEs, that still adds up to nearly 500 hours worth of work per year. That is surely time that could be better spent on productive, fee-earning activities. Also, if you don’t have many tests defined, it may mean you are not testing your application thoroughly enough and thus taking an unnecessary risk that a catastrophic event like system downtime may happen because of a bug that escaped your testing process.
How to reduce the costs of Dynamics 365 testing?
The most effective way to reduce the costs of Dynamics 365 testing is to use test automation tools. Once you have set the tests up, they are endlessly repeatable, always reliable, and with complete consistency. With automated testing you are able to cut the amount of time you spend on manual testing by 80%, or more.
