Accounting System/ERP Customer Survey

By Michael Burns published in the CAmagazine April 2006

Welcome to the results of our second annual customer survey of accounting and ERP systems. The survey was announced in the September 2005 print edition of CAmagazine, and can be viewed at http://www.camagazine.com/index.cfm/ci_id/27849/la_id/1.htm. Once again. the idea was to see how well readers like the system they’re using, and how they rate the developers and implementation partners. We also asked for some general feedback about return on investment and future plans. Here in this online article, you’ll find additional information – including charts – that space constraints prevented us from running in the print version.

To ensure the validity of the results, we accepted only one survey per organization and it had to be filled out by an accountant with a CA, CMA, or CGA designation. This is a change from last year, when we allowed only CA designations. We wanted to open up the survey to companies that don’t have CAs on board.

We received 264 valid responses compared with 129 last year. The increase may be due to the CMA and CGA responses. Also, vendors are beginning to recognize the importance of this survey and have encouraged their customers to complete it. In fact, some of the value-added resellers of accounting and ERP systems also submitted responses. Naturally, we rejected these because of the resellers’ bias.

We also gave potential respondents an additional incentive this year by holding a draw for free entry to The National IT Conference and Showcase for Accountants, to be held June 12-13, 2006 in Toronto. The lucky winner was Karen Kojima, manager of treasury & investment accounting at the Ontario Pension Board. Congratulations, Karen!

RESPONSES BY SYSTEM

As you can see, it’s a fragmented market, with the “other” category (one-offs with only one survey response) accounting for 22% of the responses. Don’t be too quick to judge market share based on the number of responses, because it seems some of the vendors were more successful than others in encouraging their clients to complete the survey. That said, it wasn’t surprising to see a similarity in the number of responses for QuickBooks and Simply Accounting, since these two systems dominate small business in Canada.

To compare the systems, we split them into high end, mid-market and small business. And to improve the statistical validity of our analysis below, we also excluded those with fewer than five responses. At the high end, the battle is between SAP and Oracle. As you may know, Oracle now owns JD Edwards and PeopleSoft. In the mid-market, the big contenders are Sage (which purchased ACCPAC Advantage), Microsoft (which purchased Great Plains and Navision) and SYSPRO.

HIGH END

MID-MARKET

System	Responses	% of total
ACCPAC Advantage	14	22%
Great Plains	15	24%
Navision	6	10%
SYSPRO	28	44%
Total	63	100%

SMALL BUSINESS

System size

The following chart will give you an idea of the size and costs related to high-end, mid-market and small business systems.

Overall ranking

We asked each respondent to score his or her system overall for financial, distribution and manufacturing functionality. The possible responses were excellent (4), good (3), fair (2), poor (1) and N/A. (We excluded N/As from our statistics, along with products with fewer than five responses. Note that this approach applies to all the remaining statistics.) We will give credit to high-end and mid-market systems with high scores of 3.5 or greater. We have not recognized high scores for small business because of a lack in comparability to mid-market and high-end systems. Small business systems cost hundreds of dollars. The average for mid-market systems in our survey was about $60,000 for licence fees. High-end systems averaged in the millions of dollars. Naturally, the customers of mid-market and high-end systems are going to want a lot more value for their investments.

Overall, respondents liked their financial systems the most, followed by manufacturing and distribution. Looking at subtotals by tier for financial and distribution systems, the high end, on average, did not score as well as the lower tiers. This may seem strange, but is due to greater expectations on the part of high-end organizations. SAP and Great Plains (Microsoft Dynamics GP) garnered high scores (3.5 and up) for financial systems, and SYSPRO did the same for distribution.

HIGH END

System	Financial	Distribution	Manufacturing
JD Edwards	2.8	2.7	2.7
Oracle	3.2	N/A	N/A
PeopleSoft	3.2	2.0	2.0
SAP	3.5	2.8	3.0
Average	3.2	2.5	2.6

MID-MARKET

System	Financial	Distribution	Manufacturing
ACCPAC Advantage	3.4	3.0	2.2
Great Plains	3.5	2.9	2.6
Navision	3.2	3.0	3.0
SYSPRO	3.4	3.5	3.3
Average	3.4	3.1	2.8

SMALL BUSINESS

System	Financial	Distribution	Manufacturing
Adagio	3.7	3.3	3.0
MYOB	3.8	2.5	1.5
QuickBooks	3.2	2.7	2.1
Simply Accounting	3.4	2.8	2.4
Average	3.5	2.8	2.2

Generic attributes

In terms of  ease of use, flexibility, stability, security, documentation, reporting, customization and value for money, the high-end systems didn’t measure up to mid-market and small business systems, especially when it came to value for money.

Notice that small business systems have consistently higher scores than their mid-market and high-end counterparts. This says a lot about the quality of these systems, but also reflects lower expectations; on average, customers spent only a few hundred dollars to acquire them. High scores for flexibility go to Oracle, Navision (Microsoft Dynamics NAV) and SYSPRO, stability to SAP and SYSPRO, and value to SYSPRO.

We can’t help but notice the high scores for MYOB for flexibility, customization and value. What’s strange about this is that in 2001 Intuit Canada acquired the Canadian operations of MYOB Canada. MYOB apparently decided to divest the Canadian operations as part of a strategy to focus on markets and countries where MYOB has more growth opportunities. The MYOB respondents are all working with older releases of the software. One of our questions was the date of the last upgrade and the average MYOB response was five years ago.

HIGH END

System	Ease of use	Flexible	Stable	Secure	Doc’n	Rpt’g	Cust’n	Value
JD Edwards	2.2	3.0	3.0	2.8	2.4	2.6	2.2	2.5
Oracle	3.0	3.6	3.8	3.4	2.6	2.8	3.2	2.0
PeopleSoft	2.5	2.7	2.8	2.7	2.3	2.5	2.3	2.3
SAP	2.6	3.1	3.5	2.9	2.8	2.5	2.5	2.5
Average	2.6	3.1	3.3	2.9	2.5	2.6	2.6	2.3

MID-MARKET

System	Ease of use	Flexible	Stable	Secure	Doc’n	Rpt’g	Cust’n	Value
ACCPAC Advantage	2.9	3.2	3.4	2.9	2.8	2.1	2.4	2.9
Great Plains	3.1	3.3	3.1	2.9	2.9	2.6	2.9	3.1
Navision	3.2	3.5	3.2	2.2	2.3	3.2	3.2	3.0
SYSPRO	3.1	3.6	3.4	3.0	3.0	3.0	3.4	3.5
Average	3.1	3.4	3.3	2.7	2.8	2.7	3.0	3.1

SMALL BUSINESS

System	Ease of use	Flexible	Stable	Secure	Doc’n	Rpt’g	Cust’n	Value
Adagio	3.4	3.8	3.4	3.1	3.2	3.2	3.7	3.6
MYOB	3.4	4.0	3.4	3.0	3.2	2.8	4.0	4.0
QuickBooks	3.2	3.5	3.0	2.8	3.0	2.7	3.3	3.3
Simply Accounting	2.9	3.3	3.3	3.2	2.9	2.3	3.4	3.6
Average	3.2	3.6	3.3	3.0	3.1	2.8	3.6	3.6

Developer ratings

For overall satisfaction, continuous improvement, documentation, support and upgrade reliability, most of the systems could use some improvement, except for the small business developers. SYSPRO scored well in the “continues to improve” category.

HIGH END

System	Overall	Continues to improve	Documentation	Support	Upgrade reliability
JD Edwards	2.8	2.6	2.6	3.0	2.5
Oracle	3.2	3.2	2.8	2.4	3.2
PeopleSoft	2.2	2.0	1.5	1.8	2.0
SAP	2.8	3.0	2.5	2.6	2.8
Average	2.7	2.7	2.4	2.5	2.6

MID-MARKET

System	Overall	Continues to improve	Documentation	Support	Upgrade reliability
ACCPAC Advantage	2.8	2.6	2.5	2.5	3.0
Great Plains	3.0	3.1	2.9	2.9	3.0
Navision	2.8	3.0	2.3	2.6	3.0
SYSPRO	3.3	3.5	2.8	3.4	3.3
Average	3.0	3.0	2.7	2.8	3.1

SMALL BUSINESS

System	Overall	Continues to improve	Documentation	Support	Upgrade reliability
Adagio	3.6	3.7	3.0	3.6	3.6
MYOB	3.5	2.0	2.0	1.3	2.5
QuickBooks	3.2	3.0	2.7	2.8	3.1
Simply Accounting	3.3	3.3	3.0	3.0	3.3
Average	3.4	3.0	2.7	2.7	3.1

Implementer ratings
For overall satisfaction, adherence to schedule and budget, support, product knowledge and ability to improve business processes, the implementation partners for mid-market and small business systems came out ahead of their high-end peers. This is ironic in that the implementation partners for high-end systems are usually very handsomely paid and work for large consulting practices. That said, the systems are more complex, so perhaps this explains the lower ratings. Congratulations to the implementers of SYSPRO, which was the only system that landed high scores in most of the implementer rankings.

HIGH END

System	Overall	On time	On budget	Support	Product knowledge	Improve processes
JD Edwards	3.2	3.0	2.6	3.7	3.4	2.2
Oracle	3.0	2.8	3.0	3.3	3.0	3.0
PeopleSoft	2.5	2.7	2.5	2.2	2.2	2.2
SAP	2.8	2.5	2.4	2.4	2.6	2.6
Average	2.9	2.7	2.6	2.9	2.8	2.5

MID-MARKET

System	Overall	On time	On budget	Support	Product knowledge	Improve processes
ACCPAC Advantage	2.6	2.5	2.6	2.4	2.6	2.3
Great Plains	3.2	3.1	2.8	3.1	3.0	2.9
Navision	3.0	2.8	2.7	2.8	2.8	2.5
SYSPRO	3.7	3.7	3.6	3.6	3.7	3.4
Average	3.1	3.0	2.9	3.0	3.0	2.8

SMALL BUSINESS

System	Overall	On time	On budget	Support	Product knowledge	Improve processes
Adagio	3.6	3.5	3.3	3.6	3.4	3.4
MYOB	3.7	3.3	3.7	2.0	2.7	2.7
QuickBooks	3.1	3.1	3.3	3.1	3.0	3.1
Simply Accounting	3.3	3.3	3.3	3.1	3.3	2.9
Average	3.4	3.3	3.4	2.9	3.1	3.0

Return on investment
We asked about payback (no. of years), improved customer service (ranging from a high of 4 to a low of 1; these numbers apply to the rest of the questions in this category), increased accuracy, increased revenue, increased control, and better decision-making. The high-end systems have a long payback time – almost four years compared with three years for mid-market and one for small business. In the other categories for return on investment, small business systems scored consistently higher than mid-market systems; and the mid-market did better than the high end. Again, this could have a lot to do with expectations. Kudos once again to SYSPRO for high scores in improved customer service, accuracy, control and decision-making.

HIGH END

System	Payback in years	Customer service	Accuracy	Revenue	Control	Decision-making
JD Edwards	5.3	2.4	2.4	2.0	2.5	2.8
Oracle	2.7	3.0	3.4	2.3	3.4	3.2
PeopleSoft	4.5	2.7	2.8	2.3	2.8	3.0
SAP	3.1	2.8	3.0	2.0	3.1	2.6
Average	3.9	2.7	2.9	2.1	3.0	2.9

MID-MARKET

System	Payback in years	Customer service	Accuracy	Revenue	Control	Decision- making
ACCPAC Advantage	2.9	2.8	3.3	2.6	3.4	3.2
Great Plains	2.7	2.7	3.1	2.1	2.9	3.1
Navision	3.7	2.8	3.0	2.3	3.3	3.2
SYSPRO	2.4	3.5	3.6	2.8	3.5	3.5
Average	2.9	2.9	3.2	2.5	3.3	3.2

SMALL BUSINESS

System	Payback in years	Customer service	Accuracy	Revenue	Control	Decision- making
Adagio	1.5	3.5	3.3	2.4	3.6	3.5
MYOB	0.5	2.5	3.0	2.5	3.3	3.6
QuickBooks	1.2	3.0	3.3	2.7	3.1	3.1
Simply Accounting	0.6	3.4	3.7	2.6	3.3	3.3
Average	1.0	3.1	3.3	2.6	3.3	3.4

Costs
We asked respondents what percentage of the total cost of implementation they spent over the past five years on licence, maintenance and external consulting fees, upgrade costs related to computer infrastructure and internal costs. Our results show that licence fees as a percentage of total costs are the lowest with high-end systems, followed by mid-market, then small business. The high-end and mid-market customers are spending more on external consulting and internal costs.

System	High-end % costs	Mid-market % costs	Small business % costs
Licence fees	25	29	57
Maintenance fees	14	16	20
External consulting fees	17	16	4
Customization	8	13	4
Upgrade infrastructure costs	13	13	5
Internal costs	24	15	10
Total	100	100	100

Planned investments
Finally, we asked what accounting/ERP system investments were planned over the next two years – replacing the existing system, upgrading to the next version, adding customer relationship management (CRM), e-commerce, business intelligence (BI), budgeting and/or forecasting, consolidation, HR and supply chain management. As expected, a few companies with high-end or mid-market systems are planning to replace their systems, but many more will be upgrading. Replacing an ERP or accounting system is a last resort because of the costs and effort involved. Many companies look to improve their business processes by other means, including upgrading their software. It also looks like the mid-market is the place to be for vendors of business solutions.

System	High end		Mid-market		Small business
	Count	%	Count	%	Count	%
Number of responses	24		63		82
Replace system	3	13%	4	6%	10	12%
Upgrade version	16	67%	46	73%	53	65%
Add CRM	5	21%	14	22%	8	10%
Add eCommerce	3	13%	20	32%	13	16%
Add BI	5	21%	11	17%	1	1%
Add budgeting	6	25%	18	29%	13	16%
Add consolidation	3	13%	8	13%	3	4%
Add HR	1	4%	8	13%	9	11%
Add supply chain management	2	8%	8	13%	4	5%

Summing up
Congratulations to the following vendors for achieving the highest scores:

Category	Systems
Financial systems	SAP and Great Plains (Microsoft Dynamics GP)
Distribution systems	SYSPRO
Flexibility	Oracle, Navision (Microsoft Dynamics NAV) and SYSPRO
Stability	SAP and SYSPRO
Value	SYSPRO
Developer	SYSPRO
Implementation	SYSPRO
Return on investment	SYSPRO

We hope our survey provided some useful insights. We were able to provide a lot more information this year because of the increased number of responses. Hopefully next year will be even better. 

2005

By Michael Burns published in the CAmagazine January 2005 (The CAmagazine article also contains charts)

Welcome to the results of our first-ever customer survey of accounting and ERP systems. The idea was to see how well users like the their system, as well as their views on the developers and implementation partners. We also asked for some general feedback about return on investment and future plans. Here in this online article, you'll find additional information - including charts - that we were unable to run in the print version due to space constraints.

To ensure the validity of the results, we accepted only one survey per organization, and it had to be filled out by a CA. We made this restriction because CAmagazine is for chartered accountants, and we knew CAs wouldn't misrepresent the numbers (this would also enhance the reliability of our results). Since our sample size is limited - a total of 129 valid responses - we have reported our findings on totals for all systems, subtotals by tier (high end, mid-market and low end) and given credit only to systems that drew at least three respondents.

What did we learn? First and foremost, that it's a fragmented market. More than a quarter (26%) of the surveys were one-offs; i.e., there was only one response for those systems. Some of the systems mentioned do not even appear in our annual ERP and accounting software survey. There are still many old legacy systems out there, which are not usually not marketed but are somewhat supported. Customers keep the systems because they generally work, although they are built with old technology and will eventually be replaced.
Overall, QuickBooks accounted for 16% of the responses, followed by ACCPAC (9%), Simply Accounting (6%), PeopleSoft (6%), Great Plains (5%), Navision (5%) and SAP (5%).

Overall ranking We asked each respondent to score their system overall for financial, distribution and manufacturing functionality. The possible responses were Excellent (4), Good (3), Fair (2), Poor (1), and N/A. (We excluded N/As from our statistics, along with products with only one response. Note this approach applies to all the remaining statistics.) Overall, respondents liked their financial systems the most, followed by distribution and manufacturing systems. Looking at subtotals by tier for financial and distribution systems, the high end, on average, did not score as well as the lower tiers. This may seem strange, but could have something to do with high-end organizations having greater expectations. But for manufacturing, the higher-end systems chalked up better scores. The best marks for financial systems go to J.D. Edwards (now part of PeopleSoft and called Enterprise One) and PeopleSoft.

Generic attributes We asked each respondent to rate their system for ease of use, flexibility, stability, security, documentation, reporting, customization and value for money. On average, the high-end systems didn't measure up to mid-market and low-end systems when it comes to customization and value for money. The customization response was not expected and again may be attributed to higher expectations. The value for money is not surprising and is supported by our return on investment results below. High scores for ease of use go to PeopleSoft, flexibility to SAP and SYSPRO, stability to J.D. Edwards and SYSPRO, documentation to SYSPRO and Acomba, and value for money to PeopleSoft. There were no high scores for security, reporting and customization.

Developer ratings We asked respondents to rate the developer of their system on overall satisfaction, continuous improvement, documentation, support and upgrade reliability. Most of the systems in all tiers could use some improvement. Only PeopleSoft ranked high in overall satisfaction.

Implementer ratings We asked each respondent to rate the implementer of their system on overall satisfaction, on time, on budget, support, product knowledge and ability to improve business process. The implementation partners for mid-market and low-end systems came out ahead of their high-end peers. This is ironic in that the implementation partners for high-end systems are usually very handsomely paid and work for large consulting practices. That said, the systems are more complex, so perhaps this explains the lower ratings. SYSPRO was the only system that landed high scores in most of the implementer rankings.

Return on investment This time we asked about payback (no. of years), improved customer service (ranging from a high of 4 to a low of 1; these numbers apply to the rest of the questions in this category), increased accuracy, increased revenue, increased control, and better decision-making. The high-end systems have a long payback time - almost five years compared to mid-market (2.6) and low-end (1.8) systems. Scores for the various systems were fairly similar in other areas except decision-making; here the high-end systems did not rate very well. This may seem strange, since high-end companies often invest in tools such as business intelligence to improve decision-making. But their decisions may also be more complex, which accounts for the scoring. Fastest payback goes to QuickBooks and Simply Accounting. High ratings for customer service go to PeopleSoft and SYSPRO, accuracy to PeopleSoft, SAP, SYSPRO and Simply Accounting, increased revenue to SYSPRO, better control to J.D. Edwards, SAP and Acomba.

Costs We asked for the approximate percentage of costs over the last past five years for licence, maintenance and external consulting fees, upgrade costs related to computer infrastructure, and internal costs represented as a percentage of the total cost of implementation they represent, for the past five years.. Our results show that licence fees as a percentage of total costs are the lowest with high-end systems, followed by mid-market, then low-end systems. The high-end and mid-market customers are spending more on external consulting and internal costs.

Planned investments Finally, we asked what accounting/ERP system investments were planned over the next two years. We asked about replacing the existing system, upgrading to the next version, adding customer relationship management (CRM), e-commerce, business intelligence (BI), budgeting and/or forecasting and other functionality. A few companies with high-end or mid-market systems are planning to replace their systems; but a large percentage will be upgrading. It looks like many companies will be investing in other functionality, budgeting, CRM and eCommerce. Business intelligence is not attracting a lot of investment dollars, except at the high end.

Post mortem We had initially planned to provide more information for each individual system, but the limited sample size precluded that approach. Next year we plan to repeat this process, and hopefully will be able to share even more information with you.