Year 4 - change sample sizes? 1967



  • We are entering year 4 of compliance. In prior years Mgmt’s testing was applied the same to all controls based on frequency, regardless of risk, with sample sizes ranging from 1(annual) to 30 (many times per day). We are revisiting our approach and risk rating controls in year 4, with 3 different sets of samples depending on the risk. In reviewing sample sizes, we note that we have been much lower than what appears to be standard. We know our external auditors are using up to 60 samples for many times per day for high risk items. The questions is - do we stick with our small sample sizes, or do we revamp our methodology and test more? We are hesitant to change something that has been acceptable for so long and is not ‘broken’ but it seems to go against the practice of ‘testing more than your auditors’ and also appears to be way below what most are doing. We don’t want to create unnecessary work, and we will need to justify the change to upper management. Currently, the external auditors rely on our work to the fullest extent. Thoughts?



  • Hi - This is a good question. I’m more of an IT person and others might have better expertise on sample size requirements (certainly a search on ‘sampling’ will show lots of prior threads.
    I’d suggest the following:

    • Make sure that you’re meeting all the minimum baseline SOX compliancy requirements with respect to what’s required based on your company size, financial risk assessments, etc. (for example, if you changed audit firms later would your current practices hold up?)
    • Most importantly, make sure the sampling process isn’t just to satisfy audit, but are providing benefits to your company. Sampling is to help YOU ascertain and verify the financial risk is being properly controlled. If 30 samples are doing a satisfactory job, then adding 30 more probably won’t uncover any more decificiencies or add any benefits for the extra work required.


  • I would not try and fix something that is not broken. That being said, I woudl consider risk-weighting your testing. Test 100% of your sample size for high risk items, 60% for medium risk and 20%-30% for low risk items. In general, routine, transaction items are low risk. These are the daily or multiple times per day items, so even adopting a higher gross sample size may net a lower sample to be tested.%0AWe are likely going to follow something like this in 2007 - %0A High; medium;low%0AMultiple x per day 36;24;12%0ADaily 30;20;10%0AWeekly 15;10;5%0AMonthly 3;3;3%0AQuarterly 2;2;2%0AAnnual 1;1;1



  • This is the third company I’ve worked with re: SOX compliance.
    Sample sizes were pretty consistent at each company, but have to admit that our external did have an influence. PwC was notorious for overtesting, EY was very light, and now we’re a KPMG shop.



  • If you are going to test the operating effectiveness of your controls then in general I do not think there is much scope to change the sample sizes but I would consider adapting it according to risk levels and kymike’s suggestion is good.
    I think the potential for wins lie with your processes themselves.
    Firstly a revisit of your risk assessmemnt for each process may highlight those that your could now argue do not expose yourself to as much risk as you previously thought. Particularly based on the findings of your SOX work in previous years. This may allow you to either descope particular processes or downgrade them to low risk.
    Secondly you can reconsider you approach to testing. Does a low risk process require sample tests for operational effectiveness? I would suggest that for a low risk process a thorough walkthrough would be sufficient to provide the assurance you need that there is no risk of a material misstatement. Thus any talk of sample sizes is then negated (or perhaps reduced to one.).
    A little too bold perhaps? I suppose it depends on the nature of your business and the associated controls and processes.



  • I must say that I’m surprised that your external auditors have been ok with you testing less than them and have not deemed your work unreliable (as some Big 4 audit teams would).
    Nonetheless, i would be inclined to keep the sample size low if you can get away with it and you have a history of positive test results…



  • Have to say I regard 60 as being particularly high for a sample size for transactional type controls and I would be challenging the auditors on over-auditing (and fees.). I would not expect to see this in circumstances other than when an error has been found and the sample size is being extended to show it as a one-off.
    25-30 would appear to be about the norm for transactional (many times per day) controls although I have seen as low as 10-15 being regarded as acceptable - especially if processes are unchanged and there is no history of errors.
    Always remember through that this is Management’s Assessment and you should not feel compelled to have your testing standards set by your auditors. Proposed changes to AS2 should reinforce this view.



  • Got one question for you folks…Once you have a control defined as a key control can you still make it a low priority/risk control and reduce the sample size for testing purpose.
    Calvin



  • Got one question for you folks…Once you have a control defined as a key control can you still make it a low priority/risk control and reduce the sample size for testing purpose.
    Calvin
    Yes. We have buy-off from our external auditors to do this.



  • Got one question for you folks…Once you have a control defined as a key control can you still make it a low priority/risk control and reduce the sample size for testing purpose.
    Calvin
    Not sure I would use the term low priority as this contradicts calling it key.
    However, setting sample sizes can be judgemental and risk can come into this judgement - based on process stability, history (or absence) of prior error, system changes, people changes, etc.



  • Denis makes a good point. I would use the term low-risk versus low priority. All key controls will have varying degrees of risk of failure or risk of financial impact to the FS should the control fail. Sample sizes can be adjusted based on relative risk of the key controls. I would also suggest that for controls that failed in the last year of testing, that their risk level be increased one notch for testing purposes. In other words, a control that has a low risk of failure, but failed in the prior year should at least be considered a medium risk for testing in the current year.



  • Does anyone know about AICPA Audit Sampling and SAS 39? I am planning to buy AICPA book, but I want to know first if these will guide me on sampling for audit controls.
    Does AICPA Guide dives a sample size table? Does these table have a exception reference?
    Regards.


Log in to reply