Remediation Testing Sample Size 1385



  • Does anyone know if there is any guidance on the sample sizes for remediation testing? I thought I had seen somewhere that it should be the same as your original sample size, but now I can’t find where I saw that at.
    Thanks.



  • The sample size is largely based on judgment, and the expectations/guidelines from the independent auditor.
    There have been a numbe of posts on sample sizes, which have been very good discssions, lately.
    IMO, the sample size would not decrease (ever) but may increase to the upper end (if not top) of the range for the frequency of the control. At that point, if there were any exceptions noted, the sample size should be adjusted and evaluated regardless of the fact that the Key Control was due to remediation. (That is, if exceptions are noted, double the sample size and re-test.)



  • I believe that the sample size should be the same as if you were testing the control fresh. You should not test a remediated control for effectiveness until after it has been operating for a period of time long enough to allow you to select a representative sample.



  • I agree with Mike’s posting from earlier and consider it appropriate to use the same sample sizes and methodology that was used to perform the initial tests of controls effectiveness.
    After controls are remediated, use the same sampling approach, and if control deficiencies are found, follow the guidelines that are outlined in the document, ‘A Framework for Evaluating Control Exceptions and Deficiencies’.
    I’ve not seen a guidance docuent that prescribes sample sizes that can be used to plan tests of operating effectiveness after remediation. If such a document were developed, sample sizes would still be determined by the materiality threshold, sampling risk, and other sampling considerations.
    If there’s any relief to reduce sample sizes going forward, I suggest transitioning as many manual controls to system controls as possible. System controls do not require repeated testing if effective change management practices are in place.
    Hope this further helps,
    Milan



  • This link below may be useful for guidance on Sampling. The document contains some good resources including sampling tables, instructions for sampling using Excel, etc.
    nao.org.uk/publications/Samplingguide.pdf
    Hope this also helps,
    Milan



  • Folks,
    Actually depends on the perception of external auditors. It’s always a good idea to confirm the testing and the re-testing sample sizes with Independent Auditors. On one client the re-testing size was exactly half the size of the testing size. On another client both sizes are same.



  • I agree with Ari (Chhaava ) in that (philsophical) sampling discussions do not ultimately matter since the independent auditors must agree. While the larger firms have fairly rigid guidelines (in this and everything else) which are failrly widely known, it would be better to stick to those. Otherwise, if dealing with a small firm that may have less rigid standards, but has some wacky idea (just because they are on a learning curve), you may be able to sway, but I wouldn’t necessarily fight the current out of principle.
    In any case, best to ask the auditor first.
    Ari: I’m surprised that in the example that you cited that the sample size decreased. Is it because of a decreased population? or, some other reason?
    I would have thought that (as kymike said), the sample size would remain constant as if you were testing the control for the first time.
    Just curious.
    John



  • To clarify:
    Let me give you a live case with E-and-Y (being the outsourced SOX efforts partner) on one of our audits.
    For a daily control, the original size was 25. More than two findings led to a re-testing plan for the ineffective control. E-and-Y convinced KPMG (independent auditors) for the re-test sample size of 15, based on the fact that on finding one deficiency, the original test sample of 25 is extended by 15 more sample to prove one deficiency as isolated.
    At other client, E-and-Y being the independent auditor forced us to re-testing sample size as same as for original testing.
    Mind you, re-testing sample size depends on the timing also. Quarterly remediation effective from the last quarter would have only one sample size.
    The big four auditors have their set sample sizes suggested to their clients. E-and-Y has 25 for daily controls, PWC has 30 and KPMG has 40, based on my experience with them for some clients.
    I hope that I could answer.



  • The sample size will vary depending on whether you are extending your normal testing when one exception is found (by adding 15 to the original sample of 25) or whether you are retesting a control that was originally found to be deficient and where the deficiency has been claimed to now be remediated (by testing with the normal sample size).



  • In discussions with external auditors, we were able to firm up the size of for retesting.%0A1) the first client was allowed to retest with 50% of the original sample size. if everything passed in the retest then all was ok. otherwise, it would be a defficiency.%0A2) another client, we were allowed to pull another sample of 10 and retest. we were allowed to pull samples of 10 each time there was 1 failure up to 3 times total. this was the guidance recieved by the partner of the external auditor.%0AIn both cases, we stated the approach we would take on retesting and the external auditors provided feedback. we tailored our feedback based on the discussion.



  • This morning I met with our Independent Auditors (EY) about our Scope for 2006. They agreed on a sample size of 25 for original testing with an extension of 15 in case of one finding otherwise to be considered a deficiency.
    Re-test sample should be 15.
    Well the sample size is more relevant for daily and multiple times daily controls. I was not about concerned sample sizes for other frequencies such as weekly (13 samples for original test) and monthly (3 samples for original test)
    I hope that this helps


Log in to reply