Sample Sizes and _and_quot;cycles_and_quot; 687



  • :lol:
    I agree, Pulling 60 samples and getting screen shots is tedious.
    I have never had to pull that many so I guess I am lucky.
    I also didn’t say we capped the sample at 25 as if we find two exceptions we pull another 25.
    I usually get someone else to pull my screen shots for me and do the documentation on them as well :lol:
    In the two SOX jobs I was on last year I was the manager, so if I knew there were screen shots from you know where coming up, I got the first years to do that test, and then I just reviewed their work.
    Of course my work was then reviewed by the senior manager, and his work was reviewed by a partner, and then the national risk partner…



  • why do you need to pull screenshots? that’s more than is required to evidence testing.



  • SOX_Monster, I am guessing you are working for E-and-Y. They are the only Big 4 I have worked with so far (others are KPMG and PWC) that cap the sample size at 25.
    My company use D-and-T which has also given us a cap at sample size 25



  • As far as I am concerned you don’t need to pull 60 screen shots.
    But you do need evidence.
    Off the top of me head I cannot think of anytime that we actually pulled 25 screen shots for one test.
    I usually tell my people to document the testing steps, pick you selection, get a screen shot or two of a couple of the tests and then document that you observed the rest of the selections.
    We take an attitude that if it can be reproduced at a later date, i.e. a screen shot of an access log, then one screen shot if any, is all that is required.
    Screen shots of 25 different computers showing that they had a viable anti-virus that was up to date at the time of the test may be required though. (I actually conducted that test myself and I just attested to the up to date AV)



  • One thing that should be considered as well is if it is an automated control, or a manual control



  • for a daily process, we were given a sample size of 50-60. and yes they do expect screen shots or some kind of evidence for all of them. an important clarification we just got is that the sample size is on an annual basis. this is important for us because we are doing sox testing now and we will do it again in 4th quarter. this means we would pull 25-30 now and then another 25-30 for the later round of testing.



  • Has anyone heard of a ‘rule of thumb’ for samples sizes that if you know the total population then you pull 30% for your sample (even when the external auditor or test script asks for a larger amount)?



  • Hi ugogirl,%0AThis sounds like it would lead you to performing a lot more testing than is necessary for the more frequent controls. For example, if a control were exercised on a daily basis, the population would be 365 over a year. 365 x 30% = ~110 samples. Not to mention some controls may be execised 3 times per day or more… 8O %0AOn the other hand, for a monthly control, this would lead you to a sample size of 4, which is in-line with the big-4’s. I think applying the 30% rule to anything beyond a weekly control frequency would create unneccessary burden. Maybe 30% for controls exercised weekly or less… 20% for annual controls… 15% for controls exercised more than twice a day…? Someone better with statistics could probably prepare the %0A’assurance’ curve for this one… 😮 %0AGood point about the control testing being performed on an annual basis - I know that some companies perofrming quarterly testing have not taken this into consideration and have ended up doing a lot more testing than would otherwise be necessary.%0ACheers,%0Alordkukuface



  • for a daily process, we were given a sample size of 50-60. and yes they do expect screen shots or some kind of evidence for all of them. an important clarification we just got is that the sample size is on an annual basis. this is important for us because we are doing sox testing now and we will do it again in 4th quarter. this means we would pull 25-30 now and then another 25-30 for the later round of testing.
    If the evidence of the control is a screenshot then this suggests automated control to me. If you are looking at automated controls then you can go down a GCC test of one route.
    Can’t imagine anything more pointless or soul destroying than pulling 50 screenshots for one control 8O



  • actually the screen shots are to show the approvals from user managment and IT management (external auditor wants to see: approval to start the project, approval of test results, and approval to migrate to production). these approvals are done via a software product that does help desk tickets, workflow, and change management. the only way to get the evidence is screen prints unfortunately.



  • You don’t need to keep any document as testing evidence that you can easily reproduce. Just ensure that your testing write-up covers what you tested and includes enough information to reproduce that testing. Usually, a test matrix with the attributes tested, the results of the tests and a conclusion as to the effectiveness of the controls is adequate.
    If you think that it is easier to keep the documents than reproduce later, that is a decision that you will have to make.



  • the external auditor has stated they want evidence in hardcopy stored in binders as part of the working papers. otherwise, we would take the easy route.



  • Who is your external auditing company?



  • It is certainly easier for the auditor to access if he doesn’t have to wait for you to recreate the screenshot. This may help to reduce auditor time and fees (though I can’t imagine that it will save them that much time)



  • HP,
    It would frowned upon here if I told you the name of the external auditing firm. However, I can say it is not one of the big 4.
    Thanks,



  • the external auditor has stated they want evidence in hardcopy stored in binders as part of the working papers. otherwise, we would take the easy route.
    Tell them to off :evil:
    If the evidence is held electronically then they need to review it electronically. SOX does not require you to do unnecessary work because your auditors are incompetent.


Log in to reply