Control frequency sample size 1640



  • I, too, have seen Key Control frequencies listed as ‘transactional’ and ‘as needed’.
    Personally, I think that it’s a cop out.
    The whole point of having categories of frequencies is to simplify and standardize the sample size with ‘some’ degree of precision.
    When a process owner tells me that a control is performed ‘as needed’, I ask: ‘when?’ and ‘how often is this control needed?’ and, ‘over the past year, how many times has it been performed?’ (trying to back into the population size and then fit it into a frequency category). I think that there is enough judgment involved with sampling and testing without adding a wildcard frequency, and then going off on a wild goosechase trying to find occurances of the control to test.
    If I cannot determine that the control is more than haphazardly performed, I seriously doubt that the control activity is really ‘Key’.
    I may be overly cynical, but if I cannot get comfortable that the as-needed ‘Key’ control is part of the routine accounting, I drop ot from the list of Key Controls and try to find other control activities that meet the control objective.



  • Thank you. I too have difficulty understanding how an ‘as needed’ could be designated a key control and it could be ‘as needed’ may reflect a design gap.



  • Another view on the ‘as needed’ control as a key control -
    Company A acquires Company B at the beginning of the year. Company A has not made a material acquisition in over 3 years.
    What ongoing controls would ensure that Company A properly allocates and records the purchase price of Company B in its consolidated financial statements? There would be no daily, weekly, etc. controls that would generally cover this. To me that’s where the ‘as needed’ controls come in. You would not test these controls for immaterial acquisitions or during periods when there was no activity.
    There may be a need to document controls over infrequent activities, but there should not be too many of these types of controls identified.



  • I know what you’re saying, but I think that in this case:
    Company A has not acquired another company in the current year, so the Merger/Acquisition Cycle (or whatever you would call it) is ‘Not Applicable’ (or, ‘out-of-scope’), so there would be no Key Controls for this cycle or the underlying due dilligence, valuation and recording processes.
    However, as far as past acquisitions, the carrying value of the related assets (from Company B acquired) presented in the financial statements would be periodically reviewed to ensure that the allowance is sufficient so that the net book value reflects the current (and possibly impaired) value.
    In this case, I would think that the CFO or Controller would perform this valuation analysis process quarterly (for the 10-Q) but no less than annually (for the annual audit and the 10-K).
    Then, any control activities within that quarterly/annual asset evaluation process would carry the same frequency.



  • Agreed. I should have been more explicit in my response to state that there are no controls with a regular frequency that would cover the valuation / recording of an acquired business in the year of acquisition if a company was not in the habit of making frequent acquisitions.



  • I have performed 2005 and 2005 sox audits on behalf of PwC and am currently managing sox for a 1st year compliance FPI who is audited by KPMG.
    In addition, some clients I have audited in the past employed Deloittes for internal testing and consulting.
    My experience is that the sample size guidance applied by Deloitte, PwC and KPMG are consistent in terms of Monthly, Quarterly and Annual controls, but vary significantly when it comes to the testing of daily and multiple times a day controls, with Deliottes having the lowest sample sizes and KPMG having the highest.
    Overall, your external auditors will expect you t apply sample sizes which, at a minimum, equal their own guidance, but would prefer if you picked a larger sample than they would intend to review.
    If you ask your external audit team what their intended sample sizes will be (they should be able to provide you with this), and, as a rule of thumb pick a sample size which is a little higher, you should be ok.



  • I should have stated earlier that as needed controls are the same as multiple times a day, with the exception being, that if the total number of occurences is lowere than your ppre-determined sample size, you should select ALL occurences for testing 😄



  • What about the controls that are performed twice a year? or each four months?
    Those are not frequently defined by customer, but I have a few who did.



  • It probably makes the most sense to test them like they were quarterly controls. It may be a little over-coverage, but it’s better than undercoverage.
    Hope this helps.



  • A couple of discussion points:%0A1) Sample sizes%0AOur sample sizes are as follows:%0AAutomatic controls - 1 sample%0AManual - Periodic%0AAnnual - 1%0AQuarterly - 2%0AMonthly - 3%0AWeekly - 5%0ADaily - 25%0AManual - As-needed%0ARelate size of population to a periodic control and use that sample size.%0Ae.g. Full population was 13, our logic says frequency was similar to monthly control, therefore select sample size of 3.%0AHowever, our external auditors (E-and-Y) had a different set of rules for As-needed sample sizes.%0ASample size = 10% of population. However a minimum of 5 and maximum of 25.%0Aeg:%0APopulation of 400.%0A10% = 40, but they limit sample to 25%0Apopulation of 200.%0A10% = 20. (correct as within 5-25)%0Apopulation = 30.%0A10% = 3, but has to be a minimum of 5, therfor sample = 5%0A2) As needed controls%0AA couple of people above have stated that they think there should be no controles listed as ‘as-needed’, and they should definitely not be listed as ‘Key’.%0AHowever, what about the HR function. There are specific controls in place in the event that a person is hired. It is not possible to relate these controls to a periodic event. (‘sorry folks, no new hires this month, the new hire for September is Johnny.’)%0AWithin the IT audit field, there are many controls that are listed as ‘as-needed’. e.g. changes to security access for staff (new hires, terminations, transfers), etc



  • Annual (low, medium, and high risk) - Test 1
    Quarterly (low, medium and high risk) - Test 2
    Monthly (low) - Test 2
    Monthly (high) - Test 5
    Weekly (low) - Test 5
    Weekly (medium) - Test 10
    Weekly (high) - Test 15
    Daily (low) - Test 20
    Daily (medium) - Test 30
    Daily (high) - Test 40
    Mult. per Day (low) - Test 25-30
    Mult. per Day (medium) - Test 45
    Mult. per Day (high) - Test 60

    Q: How much is necessary to test Low risk moreover when the financial impact is low as well.



  • We didn’t necessarily break our controls out by financial impact. 1) It’s hard to quantify, and 2) our externals preferred that we use standard sample sizes based on the above matrix. Because all of our controls are financial in nature, we don’t feel that we have to separate them further.
    Hope this helps.
    J



  • It helps.
    thanks
    R



  • Hi,
    If anyone has feedback to share about the number of control failures that would warrant the control to be deemed ineffective, that would be great.
    For example, please share any experience by providing the following info:
    Annual (low, medium, and high risk) - Test 1
    Maximum Error(s) allowed to conclude that control tested is operating effectively: 0?
    Quarterly (low, medium and high risk) - Test 2
    Maximum Error(s) allowed to conclude that control tested is operating effectively: 0?
    Monthly (low) - Test 2
    Maximum Error(s) allowed to conclude that control tested is operating effectively: 0?
    Monthly (high) - Test 5
    Maximum Error(s) allowed to conclude that control tested is operating effectively: 0?
    Weekly (low) - Test 5
    Maximum Error(s) allowed to conclude that control tested is operating effectively: 0?
    Weekly (medium) - Test 10
    Maximum Error(s) allowed to conclude that control tested is operating effectively: 1?
    Weekly (high) - Test 15
    Maximum Error(s) allowed to conclude that control tested is operating effectively: 1?
    Daily (low) - Test 20
    Maximum Error(s) allowed to conclude that control tested is operating effectively: ?
    Daily (medium) - Test 30
    Maximum Error(s) allowed to conclude that control tested is operating effectively: ?
    Daily (high) - Test 40
    Maximum Error(s) allowed to conclude that control tested is operating effectively: ?
    Mult. per Day (low) - Test 25-30
    Maximum Error(s) allowed to conclude that control tested is operating effectively: ?
    Mult. per Day (medium) - Test 45
    Maximum Error(s) allowed to conclude that control tested is operating effectively: ?
    Mult. per Day (high) - Test 60
    Maximum Error(s) allowed to conclude that control tested is operating effectively: ?
    Thanks,
    Milan



  • Hi Milan
    I thought on the multiples (i.e. more than 20x samples) if you got a failure - then you tested another batch - if no failures ok - if 1 or more goes into remediation (i.e. probability very low of systematic failure).
    Or at least thats how we have been working with our externals. Remediation testing is usually a month afterwards.
    Not sure that helps.



  • Yes, your comment helps, but what I was looking for is specifics for each control based on the control frequency (number of times that the control is performed, i.e., annually, semi-annually, quarterly, monthly, daily, etc.) AND the acceptable number of error(s) allowed under each scenario.
    From a practical standpoint, one can select a sample of 20 selections for testing, find one error, and select another 20 items, repeating this process until no errors are found. This approach seems inefficient, consumes limited resources, and at best, gives an inaccurate picture of the operating effectivenss of the internal control over financial reporting (ICFR).
    So if you or anyone else can provide detail (complete the ?s) for each test scenario, that would be more helpful.
    Milan



  • Hi Milan,%0AThe Maximum Error(s) depends on the number population. If you test e.g. 15 samples and the population is 150 or 300, the max error is different.%0AEverything is statistical counting. You have to consider %0A- Error rate of population (e.g. p=5%), %0A- Likelihood we can assert that error rate of population is maximum 5% (e.g. 5%) and %0A- Number of population (e.g. N=150). %0AThe theorem gives you, in this case, the n (testing sample) is 46. %0Anote: ‘p’ and +/- percentage is up to you%0AAnd result is: if you do not find within 46 samples any error you can assert with 95% likelihood that control works well - error § rate is not over 5%.%0AMaybe it helps or not%0AI hope that is understandable I am not sure about my English



  • Hi Ricker,
    Exactly what I needed…your reply was helpful and the example made it understandable/possible to easily implement in the SOX Controls Test Plans (in Excel).
    Thanks,
    Milan



  • Please keep in mind that Statistical sampling always give a much higher sample size which could be a drain on resources.
    The Big 4 auditors have streamlined their original sample sizes, re-test sizes, expansion sizes and rollforward sizes. These sizes are much lower than statistical size. We have been advised specifically by our external auditors that our statistically derived sample sizes are too high. Therefore, in 2005, we abandoned what Ricker suggested (regression model) and determined our original sample sizes, re-test sizes, expansion sizes and rollforward sizes based on our auditor’s expectations and understanding how they audit us and coupled with the fact me being having an extensive big 4 backgound.
    If you want I can share this guidelines with you.



  • Hi Arif,
    Of course…thanks for your input too…404cpa_at_gmail.com
    Milan


Log in to reply