Roll Forward Testing 1401



  • Pardon me if the query is already repeated.
    Tests of Daily and Weekly controls that are performed by that end of Quarter 2 and deemed Evaluated Effective, do we need to roll forward testing of these controls with reduced sample size in Quarter 4?
    I have been performing Roll Forward Testing in the past. But, at the latest work place they have ignored this suggestion of Roll Forward Testing in Quarter 4. Before, I stress this further with the management, I thought of running across fellow professionals at this forum.
    Thanking you in anticipation.



  • While the independent auditors are required to perform roll-forward testing through year end, management is not. However, I agree with you that roll-forward testing is a good idea, since it would be better to know if a Key Control is no longer effective (or, in place) for whatever reason.
    For our on-going clients, we’ve been testing 1/4 of the required sample each quarter, so we’re not in a position to need to roll-forward by the fourth quarter. Prior to testing each quarter, we do a quick ‘walk through’ just to make sure that we can note any significant changes in the accounting processes (which have been primarily limited to changes in accounting staff or managers that are performing/documenting the controls). If we had already tested a sufficient sample without exception, the walk through would probably suffice for any roll-forward testing, even though the auditors perform their own walk throughs.
    Hope this helps.
    John



  • John
    I specifically recall, while working (as an independent contractor) with Ernst and Young, two years back, they did not like the methodology of 25/4 sample size for testing daily controls each quarter.
    A friend of mine (Ex Arthur Andersen) on that project, also concurred with the EY partner stating that it is a good idea to test controls via original sample sizes till the end of 8th month, followed by a roll forward testing in December on a reduced sample size or otherwise perform all controls after the 8th month.
    Maybe PWC, KPMG and Deloittes agree to the 25/4 sample size. Who are your Auditors, if its appropriate to ask?



  • Folks,
    I ran this query by our Big 4 auditors also and they responded (verbatim given below) immediately after my previous feed.
    'As the SOX evaluation is as of December 31, the Company must perform procedures to update its understanding of controls tested earlier in the year to ensure they are still operating effectively at YE. In this past year, so many of the controls were tested near or at year-end that little update was needed, but if you do plan to test early in the year, I would recommend some limited testing in Q4 to update your evaluation. The earlier in the year you perform your testing the more extensive the year-end update procedures would need to be. That said, I think it is still beneficial to try and perform testing earlier in the year and perform more limited updates at YE. ’



  • Of COURSE it’s appropriate to ask. Anything goes here, as far as I’m concerned.
    We’ve primarily worked with PwC (DC and Baltimore offices), BDO (NYC), and a few smaller firms (NYC and So. FL).
    Until now, no one has mentioned ‘not’ spreading out the (internal/management) testing through the year (evenly), although we usually test at the upper ranges of the sample size. I’m not sure why it would matter too much if a sample was selected from an annual population. Especially since the samples that we select have been/are purely judgment (non-statistical).
    I’m curious, when you were working with E-and-Y, were you contracting on the management side? or, the audit side?
    Not that it probably makes a difference.
    But, I would think that it’s up to management to determine the sufficiency of the testing performed (theoretically), while testing through the 8th month and then rolling forward would make sense (from an audit-efficiency standpoint) for an independent auditor.



  • A year of my SOX efforts was spent as an Independent Contractor for the Management where Ernst and Young were outsourced the Internal Audit Piece. We were helping Ernst and Young as a cheaper alternative using their methodology.



  • The PCAOB Guidance over roll-forward procedures is addressed in detail. It may be found in the PCAOB document on Page 18, Audit Standard No. 2 - Internal Control, Question #51.
    Q51. If the auditor performs procedures to test the effectiveness of controls as of an interim date, how should the auditor determine the nature and extent of roll-forward procedures that are necessary to support the auditor’s opinion as of year-end?
    A51.
    pcaobus.org/Standards/Staff_Questions-and-Answers/2005/05-16.pdf
    As always, it may be possible to discuss audit scope, test plans, nature and extent of testing, with the external auditor to ensure a mutual understanding.
    However, it is not necessary to have them ‘bless’ everything that you do. In fact, some audit firms are reluctant to provide such feedback as it may impede the auditor’s independence.
    Hope this helps,
    Milan



  • However, it is not necessary to have them ‘bless’ everything that you do. In fact, some audit firms are reluctant to provide such feedback as it may impede the auditor’s independence.
    Milan
    It really won’t impair their independence as much as gives the appearance of making management decisions for the company that they are auditing. The Company being audited must have qualified staff who can make these decisions on their own. It is a fine line to walk, but most audit firms will provide feedback once management provides them with their test approach.



  • Folks,
    I ran this query by our Big 4 auditors also and they responded (verbatim given below) immediately after my previous feed.
    'As the SOX evaluation is as of December 31, the Company must perform procedures to update its understanding of controls tested earlier in the year to ensure they are still operating effectively at YE. In this past year, so many of the controls were tested near or at year-end that little update was needed, but if you do plan to test early in the year, I would recommend some limited testing in Q4 to update your evaluation. The earlier in the year you perform your testing the more extensive the year-end update procedures would need to be. That said, I think it is still beneficial to try and perform testing earlier in the year and perform more limited updates at YE. ’

    I specifically recall, while working (as an independent contractor) with Ernst and Young, two years back, they did not like the methodology of 25/4 sample size for testing daily controls each quarter.
    A friend of mine (Ex Arthur Andersen) on that project, also concurred with the EY partner stating that it is a good idea to test controls via original sample sizes till the end of 8th month, followed by a roll forward testing in December on a reduced sample size or otherwise perform all controls after the 8th month.
    Maybe PWC, KPMG and Deloittes agree to the 25/4 sample size. Who are your Auditors, if its appropriate to ask?
    Designing and implementing internal accounting and risk management controls impairs the accountant’s independence because it places the accountant in the role of management. Conversely, obtaining an understanding of, assessing effectiveness of, and recommending improvements to the internal accounting and risk management controls is fundamental to the audit process and does not impair the accountant’s independence.
    According to SEC staff, example of functions that if, performed by the auditors, would likely to run afoul of the independence rules. These include:

    1. Outsourcing a major part of the project;
    2. relying on the auditors to choose sample sizes;
    3. allowing the auditors to select which tests are to be performed; and
    4. using auditor’s software that provides conclusions as to the effectiveness of the controls.
      Item #2 in the list ABOVE clearly states that determining sample sizes is a prohibited act and would impair independence. This seems to be one of the reasons why firms like PwC developed Sampling Size Guidelines as found in their Sarbanes-Oxley Section 404 - Practical Guidance for Management. They can simply refer their audit clients to consider their guidance and steer clear of potential independence issues.
      The SEC has indicated that the following ARE allowed:
    5. Providing software templates to help document controls or perform statistical sampling;
    6. noting areas where management may want to improve internal controls;
    7. making suggestions to improve tests of controls.
      Personally, I like #1 above. However, I’ve yet to meet an external auditor who wiill share software templates to assist in documenting controls, or for sampling purposes.
      Hope this further clarifies.
      Milan


  • Milan
    Staff Answer 51 on AS2 along with Paragraphs 88 to 107 and examples B-1 to B-4 under Appendix B are merely instructions to Independent Auditors. In my opinion, a walkthrough of a daily control, as mentioned in example B-3 in the last month of the fiscal is not sufficient.
    The earlier in the year you perform your testing the more extensive the year-end update procedures would need to be.
    What I meant by RollForward Testing is something more than a walkthrough.
    Staff Answer 51 on AS2 along with Paragraphs 88 to 107 and examples B-1 to B-4 under Appendix B did not provide an answer to my requirements.
    Therefore, I had to run by you folks.



  • Further to my feed
    I would never expect to get the sample size from Independent Auditors. But, it is a good idea to share you sample size guidelines for them to be in the loop as to what direction the company is going as far as 302 and 404 certifications are concerned.
    I still believe that it is a good idea to share you SOX governance and compliance strategies with your Independent Auditors as long as you do not tread the forbidden territory.



  • Hi Chhaaya,
    Ok; I was a little miffed when I read that you had asked your auditors for guidance on the selection of sample sizes and timing of testwork in connection to controls testing.
    It seems that some in management are overly cautious to make basic decisions and conduct auditing procedures related to SOX compliance initiatives without first running their testing plans by the external auditor.
    Confirming the following,
    'According to SEC staff, example of functions that if, performed by the auditors, would likely to run afoul of the independence rules. These include:

    1. Outsourcing a major part of the project;
    2. relying on the auditors to choose sample sizes;
    3. allowing the auditors to select which tests are to be performed; and
    4. using auditor’s software that provides conclusions as to the effectiveness of the controls.’
      As for requesting additional guidance beyond that found in the 161-page document, AS-2 developed by the PCAOB, it seems natural to want to know the thoughts of others participating in this Forum for their views and experiences.
      Inasmuch as possible, I think it appropriate to maintain the same standards as those prescribed by the PCAOB to the independent auditor in AS-2. The document is more than comprehensive and would establish an acceptable standard of audit performance if presenting your audit results to the external auditor.
      Regards,
      Milan


  • maybe not so clever question…
    how roll forward testikg works. how do perform this?



  • maybe not so clever question…
    how roll forward testikg works. how do perform this?
    Your internal control effectiveness assertion is ‘as of’ year end. The concept of roll-forward testing is based on the assumption that if you perform testing early in the year, you need to perform some additional testing (roll-forward) near the end of the year to provide assurance that the controls tested earlier are still effective.
    Generally, roll forward testing is based on a much smaller sample than your earlier general testing. We have taken the position that we are going to perform inquiry testing on the low risk, more routine controls (daily, weekly transactional controls) and a sample of 1 for the monthly and quarterly controls (generally higher risk).



  • Likewise we are undertaking a ‘light touch’ using inquiry and observation on all low and medium risk key controls that were last tested before the end of Q3.


Log in to reply