DEV/QA/Production Environment deployment 2436



  • Currently there is discussion within my organization about testing and dealing with system failures. Right now we are dealing with a vendor that does not give a Dev environment to work on before implementation onto the live production environment. They seem to be of the belief that testing on a live system is ok. Crazy i know, what im looking for is documentation or a link to a best practice site that i can grab information on controls that need to be in place for Application deployment.
    I know this is a retarded request, but im just not sure how to explain this in more detail to the vendor other than screaming in there face ‘DEV and even a QA environment is a must…’ Any help would be greatly appreciated.
    Thanks for the help
    JL



  • I would just go down the screaming route :lol:
    In all seriousness though there are several discussions on here already that may help:



  • Yes, I agree with Denis 🙂 … If your company must adhere to SOX standards, the lack of a DEV or QA testing environment may represent a material financial risk , esp. if this is a Financial application.
    For example, a developer can insert their own routines in to ‘add to the bit bucket’. Below is one of the most extreme examples Denis shared with us, about allowing someone too much authority in an unchecked manner:
    http://www.sarbanes-oxley-forum.com/modules.php?name=Forums-and-file=viewtopic-and-t=2289
    Fundamentally developers should be isolated from Production data and programs as much as possible (e.g., developers should only access production on an emergency basis with logging, audit trails, special Fire ID accounts, etc.).
    I’d continue trying to work with the vendor for an adequate solution.



  • Along the lines of this discussion about DEV, TEST and PROD, has anyone devised an effective way to test that all of the components in PROD are connected and working?
    I’m sure the code itself is working because it was unit testefd in DEV, then QA and UAT in TEST. Now that it is being moved into production, how can I verify that all of the pieces work together? Is DNS working? Is the database engine working?
    In the old days, I would have run a few test entries to make sure it worked end to end. Now our auditors tell us that there ca be no test data in the PROD environment.
    Does anyone have experience with this?



  • Hi - Yes, PROD is difficult to test and breaking in security controls is usually one of the things that will go ‘bump’ in the night or day as you bring the new system up.
    The only ideas I can think of:
    – For brand new apps only, sometimes it’s an acceptable practice to run in PROD in a simulated mode a week or so in advance to test out any last minute issues. Then after the simulated PROD test is over, scratch all DBs and folders and restage everything with any adjustments. Document this well in the change management process if this approach is used.
    – Release on a weekend in PROD and have users pilot test and support team work through early issues.
    – Create a mirror PROD environment to test in, but that’s tough to match and keep updated in most instances



  • How do we verify that the existing PROD is still acting as expected? How can I prove that all of the components are working at any given time? Are there any options for this function besides trace reporting of previous transactions?



  • I guess one way to ensure consistency between Production and Test environment( where users did the UAT, and an environment considered to be a replica of Production) is to compare the changes that were transported( i may sound bit SAPish, but so used to working with SAP now) to Test Environment with the ones that were further promoted to PRD
    When a user tests a required change( config or additional functionality) he tests such change along with so many other changes that exist in the Test environment.
    By confirming that all changes that were made to the Test environment are transported to the production environment, we can get some ‘COMFORT’ on the appropriateness of the PRD environment’s functionality.
    Again, the above only gives some Comfort and not absolute assurance. After all organizations need some flexibility and cannot work in a prison like rigidly controlled environment( or atleast they claim so)



  • How do we verify that the existing PROD is still acting as expected?
    NC shares some good ideas on mirroring PROD in a UAT approach. Still ‘PROD is PROD’ and no amount of simulated testing will fully address all concerns. For example, IT security controls will certainly differ along with other unique factors that truly can’t be checked out until AFTER the change is implemented there.
    How can I prove that all of the components are working at any given time?
    Personally, I like the combination of using NC’s approach on building an environment close to PROD – and then releasing a few hours earlier than when users arrive (if it’s a big change over) to work through early issues. For example, if folks typically arrive at 8 am on Monday (release earlier on Saturday or at 6am on Monday if no other window is available).
    Have early testers (power users) shake down the system and usually after a few minor adjustments things typically work well (or you can make a ‘no go’ decision and back off as needed).


Log in to reply