Pub. 6 2017 Issue 3

19 f a l l | 2017 Back-testing (reverse-engineering the output of your model using a separate process/software) is the most frequently used method of performing model validation. To do this, the parameters, or “rules,” set up in the institution’s BSA/ AML model to determine which transactional activity results in output, or “hits,” are extracted from the model. A sample period of transactional data is also extracted from the source, for example, a core application system download. The party performing the validation then uses separate software with criteria set to mimic the rules of the model. The separate software is used to process the transactional data to determine what “hits” result. The validator then compares the results to the model results and investigates discrepancies. The results are then utilized to fine-tune and recalibrate the model. One and Done? Validation of the model is not a one-time process. The model should be validated on a periodic basis to proactively respond to technological advancements and to incorporate new prod- uct offerings by the company. OCC 2011-12 specifically states that the model should be validated at least annually as well as when material changes occur to the model. Keep in mind, validation does not have to occur on the back-end. If a new product is being offered, sample transactions can be validated to ensure correct outputs from the model. Benefits of Validation Validating your BSA/AML model has more benefits than just crossing an item off your regulatory compliance to-do list. These benefits can include: • Increasing your confidence in the model’s outputs – elimi - nating garbage, • Identifying your model’s limitations to determine neces- sary mitigating controls, • Allowing for a proactive approach to technological ad- vancements and new product offerings, and • Reduction in risk of misdetection of illicit activity. While full compliance risk cannot be eliminated, validation of your BSA/AML model can help reduce your risk and identify model limitations and loopholes. However, technology still cannot replace human instincts. By promoting an institution- wide culture of following your gut and speaking up, even when the model is not detecting suspicious activity, will also help to reduce your institution’s compliance risk. if Your Model is Producing Garbage or Gold