operating-effectiveness-procedures

Write procedures for determining whether an implemented control operated as designed, consistently over the relevant period or at the required time, through the appropriate people or systems. Return planned actions and evaluation criteria for the user's testing approach; executing tests, recording results, and reaching effectiveness conclusions belong to the user performing the work.

Writing the procedures

Use numbered steps with one coherent test objective per step. Identify the evidence to examine, the action to perform, and the observable criterion for evaluating the result. Build the criterion into the step's wording, such as a required deadline, threshold, or approval the evidence must show. Keep connected actions together when they answer the same question; separate attributes that need independent results. State common population, period, and selection details once for the procedure set. Specify the actual inquiry, observation, inspection, reperformance, calculation, or other examination needed.

Base organization-specific details and criteria on supplied or retained sources. Preserve source references, existing bracketed gaps, and distinctions between intended design and reported or observed practice. Identify the sources used when drafting from scratch. Mark unsupported evidence, criteria, periods, or selection details as descriptive bracketed gaps and complete the supported steps around them.

Control performance

Connect the tests to the control's purpose and relevant risk. Cover the attributes necessary to evaluate its operation: the performing role or system, action and scope, required frequency or trigger, timeliness, information used, precision, and exception follow-up. Explain any material attribute left outside the agreed testing approach.

Identify evidence that the substantive control activity occurred, including the required comparison, decision, investigation, or resolution. A signature or completion record establishes only what it evidences; where the control requires a substantive review, specify how to examine its performance and follow-up.

Define a deviation against each applicable control requirement. Distinguish exceptions the control was designed to identify from failures to perform the control or handle those exceptions. Make evidence limitations visible, and treat a selected item that cannot be evidenced as a deviation unless the methodology provides otherwise. Tolerable deviation rates and rules for concluding effectiveness must come from the methodology or user.

Population, period, and selection

Identify the population and its source, relevant period, and sampling unit. The population must capture the control occurrences or items relevant to the test objective. Include procedures to establish its completeness and suitability before relying on the selection.

For sampled work, state the selection method, sample size, and their basis from the user or documented methodology, including whether the results may be projected to the population. Apply frequency-based sample sizes only when the governing methodology supports them. For full-population testing, state the coverage and source. Leave unsupported selection decisions visible for the user; selecting the engagement's testing strategy is outside this writing task.

Address coverage across the relevant times, locations, systems, or other material variations in how the control operates. Where its design or implementation changed during the period, identify the versions and periods to be tested.

Evidence and dependencies

Specify appropriate completeness, accuracy, and relevance checks for information relied upon by the control or the test, including manually maintained records, system reports, and extracts. Identify the information, applicable version or parameters, and the checks needed, or reference supported work that establishes its reliability.

Inquiry alone does not establish operating effectiveness. Observation provides evidence for the activity and time observed; specify corroboration or additional coverage needed for the intended reliance. Rely on walkthrough evidence only to the extent its procedures and coverage meet the test objective.

For automated controls, address the configured rule and relevant conditions or paths, together with the supporting general IT controls or other dependencies on which reliance rests. Identify the testing or workpaper supporting that reliance and expose missing coverage.

Fit the nature and extent of testing to the control and intended reliance. A single occurrence, including a baseline or single-instance test of an automated control, needs a supported basis in the population or testing approach, together with relevant evidence about changes during the period.

Review and return

Check that every control performance attribute is either tested by a step or explained as outside the agreed approach, and that each step names its evidence and an observable criterion supporting the intended period and extent of testing. The wording is complete when an experienced auditor with no prior connection to the engagement could perform each step and determine whether the requirement was met without inventing a missing criterion. Flag conflicts with the supplied control description or testing basis without silently changing either. Where the control is unidentified, unimplemented, or has an unresolved design concern, flag the effect on the proposed testing.

Return the procedures with material gaps and evidence limitations. When reviewing existing wording, explain substantive weaknesses and propose revisions only where needed.