Video summary

Pasos para la construcción de indicadores

Main summary

Key takeaways

Educational

Main Ideas and Lessons (Methodology for Constructing Public Policy Indicators)

The video presents a step-by-step process for designing indicators used to monitor and evaluate the performance of public policies/programs. The overall goal is to ensure indicators are rigorous, useful, and interpretable, so they support objective judgments about progress toward policy objectives.

Step-by-Step Instructions / Methodology (Detailed)

1) Determine the Important Aspects to Measure

  • Analyze the objective linked to the indicator.
  • Identify the aspects that show progress or achievement of that objective.
  • Explicitly answer:
    • What will be measured?
    • Who is being measured?
    • What is the unit of measurement?

Example logic (violence against women objective):

  • What: incidence of acts of violence
  • Who: women
  • Unit: quantity of acts of violence

Outcome: The indicator reflects progress in a way that produces useful and relevant data for monitoring and evaluation.

2) Assign a Clear Name to the Indicator

After identifying key measurement aspects, give the indicator a name that reflects the measurement objective. The name should include:

  • What
  • Who
  • Where
  • When

Naming guidance:

  • An imprecise name (e.g., “Requests processed”) is ambiguous because it lacks context (who/where/when).
  • A detailed name (e.g., “Percentage of requests processed to participate in the call for proposals during the period”) improves interpretation and usefulness.

3) Calculate the Indicator (Define the Formula)

Create a precise mathematical expression aligned with the intended measurement type.

Common formula types mentioned:

  • Percentage

    • Defined as: (quotient of two variables with the same unit) for the same period × 100
    • Example structure:
      • Numerator: applications answered
      • Denominator: applications received
      • Multiply by 100 → percentage of processed applications relative to total received.

  • Rate of variation / rate of change

    • Expresses relative change over time using two observations of the same variable in different periods.
    • Example structure (2022 → 2023):
      • Divide payments in 2023 (period t) by payments in 2022 (period t-k)
      • Subtract 1 and multiply by 100 → change expressed as a percentage.

  • Average

    • Used when combining variables with different units for the same time period.
    • Example structure:
      • Average investment per beneficiary school = (investment amount in period t) ÷ (number of beneficiary schools in period t)

4) Establish the Means of Verification (Data Sources)

Define where information will come from to measure the indicator and verify objective achievement.

Means of verification can include:

  • official documents
  • internal program reports
  • databases
  • etc.

Guidance for documenting the means of verification:

  • Provide the full document name (e.g., “Considerations for the Budgetary Process”)
  • Provide the publishing/generating institution/area (e.g., the Institute for the Evaluation of Public Policies of the State of Durango)
  • Provide generation frequency (e.g., yearly)
  • Provide an electronic link to the source (example shown: www.inab.org)

If the information is not generated by the responsible institution:

  • Assess the cost-benefit of generating it.
  • If benefit > cost → use that data source.
  • If cost > benefit → rethink the indicator or use a proxy indicator.

Proxy indicator rule (explained):

  • Use a variable different from the one of primary interest but with the most direct possible relationship to the phenomenon.
  • Example: mother’s education level as a proxy for family socioeconomic level.

5) Technically Validate the Indicator (CREME Criteria)

After constructing indicators, evaluate minimum technical characteristics using the CREME criteria:

  • Clarity

    • No doubt about what the indicator measures.
    • Technical/ambiguous terms must not be interpreted differently by different people.
    • Shared understanding with program operators is required.

  • Relevance

    • Provides information on at least one relevant factor of the associated objective.
    • Focuses on an important and practical aspect.

  • Economy

    • The benefit of generating the necessary information outweighs the economic/human cost.
    • Also noted: an indicator that is not relevant or not adequate cannot be considered economical, regardless of cost.

  • Monitorability

    • Information in the means of verification must be accurate and unambiguous.
    • Calculation must be replicable by anyone:
      • method must be clear
      • required data must be clearly identified in the means of verification

  • Adequacy

    • Provides sufficient information to issue a final, homogeneous judgment about program performance.
    • Information must be relevant and appropriate to describe achievements in a given period.

  • Marginal Contribution

    • Applies only when the objective has two or more indicators.
    • An indicator has marginal contribution if it adds information not covered by other indicators and supports a more complete monitoring of performance.

If an indicator fails criteria:

  • It loses its purpose because it cannot support a valid judgment on performance.
  • Objective evaluation helps identify opportunities to improve indicator design.

6) Establish Goals for the Indicator (Targets)

Setting goals is fundamental for institutional decision-making.

Goals represent the level to be reached by a program/intervention. Goals:

  • establish maximum levels of achievement
  • communicate expected organizational performance
  • focus improvement efforts

Goal planning requires clarity on:

  • the starting point (baseline)
  • what will be achieved
  • available resources

Goal characteristics:

  • must be realistic and challenging
  • achievable with available resources
  • avoid the error of setting goals that are too easy

What is needed to establish goals?

  1. Baseline

    • Starting value of the indicator used as the reference point.
    • Obtained by substituting the initial-period data into the indicator’s calculation method.
    • If baseline information is not available internally:
      • obtain it from external sources.

  2. Desired level of improvement → Goal/Target

    • Baseline + desired improvement determines the goal.
    • Baseline enables performance judgment over time.

7) Construct the Indicator’s Data Sheet

The final step is to fill out a form (data sheet) including:

  • indicator description
  • calculation method
  • variables used, with definitions
  • measurement frequency
  • means of verification
  • indicator dimension
  • baseline and target

Speakers / Sources Featured

  • No individual speaker is explicitly identified in the subtitles.
  • Source/Institution referenced (example in means of verification):
    • Institute for the Evaluation of Public Policies of the State of Durango
  • Example document referenced (means of verification):
    • “Considerations for the Budgetary Process”
  • Example link referenced (means of verification):
    • www.inab.org
  • Music track is referenced as background (“[Music]”), but no specific music title or composer is given.

Original video