Monitor and Evaluate

This section guides both the campaign management team (CMT) and the oversight team as they monitor and evaluate the campaign. Consistent monitoring and evaluation is essential for informed decision making and to achieve strategic outcomes. In partnership, the oversight team and the CMT should monitor and evaluate the ongoing public health emergency (PHE) to align campaign performance and activities and promote effective campaign operations using the monitoring and evaluation framework (Establish the Monitoring and Evaluation Framework).
Tools and Resources Link
imagePrepare Respond Close Refine




Monitor and Evaluate

The CMT can manage tactical and operational risk, elevate decision-making to the oversight team to approve mitigation plans that would impact campaign strategic goals or require significant strategic changes, and keep the oversight team apprised of general progress throughout the campaign. The alignment of these components supports risk-based management and executive decisions and allows for rapid implementation of continuous improvements. Monitoring and evaluation activities and outputs are performed simultaneously while the campaign is operational. This facilitates the oversight team’s ability to make data-driven decisions and to task the CMT with campaign refinements.​

Three Levels of Campaign Management and Evaluation

Program Monitoring: Assess whether the campaign is achieving milestones for implementation and operations.

Strategic Monitoring: Assess whether the campaign is on track to achieve desired outcomes by using strategic measures and metrics.

Operational Monitoring: Assess whether campaign operations are efficient and effective to achieve operational outcomes.

Monitor and Evaluate the Emergency

The full impact of a PHE on a community can develop slowly. It may be difficult for federal, state, and/or local officials to observe, assess, and understand the PHE’s relationships to other aspects of people’s lives. It is sometimes only after an emergency ends that its effects become fully visible (Knowles, 2014). ​

It is essential to monitor and evaluate the emergency and the impact on the community throughout the campaign to determine if the overall response strategy or operations need refinement to address changes to the state of the PHE. ​

Use Predictive Modeling to Monitor the Emergency

Predictive modeling is a tool that can be used to develop situational awareness of the PHE in its current state and begin to make campaign decisions based on its potential trajectory.

  • Establish a modeling structure for the current and projected impacts of the PHE.
    • Modeling may inform campaign components such as community activation and partner opportunities, media options and message development, and how actions taken by the campaign might influence the PHE.
  • Identify data analysts with modeling expertise at the outset of the planning process to provide the campaign with findings to support campaign decisions.
  • Regularly assess both the actual progression and modeled forecasts of the emergency’s effect on populations.

Key Point

Models provide a way to examine how changing situations, such as supplying a vaccine in a pandemic, can alter a result (e.g., 100,000 deaths averted), given specific assumptions. By studying current patterns and trends, predictions regarding additional information, like the required administration timeframe and targeted age groups, create the basis of a well- informed plan and anticipated results.


Key Decisions and Activities to to Monitor the Emergency and Impact

  • Establish an evidence-based modeling program that leverages both actual values and predictive estimates to understand the extent of the emergency and how it changes over time.
    • Incorporate data on the number of cases, fatalities, people requiring treatment or hospitalization, population(s) to target (e.g., for blood or tissue donations, vaccination, or other intervention), and results from the campaign actions (e.g., number of signups to donate or get vaccinated, number of website visits).
    • Model and simulate to help predict where resources should be targeted.
    • Investigate the effectiveness of the campaign to increase awareness, intent, and action.
    • Visualize the key metrics in a dashboard to make the data easier to understand and more widely available to stakeholders.
  • Weigh multiple (and sometimes conflicting) factors, including resources to make campaign decisions.
    • Depending on the complexity and scope, this may be accomplished with a simple multi-criteria analysis framework or a more advanced optimization.
    • Optimization may be required, for example, to identify the best locations to supplement capacity or where to invest in the campaign.


Close and Transition
Case Example

The Federal COVID-19 Response developed and utilized an evidence- based modeling strategy to track COVID-19 trends based on assumptions made throughout the COVID-19 Convalescent Plasma donor activation campaign. Although the campaign achieved successful outcomes informed by predictive modeling, analyses could have been aided by deliberately tying modeling efforts to decision points earlier in the campaign. This activity would ensure that modeling activities are optimally aligned to the campaign’s data- driven decision-making needs. If predictive modeling is required in future campaigns, teams should encourage early engagement between the modeling team and other essential project team members for outcome traceability.



  • Develop a modeling decision framework.
    • Define the scope, including regions (e.g., state, metropolitan statistical area [MSA], county) and communities of interest.
    • Define decisions to be made during the campaign.
    • Identify key metrics to support decisions.
    • Identify a methodology to evaluate decisions.
  • Determine whether the campaign team has sufficient expertise and access to existing modeling resources and capabilities within the organization (e.g., experience in simulation modeling, data analytics, and visualization as well as coding experience in modeling and dashboard tools).
    • Determine any gaps in expertise and supplement staffing with external resources that can be procured early in campaign planning to inform decisions.
  • Conduct, at minimum, weekly analysis to see changes in models and predictions of how things may change with the emergency.
  • Provide weekly updates that include predictive modeling results, essential information, and insights to the CMT and the oversight team.

Monitor the Evolving Science and Public Sentiment

It is important to monitor the online landscape by using social listening tactics and conducting ongoing sentiment analyses of narratives shared across media channels. These activities can help inform communications and address public concerns before they are amplified and before the knowledge gap is filled by misinformation and/or disinformation. Monitoring evolving science and public sentiment may also help identify credible influencers or opinion shapers who may be approached to serve as trusted messengers for the campaign.

Key Decisions and Activities to Monitor the Science and Public Sentiment

  • Develop a regular schedule to monitor social media channels for relevant topics and how they are manipulated.
    • The schedule will provide situational awareness of current topics and themes in public discourse, the amplification of misinformation and/or disinformation, and public sentiment on the PHE.
  • Monitor published and pre-published medical journals released with the most up-to- date information on the PHE, including treatment and prevention.
  • Analyze and report social media sentiment for valuable information to review and revise campaign messaging and strategy
  • Studying shifts and intensity in online conversations may convey the public as a skewed or incomplete understanding about the underlying health issues or how to interpret and apply scientific and clinical guidance. Review how populations translate information and how they discuss it (e.g., assigning positive or negative attributes to currently available information; expressing changed emotional reactions to the PHE, the government, or the clinical community).
  • Establish a regular analysis and reporting schedule (at minimum weekly) and integrate the schedule into the overall data reporting structure.
  • Share findings with the clinical and scientific advisory body.
    • Clinicians and scientists can help determine if a mitigation intervention is warranted.
    • Enhanced public education and countermeasures can be devised to promote a scientifically accurate public discourse.
  • Share findings with the communication team.
    • Potential shifts in the messaging strategy can be identified and implemented.
  • Identify information that may be a potential risk to the overall response strategy.
  • Identify potential influencers and opinion shapers that could be partners for community engagement or message amplification.


Key Point

Sentimental analysis identifies opinions, attitudes, and feelings expressed in teh text of subjective social listening data. It can help determine whether the sriter’s attitude towards. aparticular topic is positive, negative, or neutral. Monitor social media sentiment is critical to gauging an audience and making timely adjustments to the campaign messaging to optimize positive public opinion (Bisio et al., 2017)


Monitor and Evaluate the Campaign Strategy and Operations

Monitoring and evaluating the campaign enables the team to measure activity effectiveness and determine if the campaign is on track to achieve its desired outcomes. From the start, it is important that desired outcomes, performance indicators, and metrics are all defined, even if proxy metrics must be used due to data access or measurement issues.

By the campaign’s launch, data sources, data use agreements, and a data collection plan should be developed and ready to implement. These metrics and data must be organized, monitored, and evaluated to determine whether the campaign is on track and whether operations are effective. See Example Outcomes and Metrics for example outcomes and metrics used in the FCR campaign.

Monitor Campaign Milestones

For a campaign to be successful, it is necessary to employ structured management principles with key milestones tracked and progress updates delivered to the oversight team. Milestones are the crucial interim actions or steps that must be achieved reach full campaign implementation. These activities monitor the campaign at the operational level.

Example milestones include:

  • Establishment of campaign goals and objectives
  • Asset testing and refinement
  • Campaign launch
  • Initial evaluation, metrics review, and campaign refinement
  • Integration of scientific (clinical and research) findings into campaign decision-making
  • New campaign outreach based on interim learnings
  • Reports on achievements on strategic or operational measures and metrics
  • Reports on campaign management process evaluation (Evaluate the Campaign Management Process)
  • Reports on campaign outcome evaluation (Evaluate the Campaign Strategic Outcomes)

Monitor Campaign Strategic Measures and Metrics

Campaign leaders must craft and implement well-founded measures and review findings systematically to discover whether the campaign is on track to achieve its desired outcomes. The cadence for measuring and reporting measure status will be driven by the oversight team and by data availability. These activities monitor the campaign at the strategic level.


Key Decisions and Activities to Monitor Strategic Measures

  • Follow the monitoring and evaluation framework (Section 2.6).
  • Define process and cadence to measure and review strategic metrics.
  • Measure strategic metrics and obtain measures.
  • Create visualizations and report to oversight team.


Monitor Campaign Operational Measures and Metrics

Separately or in collaboration with the oversight team, the campaign management team (CMT) will develop and use operational measures and metrics regularly to inform whether campaign activities perform efficiently and effectively. These activities monitor the campaign at the operational level.


Key Decisions and Activities to Monitor Operational Measures

  • Follow the monitoring and evaluation framework (Establish the Monitoring and Evaluation Framework).
  • Define process and cadence to measure and review operational metrics.
  • Measure and monitor the following operational areas:
    • Communication hub performance
    • Paid organic posts and social media performance
    • All paid media (e.g., paid radio, TV, out of home digital displays)
    • Asset performance
    • Community engagement metrics
  • Determine if there is a need for refinement(s) to the campaign strategy or operations (Refine Campaign Operations).
  • Follow the data-driven decision framework and log any decisions in the decision tracker.
  • Report progress and discuss options with the oversight team, as necessary.


Evaluate the Campaign Strategic Outcomes

Campaign outcomes are used to develop measures to observe campaign community education and behavioral activation effectiveness (Define Desired Outcomes). An outcome evaluation shows the degree to which the campaign is influencing the target population’s behaviors. It tells whether the program is effective in meeting its outcomes and objectives; whereas a process evaluation indicates how the campaign activities contributed to reaching overall program outcomes. The outcome evaluation is only effective after the campaign has contacted the target population and is able to collect enough data over time.

The following are methods for evaluating strategic outcomes (Monitor Campaign Strategic Measures and Metrics):

  • Modeling and simulation can be used to compare different strategies’ effectiveness to influence optimal results for people and communities.
  • Data analysis, including exploratory and predictive modeling, can be used to investigate the effectiveness of the campaign efforts on the measured outcome.

Evaluate the Campaign Management Process

Campaign management process evaluation measures how well the campaign is working. It includes analyzing metrics for campaign activities and accomplishments by describing measurable outputs (e.g., what was done, how much was done). It also measures whether the campaign is accessible and acceptable to its target population. This type of evaluation provides an early warning for any problems that may occur and allows the CMT to monitor how well the campaign plans and activities are working.

A process evaluation includes data, measures, metrics, and outputs from campaign strategy activities, such as:

  • Website metrics and telemetry
  • Paid media (e.g., radio, print, television), out of home display, and click-based media (e.g., paid search) metrics
  • Social media measures and metrics
  • Social listening metrics
  • Community partner measures and metrics
  • Community activities
  • Audience awareness or knowledge
  • Assets developed and deployed


Close and Transition
Evaluating Community Engagement Activities Via Website Telemetry

Community partners amplified campaign messaging and calls to action in a number of ways. On January 27, 2021, the Federal COVID-19 Response (FCR) team partners in the New Orleans, Louisiana MSA sent text messages to 35,000 subscribers to the NOLAReady emergency alert system with links to the The Fight Is In Us website and COVID-19 Convalescent Plasma donation messaging.

Daily (and at times hourly) monitoring of website telemetry revealed a 10x spike in website hits from New Orleans around the time the text alerts were sent. The percentage of website hits that converted into plasma donation center selections was 54% higher than the other 24 MSAs combined. Website telemetry monitoring allowed FCR to measure community awareness and activation and evaluate tactics that produced quality website traffic.

Armed with this evaluation, FCR was able to demonstrate partner impact with current and new community partners to inspire similar action.


Tools and Resources


Modeling strategic and operational metrics can help understand the effectiveness of the campaign and the trends of the emergency in the community.



Monitor & Evaluate


Close & Transition