Program Research, Planning, and Evaluations. The CDR Team believes program efficacy begins through analysis of current performance, planning to establish goals, and establishing a system of constant evaluation through measured metrics. CDR team members used that approach in the past to address problems facing the State of Florida, with demonstrated success. For example, the Florida Department of Health under the direction of several CDR team members, transformed a billion dollar plus annual health service delivery system for medically complex children from a fee-for-service, non-value based system of care with open ended expenditures that were concerning to the state budget authorities, to a risk/value-based payment system introducing predictable appropriations for the Legislature while substantially improving the service delivery and benefit options for families and medically complex children below the age of 21. In addition to meeting fiscal and service delivery transformation objectives established by the agency and its stakeholders, the new service delivery system also mitigated risks related to ongoing litigation, fraud and abuse by network providers, failures to meet Healthcare Effectiveness and Data Information Set (HEDIS) and quality outcome benchmarks and numerous other challenges. This was the largest competitive procurement (5-year/$7B) in the Florida Department of Health’s recent history and was implemented successfully following two years of planning, buy-in with public and private stakeholders and policymakers, and success in the administrative court following the CDR team member’s appointment as the agency expert/representative. Similarly, CDR team members were involved in the restructuring of Florida’s trauma system. The effort to do so took more than a year to accomplish due to rulemaking and litigation, and involved internal and contracted external program evaluation when necessary. The resulting rulemaking rubric created a continuous, annual planning and evaluation of Florida’s trauma system to determine need and placement of future trauma centers. The trauma rules were ultimately deemed valid through administrative litigation in which a CDR team member served as agency representative. CDR has built a team that understands program research, planning, and constant evaluation are essential for operational success. PROVISION OF STUDIES, ANALYSES, SCENARIOS, AND REPORTS RELATING TO A CUSTOMER’S MISSION-ORIENTED BUSINESS PROGRAMS OR INITIATIVES The CDR Team includes members with signifi...
Program Research, Planning, and Evaluations. FE Proposed Team Members • Xxx Xxxxx • Xxxxx Xxxxx, PMP • Xxxxxx Xxxx, ENP • Xxx XxXxxxxxx, ENP • Xxxxxx XxXxxx, PMP, ENP • Xxxx Xxxxxx • Xxxxx Xxxxxxxx • Xxxx Xxxxxx FE has worked with many jurisdictions and agencies to conduct strategic Project Initiation Meeting Agenda • Introductions • Clarify roles • Review project objectives and expectations • Review key issues • Review key milestones and schedule • Review/clarify deliverables • Plan interviews and identify interview participants and engagement strategies • Review status reporting methodologies • Determine progress review meeting schedule • Resolve immediate issues • Build relationships planning, designing viable frameworks and architecture for communications solutions. Our involvement includes understanding our client’s requirements and strategic technology vision as well as documenting their current and future business processes. Given the rapid pace of technology change and public expectations, multiple approaches or options may exist for achieving State of Florida agency initiatives. FE will work with the State to drive the selection of specific strategic alternatives/approaches for the identified program. Utilizing FE’s strategic planning process, our technical consultants will evaluate an agency’s current conditions and capabilities and desired outcome related to its programs or networks. Based on program and industry research, our consultants will identify actionable activities that are meaningful, timely, and achievable (both short-term and long-term). A plan to improve an agency’s operations or communications program must address current options as well as next generation technologies or alternatives to define a direction that best meets the needs of stakeholders, the public, and the State agency’s objectives. FE consultants develop Strategic Master Plans that maximize the value of the agency’s investments in systems and equipment, leverage best-in-class practices in operations, and provide a robust and resilient infrastructure that will support the State’s current demands and evolving needs. Through our program research, planning, and evaluations, FE’s consultants guide the customer agencies to achieve the following: • A consensus understanding of the current state of technologies or existing processes, citizen and stakeholder demand for services, and existing capacity and capabilities • Identification of critical challenges and opportunities to contextualize operational and technical goals...
Program Research, Planning, and Evaluations. HMA’s approach to conducting program research, planning, and evaluation begins with the identification of a team of SMEs with content specific and program design, implementation, and evaluation expertise. The HMA project coordinator and team will collaborate with the client’s executive sponsor to understand the program goals, identify measurable objectives, and develop and implement the specified tasks to facilitate process and outcome evaluations. PROGRAM RESEARCH AND DESIGN Successful, effective programs are founded on a clear understanding of program participants’ need and an understanding of best practices in addressing those needs. To assess the need for any program, HMA SMEs use a mixed-methods approach that can include literature reviews of professional peer reviewed publications, governmental material, and grey literature; reviews of existing community needs assessments conducted by local agencies or businesses; and interviews with key stakeholders including program implementers and participants and agency executives. To develop a robust understanding of community need, HMA uses a proprietary data tool to conduct analyses of publicly available data from sources such as the U.S. Census Bureau and extrapolate data to drill down on small units of geography. We can also conduct analyses of other non-public data sources supplied by the client using statistical software tools such as SEQUEL, SAS, or SPSS. Additionally, our SMEs in geospatial mapping bring great value to program planning efforts by using geospatial software such as Arc.GIS or Tableau to identify geographic variations in demographic, health, non-clinical needs, and other indicators that can inform decision-making around identifying key target populations or areas for any specific program implementation. To be resourceful in planning programs, we are careful to research best practices to identify existing programs that have been proven effective that the state can adopt or adapt. This activity can not only reduce program design costs but can also inform implementation and evaluation. HMA approaches program planning holistically, with a clear understanding of the importance of designing programs with goals and measurable objectives that facilitate process and outcome evaluations throughout program implementation. We do this to ensure that course corrections to any program can be made at any phase and at any time. We help our clients develop logic models and objectives with “SMART” criteria (...
Program Research, Planning, and Evaluations. Our first step on receipt of a task order will be to convene an inception meeting with the Customer Project Manager and team. The meeting will lay the groundwork for the remaining tasks and project plan, and to clearly define program objectives. Depending on the scope at hand, the project plan is likely to involve data collection from internal sources, a series of interviews or targeted meeting sessions, and/or external stakeholders. Balmoral will establish a proposed list of targets for data collection. An important point to be established at inception is the definition of successful programmatic outcomes and metrics that have been used or proposed to measure success. Meeting minutes will be prepared and delivered within three business days. The programmatic objectives will dictate the approach that is appropriate for the research, planning or evaluation task. Research tasks may involve generating results from internal investigation or historical records, obtaining raw data from surveys or external sources, using published literature to benchmark various metrics, or actual modeling and statistical analysis. Balmoral’s consultants have established techniques and skillsets to undertake any and all of the above. Example approaches include (1) using surveys of grant recipients and cost-benefit analysis to calculate the financial return to the Florida Office of Energy’s grants office; (2) using statistical tests such as Xxxxx Natural Breaks and Tukey Interquartile Ranges to determine if SWFWMD’s cost-share programs were performing well compared to other agencies, including research into industry standards, appropriate metrics, and criteria for award;
Program Research, Planning, and Evaluations. In collecting data for its evaluations and research projects, PCG provides a comprehensive set of services. This includes sample design, instrument construction, the data collection itself and both qualitative and quantitative analyses. Most of our sample designs involve stratified, clustered samples representative of the universe to be studied, with the analysis weighted appropriately. Qualitative: Focus group instruments are relatively short, allowing participants to have expansive discussions of the issues. Interview instruments are semi-structured, meaning that there are fixed questions, but the answers are expected to be open-ended and the respondents are encouraged to elaborate their answers as much as they desire. We use standard content analysis to analyze interview information, often facilitated by the use of NVivo software. Quantitative: Aside from focus groups, interviews and surveys, PCG also employs case readings and analysis of administrative data sets. The latter permits the entire client population to be examined without sampling, while the former allows collection of information which is not available in coded fields of administrative data systems. The analysis merges the two sources of data for those cases selected into the case reading sample, so that the more detailed information can be correlated to the data from the administrative data. The quantitative analyses focus first on client outcomes (and costs, where requested) and then on identifying those sub-populations which are most and least likely to achieve positive outcomes. In addition, we work to connect the quantifiable process measures related to a service, such as intensity and duration, to the probability of success. Both: Survey instruments combine both quantitative and qualitative information. Most of the questions are fixed answer, asking either for categorical answers, e.g., yes/no or gender, or for ratings using Likert scales. All survey instruments end with open-ended questions, asking respondents to share what they liked most and least about a program or service or simply whether they have anything they would like to add. Most surveys are designed to be completed online, but telephone and paper surveys are also used when warranted. When multiple methods are used, qualitative and quantitative analyses will be integrated so that each method informs the other. Final evaluation reports integrate the qualitative and quantitative data so that each informs the other. The...
Program Research, Planning, and Evaluations. MAXIMUS has supported several states with researching available programs, subsequently planning and implementing the selected program, and providing evaluation services once operational. Examples of our experience are described in Exhibit 2.1.3-1: Program Research, Planning and Evaluation Practices. EXAMPLE ENGAGEMENTS DESCRIPTION Gather and Review Existing Documentation We review documentation to inform us on the current service delivery model, including organizational charts, budgets, operational functions and job descriptions, and policies and procedures. Conduct Ad Hoc Research MAXIMUS staff conduct reviews and analysis, including surveys of practices in other states, to identify best practices and to inform our recommendations for implementation. Federal and state guidelines, regulations, and policies are reviewed for program compliance.
Program Research, Planning, and Evaluations. During the Learn Phase, we take the time to fully understand the customer's research objectives to ensure we select appropriate research/evaluation methods. Ex: For our CMHI evaluation projects, we discussed each participating grantee's program to understand what evaluation techniques would be appropriate for the programs and policies they implemented in their individual community. Studies, Analyses, and Scenarios We continuously validate reports generated from studies, analyses, and scenarios to ensure they accurately depict findings in a meaningful way for stakeholders. Ex: For our analysis of State Child Welfare Fatality Review Teams, we validated reports with project sponsors and facility team members to ensure we presented findings in language and visuals appropriate for the various state stakeholders.
Program Research, Planning, and Evaluations. Whether performing rapid assessments or data-rich portfolio evaluations, ICF applies highly rigorous methodologies to help clients determine optimal approaches to planning and designing a program, determining program effectiveness, and/or identifying ways to improve programs. There are two keys to our success: (1) we factor in research and evaluation early (i.e., at the program planning stage) to ensure efficient gathering of performance data during program implemen- tation and (2) we formulate a quality research question to ensure a successful research, planning, or evaluation assignment. When developing such a question with our clients, we transparently determine the focus, tools, resources, key per- formance indicators (KPIs), assumptions, and data sources required to answer it. Based on this question, we then choose the best methodology to use for our next steps. This can include a literature review (e.g., deploying tools such as Litstream, our proprietary, AI-enabled tool for systemat- ically reviewing massive databases of existing research), randomized controlled testing to isolate key variables to determine cause and effect, a quasi-experimental design to infer causation and correlation across key trends, or a statistical software packages or machine-learning tools for complex datasets. Finally, we present findings tailored to each audience’s needs in clear, actionable language. This can include two-page policy summaries, data visualizations such as infographics and dynamic dashboards, regular reports tailored to clients’ KPIs, or even peer-reviewed journal articles. Our in-house research, planning, and evaluation team includes PhD-level statisticians and seasoned policy experts in health, social programs, economics, education, environment, and energy. Integrated teams explore the intersection of these disciplines (e.g., social determinants of health, environmental effects of economic activity). We offer our clients a richer understanding of the broader contextual environment, confidence that research is sound, and data-driven recommendations and conclusions.
Program Research, Planning, and Evaluations. 🢝 🢝 During the Learn Phase, we take the time to fully understand the customer's research objectives to ensure we select appropriate research/evaluation methods. Ex: For our CMHI evaluation projects, we discussed each participating grantee's program to understand what evaluation techniques would be appropriate for the programs and policies they implemented in their individual community.
Program Research, Planning, and Evaluations. Xxxxxxx may employ a variety of approaches to support program evaluation and overall performance measurement. Specific strategies will be tailored to the needs of the customer, with techniques employed to support performance reporting and development of customized tools for ongoing performance measurement. Tactics which may be used by Xxxxxxx include problem solving methodologies, such as DRIVE (Define, Review, Identify, Verify, and Execute); process mapping/flowcharts; root cause analysis; statistical analysis/data analysis; and data visualization. This is in addition to other qualitative program research tasks that may be executed by the Xxxxxxx Team continent upon the needs of the customer. The following presents the primary tasks that will be executed by Xxxxxxx in support of performance measurement projects.
