Due to the introduction of new laws dictated by our time, most technologies are increasingly becoming more autonomous. This trend also applies to the collection of information for subsequent analysis and transformation of programs.

Monitoring systems (or monitoring and evaluation – M&E) are created for such automation.

What is a Monitoring System? This is a system that refers to all the indicators, tools, and processes that you will use to measure whether the program was implemented in accordance with the plan (monitoring) and whether the desired result was obtained (evaluation).

This article will help you understand what the main stages of developing a monitoring system exist and what they are needed for, using the example of our already implemented project of Pump Monitoring System.

How to create an effective monitoring system?

The fundamental principle of the monitoring system is to allow users to systematically collect data, process it, and make decisions to correct the operation of other components of the system. The effective monitoring system allows us to measure the trends of various indicators based on data collected at specific points. Systematic evaluation and review of activities at a certain point in time after the device operation cycle help us to track progress and provide support in assessing the sustainable interaction of certain experimental mechanisms or programs. We will show you the approximate stages of project monitoring and control below.

Step 1. Define the goals and objectives of the program

The first step to creating an M & E plan is to define the goals and objectives of the program. If the program already has a logical model or a theory of changes, then, most likely, the goals of the program have already been defined. However, if not, then the M&E plan is a great place to start. Define the goals and objectives of the program.

Defining the goals of the program begins with answering three questions:

• What problem is the program trying to solve?
• What steps are being taken to solve this problem?
• How will the program employees know that the program has successfully solved the problem?

The answers to these questions will help determine what is expected from the program, and how the staff will know whether it has worked or not.

It is also necessary to develop interim results and objectives of the program to help track successful steps towards the overall goal of the program.

Step 2: Identify the indicators

After the monitoring system defined the goals and objectives of the program, it is time to identify indicators to track progress towards achieving these goals. Program indicators should be a combination of those that measure the process or what is being done in the program, and those that measure the results.

Process indicators track the progress of the program. They help to answer the question: “Are the activities being carried out in accordance with the plan?” Here are some examples of process indicators:

• Number of tests of the running system
• The amount of potential inherent in the operation and improvement of components
• The number of processes that affect the operation of the program

The results indicators track how successful the program’s activities were in achieving the program’s goals. They help to answer the question: “Did the program activities have an impact?” Here are some examples of performance indicators:

• The number and percentage of typical malfunctions in the system/debug
• The number and frequency of receiving information

These are just a few examples of indicators that can be created to track the success of a program.

Step 3: Define data collection methods and deadlines

After creating monitoring process indicators, it is time to decide on the methods of data collection and how often various data will be recorded for tracking indicators. This should be a conversation between the program components, hardware components, and specialists. These methods will be important for what data collection methods will be used and how the results will be reported.

The source of monitoring data largely depends on what each indicator is trying to measure. The program will probably need several data sources to answer all the programming questions.

Once it is determined how the data will be collected, it is also necessary to decide how often it will be collected. This will depend on the requirements of the software and the hardware, the available resources, and the timing of the intervention.

Step 4: Collecting initial data on indicators

This includes describing and measuring the initial conditions that the results refer to. This is the first measurement of indicators that determines the starting point. At this stage, you should also think about displaying information in reports and/or on special graphical screens, if you haven’t done it yet.

Step 5: Monitoring the results

This is the administrative and institutional task of developing guidelines for data collection, analysis, and presentation; determining who will be responsible for activities; creating quality control processes; setting deadlines and costs; working out roles and responsibilities; and establishing guidelines for transparency and dissemination of information and analysis. It is emphasized that when building an M&E system, it is necessary to clearly solve the problems of ownership, management, maintenance.

Step 6: Evaluation information for decision support 

It focuses on the contribution that evaluation studies and analyses can make to this process to evaluate the results and move towards the final results. Analysis of program theory, evaluation of the possibility of evaluation, evaluation of the process, evaluation of results and impact, as well as generalization of evaluations are some of the discussed strategies that can be used when evaluating a results-oriented M & E system.

Step 7: Analyzing and presenting the results 

It is a crucial step because it determines which results to report to whom, in what format, and at what intervals. At this stage, it is necessary to consider the existing potential for information production and focus on methodologies for collecting and evaluating information, as well as on the preparation of analyzes and reports.

Step 8: Using the results 

We emphasize that the essence of the system is not just to generate information based on the results, but to provide information to the relevant users in a timely manner so that they can take this information into account when making decisions. This stage also examines the role of development partners and civil society in using the information to strengthen accountability, transparency, and resource allocation procedures.

Step 9: Maintaining the M&E system 

This step recognizes the long-term process involved in ensuring the durability and usefulness of the M&E system. Six criteria are considered key to the sustainability of the M & E system: demand, structure, reliable and reliable information, accountability, incentives, and capacity. Each aspect requires constant attention over time to ensure the viability of the system. As noted earlier, building an M&E system does not necessarily have to be performed in accordance with these 10 steps. You can define a more detailed number of steps or less. The challenge is to ensure that key strategies and actions are recognized, logically grouped together, and then executed in the appropriate sequence.

In conclusion, we believe that a good M & E system should be reliable enough to answer assessment questions, facilitate training and meet reporting needs, but not so rigid and inflexible as to deter the appearance of unexpected results.

Monitoring system requirements

Monitoring and warning requirements are often tied to the application in the background, which can extend the project lifecycle for many months.

With a predefined list of requirements, the system will be developed faster, and new applications and platforms will be deployed with a pre-enabled structure for monitoring and alerting.

These requirements must be quickly transformed to meet the needs of our customers. Specific implementations are not described, only monitoring requirements.

These indicators, taken together, should give an accurate picture of the state of the platform, infrastructure, and all components of the application. Here they are:

1. Delay

• The time it takes to service the request, whether it is a web application or some other application.
• Latency should always be measured in terms of both successful and unsuccessful requests, and both cases should be separated.

2. Movement

• A measure of the demand for the system.
• This value may differ by definition. For a website, this can be the number of requests per second. For a database, this can be “the number of requests executed per second”.
• All individual services should have meaningful traffic measurements.

3. Mistakes

• How often does the system or application crash?
• What are the causes/nature of the errors?
• Failures can be:

– Explicit: for example, a core dump or error 500

– Implicit: the request is returned with incorrect data or the website returns 200 OK, but with an error.

4. Saturation

• Measures both:
• Current usage: for example, CPU usage or memory usage.
• Predicted usage: Your battery will run out in 20 minutes

Here are the first one of the requirements. Later some new will be added, but it all depends on your customer’s needs. 

Benefits of a monitoring system

Continuous monitoring of system performance provides several advantages.

Continuous monitoring of system performance can do the following:

• Sometimes they identify the main problems before they affect them.
• Identifying problems that affect user productivity
• Collect data when the problem occurs for the first time
• Let you set a baseline for comparison

Successful monitoring involves the following:

• Periodically getting performance information from the operating system
• Saving information for future use when diagnosing problems
• Displaying information for the system administrator
• Detecting situations that require the collection of additional data, or responding to the instructions of the system administrator to collect such data or both.
• Collecting and storing the necessary detailed data
• Tracking changes made to the system and applications

Where to order the development of a monitoring system?

You can order the monitoring system design for your device from the ADUK team included our firmware developer and other. We already have successful experience in creating such an IoT remote monitoring system. We have a challenge: To develop a full-automatized and easy-access system for the monitoring status of pumping equipment. 

Pump Monitoring System – is a device that monitors data from sensors on the GUI. Data from sensors required for pump operation, pump start function, is displayed on the screen. With the help of the GUI, it is possible to control the entire system.

Workflow the Solution Followed: 

• The integrated system controls all operations on the pump
• The data transfers to the server for  storing and retranslating
• Getting information from any device using special software.

Key Feature: Optimized multithreading and monitoring from different devices.