ASSIST Telemedicine

There have been numerous projects in the area of Telemedicine that developed fantastic technical solutions which allow moving medical information instead of people. What seems to be an engineering exercise often turns out to have much more facets than initially expected. The result is that many projects struggle to turn into viable services as they lack parts.  To better understand this phenomenon and to reduce the effect of this spiral, ESA launched the AsseSSment and evaluatIon toolS for Telemedicine (ASSIST) study, sponsored by the ESA General Studies Programme (GSP).
ASSIST conducted a literature review and found only bits and pieces to assess a Telemedicine service in all its dimensions. A consistent approach did not exist. Many assessment methods restricted their view on the expenditures for such a service, some also on cost savings, but there are numerous benefits that don’t come with a price tag. For example, how do patients value the comfort of not needing to travel?
Many methods base their results on clinical trials that require large efforts to be conducted and a certain degree of technical maturity. When developing a service, maturity increases with every new version, but flexibility diminishes. One would need a method that could work from the beginning and mature along with the service to guide the development. Nevertheless the results should be rigours. Instead of using data with high internal validity, a method would need to allow estimating the validity of the data. Results can then be tested for the rigour.
Another aspect is the consideration of the different stakeholders affected by the adoption of Telemedicine. Often projects overlook that there are many of them: hospitals, doctors, nurses, general practitioners (GPs), informal carers, payers, technology providers, just to name a few. And, of course, the patients. What happens, for instance, when a service can now be conducted by the GP instead of the hospital because the practice has a Telemedicine system supporting them in difficult cases? This looks likely good for the practice, but is this good or bad for the hospital? Overall, summing up the different impacts, is the introduction of such a service good or bad? What is the effect of time? Is this bad at the beginning due to the initial adaptations, and expected to become good at a later time? The financial officer of the hospital might have a different answer to this than the hospital doctor freed from workload. When looking at the bottom line only, it would not turn out that the new service might be objected by hospitals.
The ASSIST project had thus to develop their own assessment framework. It builds on Cost-Benefit Analysis (CBA). In several countries, CBA is the recommended method for evaluation of public investments. The UK Green Book is an example. CBA looks at all impacts generated by the service whether they are investments, increased effort or improved wellbeing. It turns these impacts into monetary units which makes them comparable. The result of a CBA is the difference between positive impacts (benefits) and negative impacts (costs). Positive and negative impacts are not only of financial nature (e.g. money spent or saved), but include other aspects such as time saved for a specific stakeholder, or time needed for training. The key assessment figure calculated by ASSIS is the Socio Economic Return, which depends on the specific stakeholder, the point in time considered and what has happened until this point in time.

Every ASSIST assessment is a comparison between a given status, which the evaluator will use for comparison, and an intervention via Telemedicine. A comprehensive understanding of usual care is the best status quo measure. In the case of ASSIST, the intervention is not a single point in time but a process of changing from one status to another. It includes development, piloting and scale-up. ASSIST’s asset is that is can be used for formative evaluations projecting potential future gains and losses and direct further development and investment decisions.

ASSIST has defined 30 stereotypes of stakeholders that are meaningful to Telemedicine services. An assessment does not necessarily need to make use of all 30 different types of stakeholders. Most assessments only involve four or five of them. Each stakeholder has its own set of pre-defined indicators covering the most important impacts occurring in Telemedicine. In total, ASSIST has defined 339 indicators for its 30 stakeholders. For example, an indicator such as ‘avoided admissions’ connects the output of the technical system (control of patient’s conditions) with outcomes (better health through close control) and their impact on a stakeholder (less or completely avoided time in hospital). This has an impact on:

• the patient who doesn’t need to stay in the hospital,
• the hospital which does not need to provide the service and may lose income,
• the health insurance organization which saves money as care didn’t need to take place.

Indicators might or might not apply in the specific case. Admissions will not be avoided in every case. Maybe only the time spent in hospital is shorter, but with the proposed indicators ASSIST wants to make the users think whether this impact occurs or not. Developing and refining the indicators’ set was the biggest challenge of ASSIST. It needed to be generic enough to cover all conceivable configurations of services and to be specific enough that the user is not overwhelmed with random indicators that do not fit his situation.

The ASSIST method was implemented in a spread sheet self-assessment tool. Giving non-health economist access to expert knowledge via an easy to use tool was a major aim of the project. The tool and method were then validated with several ESA Telemedicine pilot projects. Arnaud Runge, technical officer at ESA, said: “The validation of the ASSIST toolkit has been performed involving four pilot Telemedicine projects in different stage of maturity. It was very interesting to see that through the instantiation of this toolkit, each project team discovered important aspects about the sustainability of their service. ASSIST proved to be a tool that helps project teams questioning and better understanding the consequences of their ideas, even before predicting the trend of the net impact brought by Telemedicine in the months to come. As a consequence, we are now using systematically this tool for our new Telemedicine projects”.

Empirica (D), the company leading the project, has made the ASSIST tool and method available under GPL Licence for everyone to use free of cost. It can be access under www.assist-telemedicine.net. Project partners were Telbios (I) and IRER (I).