The materialisation phase describes the realisation of the project idea (e.g., health app, novel software for online treatment and rehabilitation). The materialisation phase in a digital intervention project includes the development of the intervention itself.
It can be challenging to resolve mismatches between project planning and concept realisation. Based on the experience of the members of the advisory panel, the three most challenging tasks in the materialisation phase in a project involving digital health intervention are:
1) the cooperation between IT professionals and researchers,
2) involving/recruiting users (e.g., research participants) and consider users’
feedback to improve the usability of the intervention, as well as
3) the re-evaluation and adjustment of the project plan to changing circumstances and unmet needs.
Cooperation between IT professionals and researchers
“At first we experienced difficulties with the IT firm and their developers managing our (...) app, as they were sitting far away from us. When problems occurred, nobody could fix it, and we had to wait sometimes for days to get the app running again.
During that time, we could not collect data, and concerned study participants contacted us. That was unfortunate!”
Researchers highly rely on IT firms who, for example, develop the software for the digital health intervention. IT-related issues are common in projects, and cause delays, for example, due to lack of immediate fixing of technical issues. Researchers fear that, due to technical problems, research data may not be collected and that users get uninvolved with the digital health intervention to a point, where they might drop out of the study. In addition, IT firms are powerful suppliers, and as a layman, it can be challenging to negotiate conditions and prices.
One additional difficulty that frequently arises is the communication between healthcare and IT professionals. Healthcare professionals seem to struggle predominantly with the comprehension of technological challenges, while
IT-professionals seem to struggle with the organisational structure given in a research project. In addition, terminology between the two fields can differ and cause a different understanding of context and content.
Suggestions
The software specification has to be evaluated with the IT firm. It is a good idea to develop a legally binding contract with software specifications on project milestones, including hard deadlines and specifications on the software documentation. It is also important to negotiate future IT support services and possible extensions after the project. The contract has to ensure that project specifications, developed in the inception phase, are implemented correctly. The contract should ensure that in the case of mismatches or incomplete implementation of the specification the IT firm can be held liable.
Get support and help from supervisors and authorities when it comes to contracting and negotiating services with IT companies. The annex provides information on contracting laws and regulations involving a digital health intervention in Denmark.
Design project specifications to define mutual goals, especially with IT professionals.
Hand-drawn mock-ups and basic visualisation of the software and its interface can help to clarify communication.
The development process often reveals different understandings of specifications between the IT developer and the researcher. It is crucial to monitor the
development closely to detect and clarify misunderstandings early in the process in order to avoid further mismatches. This can be especially challenging as the
researchers lack the technical literacy and expertise for the software development.
Agree on an iterative and agile development model with the IT firm to define hard deadlines for evaluating the software at different development levels. Additional control can be obtained through models like Scrum that involve “customer”
involvement where researchers can help to formulate and prioritise the iterative development process (20). One advisor found it a positive experience to directly employ a software developer as part of the research team. In the annex contains a list of recommended resources to get started with agile development.
User involvement and feedback
“It is important to get constant feedback from users to improve the platform and solve occurring problems.”
Testing prototypes in the software development process and gather user feedback in pilot studies is crucial to validate that the digital health intervention matches the expectations and needs of the target group (e.g., Health professionals and/or patients). However, it can be challenging to recruit the representative user groups that give useful feedback and to initiate trials in clinical settings where they have to fulfil clinical, administrative and legal standards.
Suggestion
Contact health providers who have experience with piloting and conducting clinical trials and can support the recruiting process to target the right user group. In the process, prepare, for example, questionnaires to gain insights on user satisfaction and users’ needs. Receiving regular feedback can help to prevent and detect hindering features in the digital health interventions architecture, program and configuration (21) (Tools in annex). Thus, actively involving users is crucial when implementing digital health interventions.
Also, it is important to involve relevant users (e.g., health professional and patients) during the whole process. In this way, you make sure that the product matches the needs of the users. Another benefit of involving health professionals is that it can convey the feeling of ownership, which can lead to a more positive attitude towards the digital health.
Re-evaluation and adjustment of the project plan
“The main goal with this approach is to ensure that the project objectives align with users’ needs.”
During the course of the project, it can become apparent that the original project vision, as well as the project plan, needs to be adjusted due to potentially new requirements, arising challenges or identified mismatches with user needs. In this particular case, the updated vision has to be communicated to the stakeholders to ensure that the new project vision is correctly implemented (19).
Suggestion
Re-analyse regularly whether the user feedback and project vision align and if not, it can be necessary to develop a new project vision. Check with other research group members that the project vision still reflects the objectives of stakeholders and user needs (8).