Kræftens Bekæmpelse Report 2017/2018
A practical toolkit for researchers working with digital health
interventions for patients
Authors:
Johanna Gutenberg Lars Kayser
E-mail:
gutenbergjo@gmail.com
Content
Background ... 2
Nomenclature ... 2
Objectives ... 3
Incentive ... 3
Identified Challenges and suggestions 5 1. Inception ... 6
Project planning ... 6
Adapting existing studies/projects ... 7
Organisational challenges ... 8
Stakeholder involvement ... 9
Acknowledgment from the project environment ... 11
Technical feasibility ... 12
Legal feasibility... 12
Determination of assessment tools and evaluation parameters ... 13
2. Materialisation... 14
Cooperation between IT professionals and researchers ... 15
User involvement and feedback ... 16
Re-evaluation and adjustment of the project plan ... 17
3. Implementation ... 18
Establishing communication processes during roll-out ... 18
Planning a soft transition from existing interventions ... 19
Integrating new interventions into clinical workflows ... 19
4. Final assessment and evaluation... 20
Annex 22 Research methodology ... 22
TOOLKIT ... 23
1. Inception phase ... 23
2. Materialisation ... 39
3. Implementation ... 40
4. Final assessment and evaluation ... 41
Reference 43
Background
A modern, well-functioning healthcare system is characterised by successful integration of adaptive, new, disruptive and innovative technical means to improve healthcare delivery. The main drivers for a continuous transition from traditional healthcare delivery to digital healthcare delivery are the requirements of cost- effective, accessible and user-friendly interventions that actively support treatment, health promotion and empower patients in their self-management and disease prevention efforts. However, the introduction of a digital health intervention into a healthcare setting causes various challenges and incentivises further attempts to plan, navigate and implement projects successfully.
The EDITH (Evaluating Digital Therapy and Health) network was established in 2013 with support from the Danish foundation TrygFonden. The main purpose of the network is to build relationships, share knowledge and increase visibility in the research field of digital health interventions.
The main contributions for this report come from an advisory panel consisting of researchers from the EDITH network working in clinical settings or research institutions. They all have experience with the process of integrating new digital interventions in the Danish healthcare sector.
The report pinpoints the main challenges, suggestions and provides information on tools to support the development, implementation, and evaluation of digital
interventions, which target person-centred, value-based health promotion, treatment, rehabilitation and/or palliation in a Danish/Nordic context.
Nomenclature
In this report, the term digital health intervention is used interchangeably with the terms, health-IT, mhealth, telehealth, and telemedicine. These terms describe technologies used in the healthcare sector, such as medical and clinical devices, applications, desktop and Internet-based software that offer service and treatment.
The term patient is interchangeably used with client, person and user.
Objectives
I. Provide a guiding text based on identified challenges II. Present suggestions to these challenges
III. Provide a practical toolkit for researchers with resources based on the identified challenges and suggestions
Incentive
Why is it worth creating a guiding text including a toolkit for researchers based on experience from other health-it research projects?
It is motivated by the challenges in current processes of developing and executing research projects concerned with digital health interventions. That is because embedding new technologies involves complex processes of change. In part, because technologies involve a wide range of different items (content, interaction, user, platform, references, and interface) (1).
To meet these objectives, an advisory panel of eight informants shared their project experiences and provided input such as guiding articles, reviews, overviews and policy papers. By this means, it is possible to get an insight into challenges
experienced with digital health interventions in research projects. Further, it provides suggestions to address these challenges. Thus, it offers a better way to minimise the risk of failure of the research project.
A major challenge is to start an innovative process from scratch and deal with on- going and repeating challenges. For example, turning research ideas into evidence- based problems and needs, to receive funding, create a project, run the project, and understand risks and benefits, as well as to prove additional benefits and
effectiveness in comparison to the current (non-) digital processes and procedures (2,3).
Another common difficulty is to navigate a research project concerned with, digital- based treatments options with strict medical, legal and clinical regulations that are still flexible and responsive enough to include relevant stakeholders’ ideas and needs (4,5).
The advisory panel
The advisory panel consist of the following persons EDITH network
Ditte Hoffmann Jensen Psychologist, PhD Fellow at Functional Disorders - Research Clinic for Functional Disorders, Denmark
Helle Sofie Wentzer, Senior Researcher at the Danish Institute for Local and Regional Government Research, KORA, and Adjunct Professor in the
Department of Communication and Psychology at Aalborg University, Denmark Kim Mathiasen, PhD Fellow, Centre for Telepsychiatry and E-mental Health,
University of Southern Denmark, Denmark
Lars Kayser, Associate Professor, Department of Public Health, University of Copenhagen, Denmark
Malene Flensborg Damholdt, Postdoc. Institute for Culture and Society, Section for Philosophy and the History of Ideas, Denmark
Researchers and Consultants working on relevant projects or in the field of Digital health innovation, eHealth, and innovation in educational institutions Johanna Gutenberg, eHealth Researcher, Eriksholm Research Centre, Denmark Casper Lund-Andersen, PhD Fellow, Institute for Cellular and Molecular Medicine,
University of Copenhagen, Denmark
Malte Kongstad Deleuran, Research assistant at CopenhagenRehab and Center for physical activity, Denmark
Rikke Kortsen Okholm, Consultant, Institute for ”Naturfagenes Didaktik”, University of Copenhagen, Denmark
Identified Challenges and suggestions
The identified challenges have been categorised into different phases in the
framework of the “GoTo navigator” developed by Kayser et al. (under review). It is a framework used for development and management, implementation and evaluation of digital health interventions.
The framework divides the IT research process into four phases.
Inception: project planning and start in health IT
Materialisation: realisation of the project
Implementation: concept to place the digital health intervention into effect
Final assessment and evaluation: measure impact of the intervention
In the following, challenges in developing, realising and implementing a digital health intervention are categorised according to phases and suggestions. The suggestions are based on the contributions from an advisory panel and on identified literature.
1. Inception
A project starts with the project inception and start-up phase. The following sections describe the challenges experienced by the researchers in the advisory panel, who invented a digital intervention themselves or guided projects in a research context.
Most challenges experienced in the inception phase concern the project planning, understanding organisational structures of a healthcare system as well as adapting existing studies and interventions from other projects to the own project. Also, evaluating the technical and legal feasibility is challenging as it requires technical and legal literacy. Many inexperienced researchers, who try to innovate new digital health interventions, fail to involve stakeholders and communicate the scope of the project in a way that yields positive recognition for starting the projects.
Project planning
“In the beginning, I spent almost one year in trying to find out, what we should do with the technical solution. I am a psychologist and I am able to create all the content but to create the framework that was really difficult.”
The challenge in the initiation of a health IT research project is to establish a project definition and project plan that reflects the project vision. Also, if the project plan is not realistic enough, the risk that the overall process will be delayed increases.
Inexperienced researchers spend a lot of time on pre-project planning and are overwhelmed and challenged by all the required tasks; to navigate through pre- project processes, to understand and adjust the own individual process to already existing processes, procedures and products.
Suggestion
Think about what could be improved with the implementation of the new digital health intervention. One advisor from the advisory panel proposed a set of questions important to plan a digital health intervention (See below). It can be used as
inspiration. Further, identify existing guidelines and models for project planning and management (6) (See references in annex).
Examples of questions stressed by advisors Project planning
What is your project vision?
What is a realistic project duration?
What are the project deliverables?
How do you manage quality, time and costs?
What setting are you planning and designing for?
Who is the hosting organisation?
Is it a treatment, rehabilitation or is it a prevention tool?
How is the assessment done (Feasibility study)?
Who is doing the assessment?
Who evaluates the success of the treatment?
Where lays the responsibility of therapy?
Where lays the legal responsibility?
Data privacy, data security, and GDPR?
How is it all followed-up?
Users
Who is the user (Target group)?
How does the user find you?
What is the user pathway?
What is the user interface?
Organisation
What setting are you planning and designing for?
Who is the hosting organisation?
What sort of software do you plan to use (Desktop or Internet software)?
Adapting existing studies/projects
“The inspiration for this project came from a similar project in Japan. It needed translation and adaption into a Danish context. The project process from planning to the development took about 1.5 years.”
“I think because the primary problem often is that you start these projects as research and then you do not have the framework to continue the treatment afterwards because maybe you have borrowed a treatment platform from, for example, Karolinska University or another hospital. So it is actually not possible to take your source directly into practice.”
It can be a good idea to use existing methods/interventions from other projects, or parts of it, when developing new digital health interventions. However, it can be difficult and time-consuming to extract, translate and customise information and material from pre-existing studies/projects into the specific project context. Another challenge emphasized by the informants, was the problems that arise when the project must continue after the research period is finished. If an existing intervention has been used there might not be an agreement on whether the intervention can be used in the future.
Adaptation of content in a meaningful way
Translations in the Danish cultural and organisational context
Intervention style and format
Suggestion
It is recommended to investigate whether similar interventions exist in Denmark and internationally. Be aware of differences/similarities in practice and validate the digital health intervention so that it fits, for example, into the organizational context and is accepted by the user group. Additionally, be aware of deviating terminology, in terms of keeping it culturally relevant and authentic.
Assess economic and human factors and requirements to gain an understanding of how to translate an existing technical solution in a technically feasible way.
Furthermore, it is essential to have an agreement on what will happen when the research period ends.
Organisational challenges
“All doors were really closed and the hospital hadn’t had the structures and a treatment reimbursement system to offer Internet-based treatment options, yet.”
It can be time-consuming and challenging to understand organisational structures. It is often difficult to get an insight into organisations, companies and individuals ways of operating.
It is also challenging to develop a digital health intervention, which suits the patients, the healthcare professionals and fits into a given organisational structure.
It can also be difficult to convince people working in the current infrastructure to adapt to new digital health interventions.
Suggestion
When innovating for a healthcare setting, it is advisable to analyse the innovative capacity of the specific healthcare setting, as well as the attitude towards novel digital health interventions. Therefore, it is recommended to make a strong business model including topics such as reimbursement systems and to learn about the organisation infrastructure in the early planning phase of the project (7). With a strong business model for the new inventions, you demonstrate the benefits and the capability of innovating an intervention that fits into the current organisational
structure.
In order to make a strong business case, it can be helpful to follow a business model template, such as business model canvas, which includes critical steps that help to think about the value you are generating for an organization (e.g., hospital), the infrastructure of the organization, the users of your digital health intervention, and the reimbursement. Information on how to write a business model can be found in the annex.
Stakeholder involvement
“It took me so long to find out who is relevant to the project and who to contact about legal and organisational requirements, tariffs systems, IT, Illustration and
animations.”
“Over time I compiled a list of relevant contact persons and their functions beneficial for the project.”
Many researchers bring their discipline-specific expertise and rely on other
professionals’ determination to drive the development of a digital health intervention in healthcare. In research projects, it often occurs that the project design
insufficiently defines relevant stakeholders and their roles. Further, inter-and
multidisciplinary collaborations cause different challenges in sensitive areas such as communication, working culture, and job specification.
Failure to establish crucial communication links makes correct and qualified
information less accessible to the researcher and often causes frustration among the people involved in the project.
Another part of the challenge is to convey the feeling of ownership for all parties involved in the project from a very early stage. In the process, it will ensure that all parties, at the individual researcher level, feel understood, engaged and committed throughout the course of the research project.
Suggestion
The success of digital interventions improve with the involvement of stakeholders.
Therefore, it is important to:
Identify project roles of stakeholders with different perspectives involved in the project.
Ensure that project member, internal suppliers (e.g., software development team or external contractors), target users (e.g., patients, health professionals), and other project-related stakeholders are informed about the project details relevant to their role throughout the entire period. This conveys the feeling of ownership by involving the stakeholders (18).
Analyse the communication flow in the organisation. Try to detect potential issues leading to miscommunication with target users, health professionals and
stakeholders. Create a communication plan in order to focus the message and reach the target audience. The annex provides a resource collection with contact information on relevant organisations, services, tools and information for each region of Denmark and a communication plan template in 9 steps.
Analyse how the digital health intervention effects these stakeholders
Understand it as an iteration process
Plan workshops and dialogues with central groups.
Plan and allocate some time to work for this part of the process (2). The annex includes tools and references with information on how to identify relevant stakeholders and their role (2).
Acknowledgment from the project environment
“[…] since it got recognised in a positive way, everybody is very supportive and a lot of things have happened in these five years, also in the organisation, they are actually prioritising this kind of project more and they are interested in implementing it[… ]”
An identified challenge is to receive positive recognition and support from
stakeholders on meta-, meso- and micro-level in a healthcare system. Research projects, well recognised and supported by healthcare authorities and healthcare settings from the very beginning, have a better chance of being finished in time and being successfully implemented in a healthcare setting (19).
Apart from that, there are challenging external factors, for example, fundamentals of politics, policy changes and decision‐making processes on the macro level. The consideration of unforeseeable external factors requires a certain degree of
flexibility, time and resources in the course of a research project. This collides with rigid organisational structures in a healthcare system and researchers’ limited resources.
Suggestion
It requires detailed information on local organisations, partners and relevant stakeholders who provide expertise and consultancy in different subject areas.
The main and most critical part is to build a mutual understanding and transparency on each other's work, to create a well-functioning, meaningful organisational
structure.
Technical feasibility
“We started from scratch and we were supposed to develop both the psychological content but also the technical content to treat patients. We had no experience with how to develop such platform and what requirements that had to be fulfilled to do so.”
Estimating the complexity of technical solutions is difficult. A lack of technical expertise makes it hard to obtain realistic estimates. Thus, hidden costs and risks are easily overlooked and not included in the pre-project planning.
Suggestion
It is useful to design and conduct a feasibility study very early in a project. A
feasibility study is a common tool to test technical complexity and estimate technical challenges that may occur. In addition, it is recommended to define risks and include a risk analysis and a risk mitigation plan. The risk-analysis helps to identify risk related to technical challenges. A risk mitigation plan helps to manage, reduce and eliminate those risks to a level that ensures that the overall project success is not jeopardized. It is recommended to implement the risk mitigation plan, continually monitor it, and adjust it with the intent of changing the course-of-action if needed.
Legal feasibility
“The technical solution caused most problems, how to fulfil the legal requirements [… ]” For example, if we have a two-step login system,[… ] you need a confirmation, so we have chosen to use NEMID but this took many months to implement this part on the platform because of data security[… ].”
The development of digital health interventions has to oblige legal requirements to support privacy and data security requirements, as well as data protection issues for data sharing to assure patients data security and safety (9). Data security issues are one of the major challenges in health IT (9–11).
Suggestion
Since May 2018, Data protection is governed by the EU general data protection regulation (GDPR). Since then, institutions dealing with personal data have
responsible persons (data protection officers) concerned with data security and data protection. They can help to follow the current EU general data protection regulations (GDPR). Before consulting GDPR experts, you can think about how you are going to get the user's consent on data usage, how and where you want to store the data and how you are going to inform the users about data usage and data storage.
Also, it is recommended to consider information from legal services. Most
researchers working with health IT projects are eligible to use the legal department from the local University and/or from the Region. The annex provides a resource list with contact information for each Region in Denmark, as well as information and references to EU general data protection regulations. Further, there is a reference to a legal assessment tool (LAT) that addresses privacy and data protection issues for data sharing.
Determination of assessment tools and evaluation parameters
“It is important to have meaningful and robust measures that generate evidence and measure the impact of an intervention”
“How feasible is a project? If the need is not big enough or the patient group is very small [… ]
According to all consulted advisors, digital health interventions must be based on a robust study design and scientifically robust assessment tools, including comparable parameters that measure clinical utility/effectiveness. Also, it is important to choose an appropriate assessment tool with meaningful evaluation parameters, as it is important to measure the impact of the digital health intervention in order to receive the necessary funding.
It can be challenging to govern a research project that has to follow traditional research guidelines , as innovative projects often require very flexible and quick changes in strategy and organisation as well as adjustments from theoretical parameters translatable to practice(2). Thus, the challenge is to identify reliable
methodologies for the assessments of digital health interventions in the pre-project phase that comply with both and the delivery of reliable research in an agile,
innovative way.
Suggestion
Being dependent on funding requires early determination of reliable methodologies for the assessments (8) (evaluation and success criteria) of digital health
interventions in the pre-project phase.
The annex provides references to international studies that cover context aware methods (11,12), and tools on how to identify and recruit the right target group/user (13–15), as well as on how to translate information into the own cultural and
organisational context (16) and intervention style and format (17).
It includes concepts and guidelines for summative studies (e.g., randomised (mhealth and ehealth) clinical trials) and formative methodologies.
2. Materialisation
The materialisation phase describes the realisation of the project idea (e.g., health app, novel software for online treatment and rehabilitation). The materialisation phase in a digital intervention project includes the development of the intervention itself.
It can be challenging to resolve mismatches between project planning and concept realisation. Based on the experience of the members of the advisory panel, the three most challenging tasks in the materialisation phase in a project involving digital health intervention are:
1) the cooperation between IT professionals and researchers,
2) involving/recruiting users (e.g., research participants) and consider users’
feedback to improve the usability of the intervention, as well as
3) the re-evaluation and adjustment of the project plan to changing circumstances and unmet needs.
Cooperation between IT professionals and researchers
“At first we experienced difficulties with the IT firm and their developers managing our (...) app, as they were sitting far away from us. When problems occurred, nobody could fix it, and we had to wait sometimes for days to get the app running again.
During that time, we could not collect data, and concerned study participants contacted us. That was unfortunate!”
Researchers highly rely on IT firms who, for example, develop the software for the digital health intervention. IT-related issues are common in projects, and cause delays, for example, due to lack of immediate fixing of technical issues. Researchers fear that, due to technical problems, research data may not be collected and that users get uninvolved with the digital health intervention to a point, where they might drop out of the study. In addition, IT firms are powerful suppliers, and as a layman, it can be challenging to negotiate conditions and prices.
One additional difficulty that frequently arises is the communication between healthcare and IT professionals. Healthcare professionals seem to struggle predominantly with the comprehension of technological challenges, while IT-
professionals seem to struggle with the organisational structure given in a research project. In addition, terminology between the two fields can differ and cause a different understanding of context and content.
Suggestions
The software specification has to be evaluated with the IT firm. It is a good idea to develop a legally binding contract with software specifications on project milestones, including hard deadlines and specifications on the software documentation. It is also important to negotiate future IT support services and possible extensions after the project. The contract has to ensure that project specifications, developed in the inception phase, are implemented correctly. The contract should ensure that in the case of mismatches or incomplete implementation of the specification the IT firm can be held liable.
Get support and help from supervisors and authorities when it comes to contracting and negotiating services with IT companies. The annex provides information on contracting laws and regulations involving a digital health intervention in Denmark.
Design project specifications to define mutual goals, especially with IT professionals.
Hand-drawn mock-ups and basic visualisation of the software and its interface can help to clarify communication.
The development process often reveals different understandings of specifications between the IT developer and the researcher. It is crucial to monitor the
development closely to detect and clarify misunderstandings early in the process in order to avoid further mismatches. This can be especially challenging as the
researchers lack the technical literacy and expertise for the software development.
Agree on an iterative and agile development model with the IT firm to define hard deadlines for evaluating the software at different development levels. Additional control can be obtained through models like Scrum that involve “customer”
involvement where researchers can help to formulate and prioritise the iterative development process (20). One advisor found it a positive experience to directly employ a software developer as part of the research team. In the annex contains a list of recommended resources to get started with agile development.
User involvement and feedback
“It is important to get constant feedback from users to improve the platform and solve occurring problems.”
Testing prototypes in the software development process and gather user feedback in pilot studies is crucial to validate that the digital health intervention matches the expectations and needs of the target group (e.g., Health professionals and/or patients). However, it can be challenging to recruit the representative user groups that give useful feedback and to initiate trials in clinical settings where they have to fulfil clinical, administrative and legal standards.
Suggestion
Contact health providers who have experience with piloting and conducting clinical trials and can support the recruiting process to target the right user group. In the process, prepare, for example, questionnaires to gain insights on user satisfaction and users’ needs. Receiving regular feedback can help to prevent and detect hindering features in the digital health interventions architecture, program and configuration (21) (Tools in annex). Thus, actively involving users is crucial when implementing digital health interventions.
Also, it is important to involve relevant users (e.g., health professional and patients) during the whole process. In this way, you make sure that the product matches the needs of the users. Another benefit of involving health professionals is that it can convey the feeling of ownership, which can lead to a more positive attitude towards the digital health.
Re-evaluation and adjustment of the project plan
“The main goal with this approach is to ensure that the project objectives align with users’ needs.”
During the course of the project, it can become apparent that the original project vision, as well as the project plan, needs to be adjusted due to potentially new requirements, arising challenges or identified mismatches with user needs. In this particular case, the updated vision has to be communicated to the stakeholders to ensure that the new project vision is correctly implemented (19).
Suggestion
Re-analyse regularly whether the user feedback and project vision align and if not, it can be necessary to develop a new project vision. Check with other research group members that the project vision still reflects the objectives of stakeholders and user needs (8).
3. Implementation
The implementation phase describes the act of carrying out the new digital health intervention aligned with the predetermined vision, purpose and the project design.
“It is the hardest part, to actually go from project planning to implement something new in a workplace with the daily routine.”
The hardest part of a project is often to reach the implementation phase due to the nature of research projects and the aforementioned challenges in the preceding phases in projects concerned with digital health interventions. Furthermore, different healthcare setting such as primary care/secondary healthcare require different implementation strategies.
Establishing communication processes during roll-out
“That is the most difficult phase as it crucial to develop a communication process that reflects the project results and delivers information in a way that gets people on board to support and ensure the project implementation.”
Errors often occur due to a complex communication and working infrastructure in a healthcare setting, which can cause misunderstanding and misinformation about important project details. One advisor elaborated on an example experienced in a healthcare setting, where two different stakeholder groups, a technician and a nurse, were not able to comprehend and retrace each other’s work. This caused tension, mistrust and double work efforts on both sides, as well as frustration and a reluctant attitude to use the newly developed digital health intervention.
Also, its success depends on well-established communication, in consideration of internal authority forces and organisational structures in a healthcare setting (22).
Suggestion
Develop a communication process between researchers and decision makers that ensures the best possible outcome for the implementation of a new digital health intervention, in alignment with the overall organizational strategy and processes.
Also, give attention to the human and socio-technical factors in the particular
healthcare setting. This shows competence, consideration, and appreciation for the user of the digital health intervention, which enhances a successful implementation (23).
The annex provides a communication plan template.
Planning a soft transition from existing interventions
“For future projects, I wish I had a visionary plan where the implementation would be clearer but it is just not the way it works because of so many uncertain factors”
The major challenge is to plan the transition from an existing intervention to a new system. In particular, because of the complex setting and all the uncertain factors.
Insufficient planning of the implementation can have extensive consequences, as it can affect the quality of healthcare delivery and might consequently harm patients.
Suggestion:
When replacing an existing intervention, special attention has to be given to data safety and documentation.
It needs to be clarified in a legally binding document, who is accountable if, for example, the system fails or data loss occurs.
Conduct a risk analysis: identify risks in the workflow and try to minimize it. Create a manual on how to mitigate the risk to assure a soft transition from existing
solutions/interventions.
Create a handbook/manual for all functions and a help desk or hotline service to assist with any problems regarding the new system.
Integrating new interventions into clinical workflows
“It is crucial to ensure that there are training opportunities for staff[… ]. The success or failure of an innovation [… ] will ultimately depend on the attention given to
understanding and managing the people, who have to work with it daily. “
When health professionals, who are supposed to use the new digital health intervention, have not played an active part in the development process, it is
challenging to gain an appreciation and acknowledgement for the new digital health intervention. In addition, frequent system errors and bugs can result in a reluctance to the new intervention. Depending on the digital health intervention, it can be challenging to create a system where relevant information is fully transferable into the new system (interoperability and data safety), without disrupting current
workflows. Thus, the human and socio-technical errors can hinder a successful implementation.
Currently, it is difficult to determine reliable information on how healthcare institutions handle organisational changes, additional costs, and delays due to the
implementation process of new digital health interventions (24,25).
Suggestion
In the first place, it can be helpful to identify a healthcare setting that is open to change and already has experience with implementing new digital health interventions. Also, plan training sessions for the pre-defined user, e.g., health professionals, and empower them to become experts on the digital health intervention.
Communicate and present an outlook in the future and clearly demonstrate additional benefits and advantages the implementation of the new digital health intervention. This entails acceptance for e.g., initial technical problems and entitles the innovators to gradually further develop the digital health intervention; adjust the intervention to the given setting, and, over time, to unleash its full potential. The annex provides a tool with ten considerations for the successful implementation and adoption of health information technology developed by Cresswell et al. (2013).
4. Final assessment and evaluation
After implementation, researchers can measure the impact of a new digital health intervention with summative ex-post evaluation methods to elaborate whether the project objectives were achieved.
Suggestions
In any case, the final post-implementation evaluation is recommended, even if the implementation failed. This gives the researcher and the healthcare setting, as well
as other researchers, the possibility to learn from the project experiences and outcome, and thus contributes to further research and documentation (26). The annex provides information on how to conduct an evaluation.
Annex
Research methodology
The structure of the report follows the framework of the “GoTo navigator” developed by Kayer et al. (2017). It is a method used for the deployment of process
development management, implementation and evaluation of digital health interventions. Initially, it is important to emphasise that the search of needs, recommendations, toolkits and guidelines on digital intervention serves as
background information. Based on the selected literature, it is possible to cover the most important developments in the area of interest and knowledge of methods for improving eHealth projects. The database used for the literature search was
PubMed/MEDLINE. The analysis is done by the selection of relevant information in the title, and the abstract followed by reading relevant literature with the following search terms (guidelines OR recommendation OR “toolkit”) AND (eHealth OR digital health OR digital health intervention). The literature was limited to MEDLINE, peer- reviewed publications, in English only.
A Cultural probes interview design gave the structure that enabled a conversation on the experience working with developing digital health interventions in Denmark.
The “Cultural Probes” method serves for the collection of input from a preselected group of Edith network members. Cultural Probes is a method originally developed by Bill Gaver in 1999, as part of the EU Presence Project. Cultural Probe has
become a form of umbrella concept and now covers several different approaches to collect information in connection with design development. The design of the
approach of the Cultural Probe initiates inspirational responses to a certain topic.
The main idea is that the probes are part of a strategy to address common
challenges in conceptualising projects for unfamiliar groups and topics (27). Inspired by Cultural Probes, this project includes steps of the “GoTo navigator” as "probes."
The purpose of "probes" as a set of stimulus cards is to give the participants the chance to either work with them as an incentive to categorise their experience in different topic areas or speak freely about their experience working on digital health projects.
This method helps to identify which topic areas in the “Go-to navigator” applied to their projects. The most recurring experiences, including obstacles, challenges, benefits and strength in the projects are the basis for the components of the toolkit.
For each participant, set of coloured “navigator”-cards are used, and the supplied information of contributions are analysed with the help of audio-recorded files that are described and analysed accordingly. A thematic analysis approach inspired the analysis of data; first, an inductive step and then a deduction analysis. The inductive step consists of a thematisation (already pre-formulated on topic cards) of all care cards in order to identify cross-cutting problem areas and challenges
(subcategories).
Further, the report includes literature recommended through by the advisory panel and team members of the Edith network. In addition, grey literature via web pages, Google and Google Scholar search engines were searched for relevant information.
TOOLKIT
The following consist of literature, models, and suggestions to support and guide individuals who work with developing, implementing and evaluating digital
interventions for patients.
1. Inception phase Project planning
Stanford Biodesign Model from the book: Biodesign: the process of innovating medical technologies
The Stanford Biodesign model is a great tool to develop humen centred digital health interventions. Stanford Biodesign Model(2) defines innovation as a new idea,
process or product that requires need-based invention and implementation of medical technology innovation (Yock et al., 2015). In general, the scope lies on a systematic evaluation of healthcare needs, invention, and concept development as a process. This process can be applied to any medical discipline. Stanford Biodesign gained recognition in the United States and beyond because of its encouraging elements including creativity, network and synergy effects on process management.
The Stanford Biodesign enables innovative thinkers in the field of medicine and other disciplines, to build a bridge between science comprehension and clinical practices.
The process is published in textbook form with the title The Biodesign: The Process of Innovating Medical Technologies. This book includes and describes the method of making innovation happen.
The process is divided into three main phases namely identification, invention and implementation. The identification phase describes the identification of unmet healthcare related needs. The invention phase is sought to brainstorm how to address the most pressing need and develop and select a viable business model concept for it. The last phase is the implementation phase. The implementation phase is the first step where it is sought to translate the most promising business idea into practice, under consideration of, for example, intellectual property rights and regulatory legal factors.
Reference:
Yock, P. G., Zenios, S., Makower, J., Brinton, T. J., Kumar, U. N., Watkins, F. J., ... & Kurihara, C. Q. (2015).Biodesign: the process of innovating medical
technologies. Cambridge University Press.
West Midland Toolkit (WMT)
The West Midland toolkit is an eight-step innovation model developed for the project concerned with digital health innovation. The model gives a framework with steps to go through when planning a project in concerning health-IT.
Reference:
The West Midland Toolkit, Retrieved from
https://www.togetherforshortlives.org.uk/resource/west-midlands-toolkit/
Recommended literature to project planning and management of health IT projects
Ballard, G., & Howell, G. (2003). Lean project management. Building Research & Information, 31(2), 119-133.
Beck, K., Beedle, M., Van Bennekum, A., Cockburn, A.,
Cunningham, W., Fowler, M., ... & Kern, J. (2001). Manifesto for agile software development.
Boogerd, E. A., Arts, T., Engelen, L. J., & van De Belt, T. H. (2015).
“What Is eHealth”: time for an update?. JMIR research protocols, 4(1).
Ciccarese, P., Caffi, E., Quaglini, S., & Stefanelli, M. (2005).
Architectures and tools for innovative health information systems: the guide project.
International journal of medical informatics, 74(7-8), 553-562.
Fiordelli, M., Diviani, N., & Schulz, P. J. (2013). Mapping mHealth research: a decade of evolution. Journal of medical Internet research, 15(5).
Grew, J. C., Svendsen, M. N., & Mortensen, B. B. (2011).
Teoribaserede patientanalyser i Medicinsk Teknologivurdering: En metodepublikation.
Hughes B, Cotterell M. Software project management. (2002) Software project management. Tata McGraw-Hill Education.
Kayser L, Furstrand D, Rasmussen E, Jensen L, Monberg A, Karnøe A. GoTo Navigator -a process navigation tool for digital health solutions to be used to design and align with an efficient trajectory for goal setting to evaluation of outcomes. (in review).
Martínez-Pérez, B., De La Torre-Díez, I., & López-Coronado, M.
(2013). Mobile health applications for the most prevalent conditions by the World Health Organization: review and analysis. Journal of medical Internet research,15(6).
Morrison, L. G. (2015). Theory-based strategies for enhancing the impact and usage of digital health behaviour change interventions: A review. Digital Health, 1, 2055207615595335.
Schwaber, K., & Beedle, M. (2002). Agile software development with Scrum (Vol. 1). Upper Saddle River: Prentice Hall.
Stellman, A., & Greene, J. (2005). Applied software project management. " O'Reilly Media, Inc.".
Zanaboni, P., & Lettieri, E. (2011). Institutionalizing telemedicine applications: the challenge of legitimizing decision-making. Journal of medical Internet research, 13(3).
Adapting existing studies/projects
Recommended literature to adapting existing studies/projects
Arthur, W. B. (2009). The nature of technology: What it is and how it evolves. Simon and Schuster.
Yock, P. G., Zenios, S., Makower, J., Brinton, T. J., Kumar, U. N., Watkins, F. J., ... & Kurihara, C. Q. (2015).Biodesign: the process of innovating medical technologies. Cambridge University Press.
Organisational challenges
Recommended literature to organisational challenges
Baldwin, C. Y. (2012). Organization design for distributed innovation.
Cardinale, A. M. (2018). The Opportunity for Telehealth to Support Neurological Healthcare. Telemedicine and e-Health.
Cohen, E. (2013). Accelerating digital health innovation: Analyzing opportunities in the healthcare innovation ecosystem (Doctoral dissertation, Massachusetts Institute of Technology).
Lennon, M. R., Bouamrane, M. M., Devlin, A. M., O'Connor, S., O'Donnell, C., Chetty, U., ... & Watson, N. (2017). Readiness for delivering digital health at scale: lessons from a longitudinal qualitative evaluation of a national digital health innovation program in the United Kingdom. Journal of medical Internet
research, 19(2).
The following section gives an overview of touch point In Denmark, sorted by Regions.
Region Hovedstaden (The Capital Region of Denmark)
In January 2012 the Capital Region of Denmark established a unit called research and innovation “Forskning-og-innovation”. The unit offers researchers and health professionals guidance when it comes to:
Financing of projects
Legal assistance to contracts
Commercialization of inventions
Collaborations with higher education institutions
University of Copenhagen
Technical University of Denmark
Metropol University of Applied Sciences
Sund Vækst Huset
Afdeling for Velfærdsinnovation Københavns Kommune
Sundheds- og Omsorgsforvaltningen Center for Innovation og Digitalisering Henrik Pontoppidans Vej 4, 1. sal 2200 København N, Denmark Web:
https://velfaerdsinnovation.kk.dk/artikel/sund-vaekst-huset
SundVækst Huset is a part of the Department of Welfare Innovation at the Municipality of Copenhagen. SundVækst cooperates with citizens, employees, companies and research institutions to jointly develop and support the best welfare technology solutions.
Contact:
Region Hovedstaden-Center for regional udvikling, Vækst & Viden
Copenhagen Health Science Partners
Cooperation between the Capital Region and Copenhagen University in
Copenhagen Health Science Partners (CHSP) aims at promoting synergies and cooperation between excellent research, clinical work and education, thus helping to strengthen the future health care system.
Web:
https://www.regionh.dk/til-fagfolk/Forskning-og-innovation/Strategiske-rammer-og- samarbejde/samarbejde-vidensinstitutioner-CHSP/Sider/Copenhagen-Health- Science-Partners.aspx
Region Sjælland (Region Zealand)
Region Zealand offers under the section Region Sjælland│Sundhed│Forskning│For fagfolk│Værktøjskasse a toolbox for researchers. In the toolbox, there is practical information about statistical guidance, statistical programs, it-tools, templates, legal assistance and databases (see below).
More information:
Region Sjælland
Produktion, Forskning og Innovation (PFI) Alléen 15
4180 Sorø Tlf. 70 15 50 00 Fax 70 15 50 09
Web: https://www.regionsjaelland.dk They offer:
1. Statistical guidance and statistic programs
Researchers and PhD fellows in Denmark are affiliated with an educational
institution and thus have access to their statistics programs. PhD fellows rely on their formal PhD supervisors for statistical guidance. A researcher employed in the
Region Zealand is eligible to receive 10 hours of statistical guidance from a local research support unit.
2. IT Toolkit
Region Zealand made an agreement with a Danish company called “EasyTrial”. The IT-software is developed from Danish doctors to provide an Internet-based system that allows researchers and PhD fellows to manage relevant tasks associated with participants in a clinical study. A license for researchers costs 5.000 Danish kroner.
The same license can be used for different projects for multiple project members and users. There is also the possibility to use the software for a test period for free.
Web:
https://www.regionsjaelland.dk/Sundhed/forskning/forfagfolk/v%C3%A6r kt%C3%B8jskasse/Sider/EasyTrial.aspx
Templates
Here they offer templates and other practicalities that can assist you in your research process, for example templates for research projects, records, PowerPoint, consent statements, etc.
Web:
https://www.regionsjaelland.dk/Sundhed/forskning/forfagfolk/v%C3%A6rkt%C3%B8j skasse/Sider/Skabeloner-til-foredrag.aspx
3. Legal assistance to contracts Web:
https://www.regionsjaelland.dk/Sundhed/forskning/forfagfolk/v%C3%A6rkt%C3%B8j skasse/Sider/Juridisk-bistand-til-kontrakter.aspx
4. Access to registry data Web:
https://www.regionsjaelland.dk/Sundhed/forskning/forfagfolk/v%C3%A6rkt%C3%B8j skasse/Sider/Forskerservice-hos-SSI.aspx
5. Good research practice Web:
https://www.regionsjaelland.dk/Sundhed/forskning/forfagfolk/v%C3%A6rkt%C3%B8j skasse/Sider/God-videnskabelig-praksis.aspx
6. Access to journals and databases Web:
https://www.regionsjaelland.dk/Sundhed/forskning/forfagfolk/v%C3%A6rkt%C3%B8j skasse/Sider/Tidskrift-og-databaser.aspx
7. Clinical trial book Web:
https://www.regionsjaelland.dk/Sundhed/forskning/forfagfolk/v%C3%A6rkt%C3%B8j skasse/Sider/Clinical-Trials-Handbook.aspx
Region Syddanmark (The Region of Southern Denmark)
The Region of Southern Denmark established the Health Innovation Centre of
Southern Denmark.The purpose is to offer information and support when developing eHealth solutions and social care in the Region of Southern Denmark. It also offers a referral to legal guidance and proposes different cases as inspiration material.
The Health Innovation Centre of Southern Denmark has three strategic areas:
Innovation
Cooperation
Funding
The Health Innovation Centre of Southern Denmark Syddansk Sundhedsinnovation
Forskerparken 10G 5230 Odense M
WEB: http://www.syddansksundhedsinnovation.dk
Region Midtjylland (Region of Central Jutland) Centre for Telemedicine and Telehealthcare
Olof Palmes Allé 15 8200 Aarhus N.
Denmark Web:
http://www.rm.dk/sundhed/faginfo/center-for-telemedicin/
The Centre for Telemedicine offers support concerning
Guidance to the good video consultation
Law and Telemedicine
Frequently Asked Questions about Law and Telemedicine
Design and development
Minilab
How to apply for funds
Telemedicine on a large scale
Telemedicine and ethics
Evaluation methods: MAST and checklist
Tips to evaluate your idea
Help with business case
Medtech Innovation Consortium (MTIC) Tueager DK
8200 Aarhus N Web:
http://www.mtic.dk/
The Medtech Innovation Consortium works with private health technology and welfare technology companies and healthcare with business-based health innovation.
Region Nordjylland (The Region of Northern Jutland)
The Danish Center for Healthcare Improvement (DCHI)-“Dansk Center for Forbedringer i Sundhedsvæsenet”
Fibigerstræde 11 9220 Aalborg Øst Web:
https://www.dchi.aau.dk/
The Danish Center for Healthcare Improvement (DCHI) is a research centre
established in collaboration between Aalborg University and the Region of Northern Jutland. Through research, evaluations, studies and student projects, the Center develops new knowledge and new methods for use in the healthcare system.
Forskningens Hus-Ideklinikken Sdr. Skovvej 15
9000 Aalborg Web:
http://ideklinikken.rn.dk/
The Idea Clinic makes an assessment of how best to solve your problem or how best to realize your idea or invention - whether it is a patentable and sold product - or it is about improving a workflow that can facilitate your daily work work and routines in hospitals and hospitals.
Stakeholder involvement
Step-by-step guideline for stakeholder involvement for business modeling in eHealth technology implementation by Van Limburg et al. (2015):
Start with a literature review on comparable interventions to get a feeling for the domains, jargon, and global issues and stakeholders.
Involve 1 or 2 domain experts in the research and development team to reflect future findings, ideally experts who have an affinity with technology and research processes.
Make an overview of all possible stakeholders based on literature on comparable interventions in the domain.
Assign stakeholder types to possible stakeholders, verify if certain types are missing and why.
Validate the entire overview by snowball sampling a complete stakeholder list with these key stakeholders.
Let experts select key stakeholders from the complete stakeholder list.
Organize a focus group with at least one in-person representative of each key stakeholder:
o Start with discussing each stakeholders’ role in the current processes.
o Let them complete the stakeholder list for missing stakeholders based on the process.
o Ask stakeholders to rank the importance of stakeholders, or alternatively let experts do it later.
o Discuss what bottlenecks are experienced.
o Discuss opportunities for improvement and opportunities for eHealth.
Summarise bottlenecks and opportunities and determine with the research team which opportunities are there for eHealth technology and whether these fit the project goals.
Ideate an eHealth technology (when possible, make mock-ups or a prototype of the ideas).
Plan interviews with stakeholders, or if possible, multiple stakeholders of the same stakeholder type, for value co-creation dialogues for the ideated eHealth technology.
Prepare the value co-creation dialogue interview with questions that address all business model components (also prepare subquestions that propose possible ideas or values on each business model component to help the interview along.
Focus on what the technology should contribute to their daily routines, not technical requirements).
Code transcripts of the focus groups and interviews, extract all implementation- related comments and combine all values and critical factors in the business model canvas.
Discuss the resultant business model with the research team.
Optionally, for transparency and extra validation, explain the business model to stakeholders and let them reflect on it or write a document that explains the implementation strategy based on the business model as the model itself may be unclear to share with the relevant stakeholders.
Additional literature to relevant stakeholders, their role and value networks Reference:
Catwell, L., & Sheikh, A. (2009). Evaluating eHealth interventions: the need for continuous systemic evaluation. PLoS medicine, 6(8), e1000126.
Feng, W., Crawley, E. F., de Weck, O. L., Keller, R., & Robinson, B.
(2010). Dependency structure matrix modelling for stakeholder value networks.
van Limburg, M., Wentzel, J., Sanderman, R., & van Gemert-Pijnen, L. (2015). Business modeling to implement an eHealth portal for infection control: a reflection on co-creation with stakeholders. JMIR research protocols, 4(3)
Yock, P. G., Zenios, S., Makower, J., Brinton, T. J., Kumar, U. N., Watkins, F. J., ... & Kurihara, C. Q. (2015). Biodesign: the process of innovating medical technologies. Cambridge University Press.
Acknowledgment from the project environment
(See organizational challenges)
Technical feasibility
Bowen, D. J., Kreuter, M., Spring, B., Cofta-Woerpel, L., Linnan, L., Weiner, D., ... & Fernandez, M. (2009). How we design feasibility studies. American journal of preventive medicine, 36(5), 452-457.
Kayser L, Furstrand D, Rasmussen E, Jensen L, Monberg A, Karnøe A. (under review). GoTo Navigator -a process navigation tool for digital health solutions to be used to design and align with an efficient trajectory for goal setting to evaluation of outcomes.
Lancaster, G. A., Dodd, S., & Williamson, P. R. (2004). Design and analysis of pilot studies: recommendations for good practice. Journal of evaluation in clinical practice,10(2), 307-312.
Thabane, L., Ma, J., Chu, R., Cheng, J., Ismaila, A., Rios, L. P., ... &
Goldsmith, C. H. (2010). A tutorial on pilot studies: the what, why and how. BMC medical research methodology, 10(1), 1.
Legal feasibility
Information on laws and regulations involving data security in Denmark Datatilsynets
Web:
www.datatilsynet.dk
GDPR
Health data in the workplace
Retrieved from https://edps.europa.eu/data-protection/data- protection/reference-library/health-data-workplace_en The act on processing of personal data
Web:
https://www.retsinformation.dk/
Web:
http://united4health.eu/wp-content/uploads/2015/10/D5.5-v1.0-U4H- Industry-Report-on-Telemedicine-Legal-and-Regulatory-Framework.pdf
Additional references
Kuchinke, W., Krauth, C., Bergmann, R., Karakoyun, T., Woollard, A., Schluender, I., ... & Ohmann, C. (2016). Legal assessment tool (LAT): an
interactive tool to address privacy and data protection issues for data sharing. BMC medical informatics and decision making, 16(1), 81.
O’Connor, Y., Rowan, W., Lynch, L., & Heavin, C. (2017). Privacy by Design: Informed Consent and Internet of Things for Smart Health. Procedia Computer Science
Region Hovedstaden (The Capital Region of Denmark) Center for Information and Innovation Law (CIIR) Karen Blixens Plads 16
DK-2300 Copenhagen S Telefon: +45 35 32 31 92 Fax: +45 35 32 32 04 Web: http://jura.ku.dk/ciir/
In 2010, the Faculty of Law at the University of Copenhagen established the Centre for Information and Innovation Law (CIIR), as a part of the Faculty of Law at the University of Copenhagen. It serves as a research-based forum for legal issues in the areas of information and innovation, including copyright, trademark law, patent law, business secrecy protection, personal data and personal privacy in the Capital region.
CIIR's research concerns both national and international regulation and is for a number of research projects characterised by interdisciplinary illumination of the individual legal areas.
The Capital Region of Denmark offers under the section Research and Innovation (“Forskning & innovation”), information on legal assistance to contracts, process and a handbook for research contracts.
The Capital Region of Denmark
They provide legal assistance about research and innovation.
Region Hovedstaden
Kongens Vænge 2 - 3400 Hillerød Web:
https: //www.regionh.dk/til-fagfolk/forskning-og-innovation /Juridisk%20bistand%20til%20kontrakter/Sider/default.aspx
Region Sjælland (Region Zealand)
Region Zealand has entered an agreement with the University of Southern Denmark (SDU) on legal assistance for research projects by legalising:
• Cooperation agreements on research and development
• Clinical Trials (Clinical Trials)
• Material Transfer Agreements (MTA)
This means that a researcher can forward all of the above-mentioned agreements and contracts for review by SDU lawyers. For drug trials, it is required that SDU lawyers approve the agreements. The lawyers advise that researcher and PhD fellows consult the SDU lawyers when considering a collaboration in which a contract has to be developed. The lawyers can help to prepare the right type of agreements that fit the cooperation in question, and in some cases, they are able to prepare and send a template.
Region Syddanmark (The Region of Southern Denmark) The Health Innovation Centre of Southern Denmark Forskerparken 10G
5230 Odense M
Web:
http://www.syddansksundhedsinnovation.dk/service-menu/kontakt/
The Region of Southern Denmark established “The Health Innovation Centre of Southern Denmark”. The centre offers information and support for initiating and implementing eHealth solutions in the Regions of Southern Denmark. They do not directly offer legal service concerning digital health or innovation (research) projects but offers a referral to legal guidance.
Region Midtjylland (Region of Central Jutland)
Region of Central Denmark established in 2012 the Centre for Telemedicine. The centre offers guidance when dealing with digital health interventions in healthcare.
Under the section “Jura og telemedicin”, you can find information on legal questions when working with telemedicine. .
Centre for Telemedicine in the Central Denmark Region Skottenborg 26
8800 Viborg Web:
http://www.rm.dk
Danish Trademark & Patent Office (Information, registration and searches, trademark, design, patents & Utility models)
Web:
http://www.dkpto.org/
Innovation & Entrepreneurship
Business House Copenhagen, Copenhagen Municipality, Information and advice on starting a business
Web:
English: http://international.kk.dk/business
Danish: http://www.kk.dk/erhvervhttp://www.kk.dk/erhverv
Region Nordjylland (The Region of Northern Jutland)
The Region of Northern Jutland provides legal advice to all sectors and departments of the region
The “Research House” (Forskningens Hus) help researchers at Aalborg University Hospital prepare, negotiate and approve agreements on research with external partners.
Forskningens Hus Sdr. Skovvej 15 9000 Aalborg Web:
http://www.aalborguh.rn.dk/Forskning/Forskningens-Hus
Jura og Forsikring Regionshuset Niels Bohrs Vej 30 9220 Aalborg Øst Web:
http://www.rn.dk/Om-Region-Nordjylland/Organisationsbeskrivelse/Jura-og-forsikring
Determination of assessment tools and evaluation parameters
Davies, A., & Newman, S. (2011). Evaluating telecare and telehealth interventions. WSDAN Brief Pap Kings Fund, London.
Eysenbach, G., & Consort-EHEALTH Group. (2011). CONSORT- EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. Journal of medical Internet research, 13(4).
Eysenbach, G. (2013). CONSORT-EHEALTH: implementation of a checklist for authors and editors to improve reporting of web-based and mobile randomized controlled trials. In MedInfo (pp. 657-661).
Grew, J. C., Svendsen, M. N., & Mortensen, B. B. (2011).
Teoribaserede patientanalyser i Medicinsk Teknologivurdering: En
Technology Assessment for HIT development: Learning, feedback and user involvement. 2013.
Sampietro-Colom, L., Lach, K., Cicchetti, A., Kidholm, K.,
Pasternack, I., Fure, B., ... & Kiivet, R. A. (2015). The AdHopHTA handbook: A handbook of hospital-based Health Technology Assessment (HB-HTA); Public deliverable; The AdHopHTA Project (FP7/2007-13 grant agreement nr 305018);
2015.
2. Materialisation
Cooperation between IT professionals and researchers
(See understanding organisational structures and legal feasibility)
User involvement and feedback
Clemensen, J., Larsen, S. B., Kyng, M., & Kirkevold, M. (2007).
Participatory design in health sciences: using cooperative experimental methods in developing health services and computer technology. Qualitative health research, 17(1), 122-130. Holden RJ, Karsh B-T. The technology acceptance model: its past and its future in health care. J Biomed Inform. 2010;43(1):159–72.
Martikainen, S., Korpela, M., & Tiihonen, T. (2014). User participation in healthcare IT development: A developers’ viewpoint in Finland. International Journal of Medical Informatics, 83(3), 189-200.
Simonsen, J., & Robertson, T. (Eds.). (2012). Routledge international handbook of participatory design. Routledge.
Re-evaluation and adjustment of the project plan
References to re-evaluation and adjustment of the project plan
Stellman, A., & Greene, J. (2005). Applied software project management. " O'Reilly Media, Inc."
The Agile development
Beck, K., Beedle, M., Van Bennekum, A., Cockburn, A.,
Cunningham, W., Fowler, M., ... & Kern, J. (2001). Manifesto for agile software development.
El Sheikh, A. A. R., & Alnoukari, M. (2012). Business Intelligence and Agile Methodologies for Knowledge-Based Organizations: Cross-Disciplinary
Applications. Business Science Reference.
3. Implementation
Establishing communication processes during roll-out
Mair, F. S., May, C., O'Donnell, C., Finch, T., Sullivan, F., & Murray, E. (2012). Factors that promote or inhibit the implementation of e-health systems: an explanatory systematic review. Bulletin of the World Health Organization, 90, 357- 364.
Begrebsbasen
The Danish Health Data Agency (Sundhedsdata-Styrelsen) developed a platform called “begrebsbasen”. The idea is to provide a database with health-relevant terms to create a common understanding of healthcare terms across the healthcare
system.
Communication plan template Web:
https://www.crs.org/sites/default/files/crs-files/communication-toolbox-template- develop-a-communication-plan.pdf
Planning a soft transition from existing digital health interventions
Devlin, A. M., McGee-Lennon, M., O’donnell, C. A., Bouamrane, M.
M., Agbakoba, R., O’connor, S., ... & Browne, S. (2015). Delivering digital health and well-being at scale: lessons learned during the implementation of the dallas program in the United Kingdom. Journal of the American Medical Informatics Association, 23(1), 48-59.
Ludwick, D. A., & Doucette, J. (2009). Adopting electronic medical records in primary care: lessons learned from health information systems
implementation experience in seven countries. International journal of medical informatics, 78(1), 22-31.
Integrating new digital health interventions in clinical workflows
Ten key considerations for the successful implementation and adoption of large-scale health information technology
From: Cresswell, K, Bates D, Sheikh A. Ten key considerations for the successful implementation and adoption of large-scale health information technology. JAMIA. 2013;20:e9–e13 J Am Med Inform Assoc | Published by the BMJ Publishing Group Limited. (Last accessed 28.06.2017)
4. Final assessment and evaluation
Relevant references
CONSORT EHEALTH 2011 checklist EHEALTH checklist (beta –V.1.5) of information to include when reporting an eHealth (web‐based/Internet‐based intervention and decision aids, but also social media, serious games, DVDs etc) or mhealth trials (mobile applications)
Davies, A., & Newman, S. (2011). Evaluating telecare and telehealth interventions. WSDAN Brief Pap Kings Fund, London.
Douma, K. F., Karsenberg, K., Hummel, M. J., Bueno-de-Mesquita, J. M., & Van Harten, W. H. (2007). Methodology of constructive technology
assessment in health care. International journal of technology assessment in health care, 23(2), 162-168.
Eysenbach, G. (2013). CONSORT-EHEALTH: implementation of a checklist for authors and editors to improve reporting of web-based and mobile randomized controlled trials. In MedInfo (pp. 657-661).
Høstgaard, A. M., Bertelsen, P., Petersen, L. S., & Nøhr, C. (2013, August). Constructive Technology Assessment for HIT development: Learning;
feedback and user involvement. In Scandinavian Conference on Health Informatics 2013; Copenhagen; Denmark; August 20; 2013 (No. 091, pp. 33-37). Linköping University Electronic Press.
Kayser L, Furstrand D, Rasmussen E, Jensen L, Monberg A, Karnøe A. (under review) GoTo Navigator -a process navigation tool for digital health solutions to be used to design and align with an efficient trajectory for goal setting to the evaluation of outcomes.
