TELEMEDICINE REPORT TO CONGRESS

January 31, 1997


EVALUATION

A. OVERVIEW

The advance of telemedicine technology has created a host of relevant new clinical and educational applications. From video conferencing to store-and-forward capability, health care professionals are finding many uses for this technology. The next challenge lies in answering critical technical, clinical, and organizational questions about what works and what does not.

The need for more evaluation of this technology has been a pressing concern for the Joint Working Group on Telemedicine. A recent report from the Institute of Medicine (IOM) and a soon-to-be released General Accounting Office (GAO) report have stressed the importance of the JWGT as a structure for coordinating Federal programs and, more specifically, creating a process of sharing evaluative information across agencies.

This chapter will provide an overview of collaborative and individual efforts of the JWGT-participating agencies. These activities include the Federal Telemedicine Gateway, the Joint Evaluation Framework, the Institute of Medicine report, and the Office of Rural Health Policy/Abt survey of rural telemedicine. The chapter will also outline what is known about the use of telemedicine and provide a list of lessons learned from the various Federal projects and initiatives.

B. THE FEDERAL TELEMEDICINE GATEWAY AND INVENTORY OF HEALTHCARE DELIVERY PROJECTS

Despite the growing interest in telemedicine, and the need for "hard data," little is currently known about the extent of telemedicine in the United States. To address this need for more information, the JWGT is examining the use of World Wide Web technologies to create an inventory of Federal telemedicine activities. The intent of this initiative is to simplify the collection of and timely access to data on Federal telemedicine activities. The development of software and procedures to ease maintenance of data and ensure security and privacy is currently underway. These developments will enable individual agencies to maintain and update their own project information in a quick and cost-effective manner.

Using a Department of Defense contractor and technical assistance from the Departments of Defense (DoD), Health and Human Services (HHS), Veterans Affairs (VA), Agriculture (USDA), and the National Aeronautics and Space Administration (NASA), the JWGT created an inventory and data base that eventually will be available to the public on a web site. The Federal Telemedicine Gateway web site includes an inventory of federally-funded projects as well as linkages to other web sites that contain information on telemedicine activities in the private sector and states.

The inventory includes information from demonstration grant projects from the Department of Commerce, the Department of Health and Human Services, the Appalachian Regional Commission and the Department of Agriculture. It also includes direct provision of health care services through the Indian Health Service, the National Aeronautics and Space Administration, the Department of Defense, the Department of Veterans Affairs, and the Department of Justice.

At the most fundamental level, the inventory contains a data base that includes basic descriptive information on each Federal project, including names of individuals to be contacted for further information. Wherever possible, available information is organized in a uniform format, allowing analyses across projects. The data base is incorporated into a geographic information system that facilitates linkage of information in the inventory to other basic data about communities in which telemedicine projects are implemented. Basic analytic reports are to be developed from the information maintained in the inventory.

C. SUMMARY OF THE GATEWAY AND INVENTORY

The inventory and Gateway can and should be viewed as a prototype. In particular, once developed, additional data, such as those on distance learning or public health applications of telecommunications, can be added to the inventory. Several states have expressed interest in being linked to the inventory once it is developed. Moreover, the basic design of the inventory could be very useful to other initiatives that need to be tracked by the Federal government, e.g., Medicaid or welfare reform demonstrations.

Currently, the inventory is undergoing technical evaluation. Initial evaluation has revealed some of the limitations of the web technology available today. It also has shown the weaknesses in the information reported and maintained by Federal agencies, highlighting the need for greater attention to routine data collection on federally-funded programs. Release of the complete inventory is anticipated by the end of February 1997, assuming that all technical problems can be addressed by the contractor by mid-January.

Access to the Gateway is currently available through the World Wide Web (WWW), with levels of access assigned according to organizational affiliation and project participation. Access to the database is available at http://www.tmgateway.org.

D. THE FEDERAL EVALUATION FRAMEWORK

The absence of evaluative information continues to be a hurdle to a wider use of telemedicine. Policy makers at both the Federal and state levels may find it hard to answer critical questions regarding cost, infrastructure, quality, and effectiveness without more sound and thorough information about the use of telemedicine to improve health care delivery. However, there are efforts underway to address these concerns. To facilitate coordination among Federal agencies and with the states and private sector, the JWGT has developed a framework(1) for evaluating telemedicine projects. The framework (outlined in Box 2 ) represents a blueprint for sharing information across federally-funded projects and studies. It provides examples of the questions that need to be asked dfor six major areas or domains of concern. These questions are generic and could apply equally to Federal and non-Federal projects and studies.If successful, the framework also should facilitate cooperative evaluation efforts with private-sector telemedicine projects.(2)

E. OTHER EVALUATION ACTIVITIES BY JWGT AGENCIES

ORHP has funded a cooperative agreement with the Telemedicine Research Center to develop uniform data collection instruments and collect uniform evaluation data in the 20 networks funded by ORHP. HCFA is funding an evaluation of health systems utilization and cost-effectiveness with telemedicine. The two agencies also are pursuing development of common evaluation instruments. If successful, these instruments would allow for comparison of information across the two agencies, thus increasing the amount of data available and strengthening each agency's ongoing evaluation efforts.

At the Departement of Commerce, NTIA's Telecommunications and Information Infrastructure Assistance Program requires each of its grantees to conduct evaluations of their telemedicine projects. In addition, NTIA recently released Lessons Learned from the Telecommunications and Information Infrastructure Assistance Program, a report based on the experiences of its grantees with a variety of information infrastructure projects in health, education, and other social services.

The Department of Defense has embarked on a comprehensive evaluation of telemedicine within some of its demonstration projects and the use of telemedicine in support of deployed troops (e.g., in Bosnia, Haiti, and Somalia). The DoD evaluations include the development of outcomes measures to assess the clinical efficacy of telemedicine. The NLM has funded contracts to evaluate the information content necessary to make good clinical decisions using various telemedicine technologies. The Agency for Health Care, Policy & Research (AHCPR) has awarded one grant and may award additional grants to examine the cost and medical effectiveness of telemedicine. The results of these studies, and others, will help answer critical questions about what works and what does not in telemedicine.

F. THE INSTITUTE OF MEDICINE REPORT ON TELEMEDICINE EVALUATION

The Institute of Medicine (IOM) report, funded by the NLM and other Federal agencies, provides one of the more comprehensive assessments of telemedicine evaluation to date. This report provides further refinement of the JWGT framework and outlines many of the next steps to be taken in evaluation as shown in Box 3. The IOM's committee evaluation framework includes four components: principles, planning processes, evaluation elements, and evaluation questions. These principles call for the evaluation to be:

G. THE OFFICE OF RURAL HEALTH POLICY/ABT SURVEY OF RURAL TELEMEDICINE

In January, 1997, ORHP released the first major national survey of rural telemedicine. "The Exploratory Evaluation of Rural Applications of Telemedicine", conducted by Abt Associates, Inc., which includes information about the extent to which telemedicine is used in rural areas, by whom, for what purposes, and the costs.

Key Findings

By the beginning of 1997, nearly 30% of the 159 rural hospitals surveyed in the winter of 1996 are expected to be using some sort of telemedicine technology to deliver patient care. Of these, 68% are expected to offer only teleradiology.













More than 40% of the telemedicine programs surveyed had been providing teleconsults for one year or less.

Radiology and cardiology were the most common clinical applications reported, followed by orthopedics, dermatology and psychiatry.

Telemedicine systems were also used for non-clinical applications such as continuing education for health professionals. Fifty-eight percent of the sample had used their equipment for four or more different non-clinical uses.

Despite growth and expansion, the cost of telemedicine remained high. The average equipment purchase, excluding switches and new lines ranged from $134,378 for "spoke" sites to $287,503 for "hub" sites.(3) Reported annual transmission costs were also high, ranging from an average of $18,573 for spokes to $80,068 for hubs. Slightly less than 20 percent spent less than $50 per session in transmission costs, while seven percent spent at least $500 per session. High costs, combined with low utilization in the early years of operation, yielded high unit costs. A teleconsult cost the median or typical hub site $1,181, while the median spoke site spent $476 per consult, exclusive of any reimbursement to clinicians.

Federal and state grants were common sources of direct funding for telemedicine programs, and the majority of sites also received hospital financial support. Third-party reimbursement for telemedicine was elusive; fewer than 25% of hub facilities had successfully negotiated payment with insurance carriers and many had not yet undertaken such negotiations.

The most common transmission technologies involved copper telephone lines (78 percent of Telemedicine facilities and 83 percent of Teleradiology Only facilities), and dedicated telecommunication services such as T1 (76 and 29 percent of Telemedicine and Teleradiology Only facilities respectively). Fiber optic lines were also commonly reported (52 percent of Telemedicine facilities) as were switched services such as switched 56 Kbps and ISDN. Satellite or microwave transmission were each mentioned by less than 10 percent of respondents. Thirty eight percent of responding sites reported availability of not only a dedicated service, but also a switched service.

The majority of telemedicine facilities reported the use of real time technologies for the transmission of data and images (90 percent). Two-thirds had store-and-forward technologies available, and most had both.

Utilization was low in the first years of most rural telemedicine programs. Only 17 percent used their system more than once each day.

Sixty percent of facilities that had been operating between one and two years had a narrow range of clinical applications whereas those operating for two or more years were more likely to have a broader set of applications (62 percent).

H. LESSONS LEARNED

In addition to the information provided by the IOM and ORHP/Abt studies concerning telemedicine trends, there have been important lessons learned on a project by project basis from the experience of federally-funded telemedicine projects.

The majority of federally-funded telemedicine projects are currently in the early stages of development. Most have been in existence for only three years or less. Thus, similar to any startup company, these projects have faced steep learning curves. Despite this relatively short history, project developers have gleaned some early lessons from the pre-planning, startup, and sustainable phases of the telemedicine projects' development.

Pre-planning Phase

Pre-planning is probably the most important phase in the development of a telemedicine system. In this phase, the telemedicine planners must identify their client's needs, the scope of their market, the type of technology that fits the market, and the type of infrastructure that will meet their needs.

Develop a business plan. Among Federal grantees, those organizations that developed a solid business plan were more successful than those that did not. In the business plan, it is important to anticipate and plan for sustainable service once subsidies are gone. In addition, a business plan should include:

A thorough needs assessment. Find out about the community's clinical and educational needs. Canvass users from the beginning. User input is critical to the success of the project, helping to avoid mismatches between equipment and needs. Early user input is critical not only in determining the type of technology used but also in the design of the system.

Technology matched to clinical needs. Clinical needs, not technology, should drive the development of a telemedicine system. Successful projects used a thorough technology assessment to select the most simple and least expensive equipment to meet their clinical requirements.

A clear understanding of the existing telecom delivery system. The goals and objectives within that delivery system need to be understood and incorporated into the system and organizational design. System designers must begin with a careful analysis of the present telecommunications infrastructure and build upon it. They must consider modes of transmission, such as copper, fiber optics, satellite, and cable. Types of technology such as PC-based vs. dedicated teleconference systems stems must be evaluated in terms of the benefits vs. cost.

Flexibility. The rapidly changing nature of telemedicine technology puts a premium on creating flexible systems that can adapt to new equipment. Designers of telemedicine delivery must constantly strive to develop systems that are readily adaptable to improvements in technology as well as reductions in cost. To the extent possible they should incorporate off-the-shelf equipment that does not require a great deal of customizing. Customizing can be expensive and may demand greater technical expertise in geographic areas where such expertise is not readily available.

Simplicity--"Keep it simple". This is a principle familiar to those serving in the armed services (See Box 5). The choices in telecommunications and computer based technology, coupled with the variety of medical applications possible, creates an overwhelming array of options when developing telemedicine systems. Experience leads us to believe that the most successful systems are those that use the simplest and least expensive technology to meet a need.

Human factors. The design of any telemedicine system must be viewed as useful to the practitioners. Physicians and other health care professionals have difficulty adapting to technology that does not meet their needs. The technology must fit within the scope of practice.

Negotiating telecommunication costs. Successful projects quickly learned to negotiate and work with telecommunications vendors and telephone companies to keep transmission costs low. This was particularly important in rural areas, where the telecommunications infrastructure and resulting rate structure vary greatly. In some areas, telephone lines may have to be installed or connected before they will support telemedicine.

Get a price for transmission costs as early as possible, negotiate lower rates, or consider using a lower bandwidth telemedicine system if it provides adequate resolution and transmission speed. Be as creative as possible in structuring line connections to cross as few Local Access and Transport Area (LATA)(4) boundaries in order to keep transmission costs down. Work with the State Public Utility Commission to get the lowest possible rates.

Startup Phase

Identify and support a champion. Effective leadership is the cornerstone of any telemedicine system (See Box 6). Those projects that supported a champion were more likely to be successful than those that did not. It was also important to target leaders from within the community's power structure who were able to dictate action and commit resources--both financial and human. Minimally, there should be clinical leadership because it is practitioners who drive telemedicine use.

Communicate a common vision. As in any successful business, a common vision of goals, priorities and needs must be communicated to all levels of the team--from the leaders to the practitioners to the technical support.

Start evaluation right away. It is important to collect data right away. This will be valuable in your negotiations with insurers for reimbursement. They will base any decision on data relating to quality, access, and costs.

Foster multiple uses. Successful networks do more than just clinical consultations. In the early stages, the clinical consult figures tend to be very low. Without multiple uses, it is hard for a network to become self-sustaining, quickly. Making the network available to other groups for administrative meetings, continuing education, degree programs, grand rounds, and community meetings can help lead to long-term viability. (See Box 7 )

Sustainable Phase.

Maintain training and on-going technical assistance. Securing a good source of technical assistance is important from the "get go" of the project. Often, the only readily available source of technical advice may be vendors. Thus, network participants must develop a critical mass of technical expertise at both the receiving and sending sites. This responsibility cannot be vested in one individual per location, but rather enough to cover all the hours that the network is utilized. Training on this equipment should also be extended to the health professions so that providers will be familiar and comfortable with this technology as they move from training to practice.

Standards and protocols should work together: In order for the various telemedicine equipment and software to work together, standards and protocols must be established and must work together. Standards have been critical to the success of Columbia Presbyterian Medical Center's tuberculosis project (see Box 9). Information exchange, privacy and security were all dependent upon the existence of consistent standards among the participants. For example, the three organizations had to map data elements from one organization to the other. By choosing the Health Level 7 standard as the data messaging standard and TCP/IP as the data transmission standard, this process was helped greatly. Moreover the Center found that "without some kind of agreement or standard policy, privacy and security levels may differ greatly and preclude necessary information and data exchange."(5) The center also attributed its success to the fact that this project was designed from the bottom up. Thus, the people facing a particular problem came up with solutions to the problem and were committed to the success of the project. The champions were close to the problem and they created a solution that was tightly focused.

I. NEXT STEPS

Detailed studies, separate from the demonstrations, are underway to begin to collect better clinical and cost data from the various agency demonstrations. Results from these studies, however, will take time. Current projects are just beginning to emerge from the proof of concept stage and it will take time to gather a critical volume of experience and data to answer many of the questions posed regarding telemedicine. For this reason, the agencies have been interested in developing common data collection tools that will permit aggregation of data across projects, whenever possible. Some of the next steps for evaluation include:

Uniform Evaluation Tool Development. As noted above, ORHP has awarded a grant to the Telemedicine Research Center to develop and evaluate instruments for creating a common data set for application across all agencies. Emphasis will be on collecting common data elements on clinical encounters, costs, and the structure of telemedicine provider organizations, whenever possible. The instruments will be first used in ORHP's 20 demonstration projects. The ORHP efforts will be closely coordinated with those of NLM and HCFA, the latter has awarded a major contract for an evaluation of telemedicine specifically focused on information required to develop payment policies. (See the discussion on HCFA payment demonstration.)

NTIA and the Rural Utilities Service are also coordinating their evaluation efforts with other agencies through the JWGT. It is hoped that this effort will result in data collection tools that would allow for analyses of data across projects.

Evaluation of Medicaid Telemedicine Programs. Currently, 10 states offer some telemedicine coverage under their Medicaid programs, but there is no general evaluation effort or opportunity for them to share their experiences, successes, and failures. The JWGT will assess the current level of evaluation activity in the Medicaid program and work with selected states to promote better evaluation. In particular, the JWGT will work with HCFA and ORHP to develop strategies for using state offices of rural health and HCFA regional offices to develop an ongoing mechanism to track Medicaid activities.

Evaluation of Telemedicine in Managed Care Settings. Evaluation studies of telemedicine need to be expanded to managed care settings, and in particular, to rural managed care settings. Currently, there is very little penetration of managed care in rural settings, but several managed care plans, most notably in Minnesota and California, believe that telemedicine might provide a more cost-effective way for the plans to reach rural communities with needed services. Questions remain as to whether telemedicine technologies will be beneficial to rural communities in the long run through the provision of specialty care that would otherwise not be available, or will they result in reduced access and availability of care because specialists are no longer visiting these communities to provide care? HCFA is discussing the use of telemedicine with several large Medicare risk-based managed care plans. ORHP has had a number of inquiries from managed care plans that would like to be able to apply for ORHP grants this year to evaluate telemedicine programs.

Quality and Efficacy of Care. Very little current research is systematically evaluating quality and efficacy of telemedicine services. This is a very complicated area of research that needs to be pursued over the next two years, if this nation is to be a credible leader in telemedicine. DoD and the Agency for Health Care Policy and Research (AHCPR) are working together to develop strategies for assessing the clinical efficacy of telemedicine for specific specialty applications. This work may serve as a foundation for other agencies to build upon. State-funded initiatives are also beginning to establish research and demonstration efforts to assess the clinical efficacy of telemedicine services provided in different specialties. The JWGT plans to actively broker partnerships between these state and Federal efforts.

Evaluation of Telemedicine in Post-Acute Care (Home and Long-Term Care) and in Non-Health Care Settings. Based on phone inquiries from the health care industry, this is clearly a priority area. One study in Ohio suggests that over 30 percent of emergency hospital readmissions of Medicaid patients from nursing homes might be prevented by timely teleconsulting triage with the patient's primary care practitioners. The JWGT will work with agencies currently funding projects in post-acute and non-health care settings to develop standard evaluation tools.


FOOTNOTES

1. Puskin, D. Brink, L. Mintzer, C. Wasem, C. (1995). Joint Federal Initiative for Creating a Telemedicine Evaluation Framework. The Telemedicine Journal. 1 (4).393-397.

2. Ibid.

3. "Hub" facilities are those providing health care services whereas "spoke" facilities are those receiving the services. Typically, a specialist at a hub facility could serve patients at different long distance spoke sites.

4. Local telephone service areas created by the divestiture of AT&T into the seven Regional Bell Operating Companies. Telecommunications crossing LATA borders are typically higher in cost .

5. From the quarterly evaluation of the Columbia Presbyterian Medical Center, NTIA, Department of Commerce, 1995 .