If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Department of Cellular Biology, Functional Biology and Physical Anthropology, Faculty of Biological Sciences, University of Valencia, Burjassot, 46100, Valencia, Spain
Service of Obstetrics and Gynecology, Hospital Clínico Universitario – INCLIVA, Av. Blasco Ibáñez 17, 46010, Valencia, SpainDepartment of Pediatrics, Obstetrics and Gynecology, University of Valencia and INCLIVA, Av Blasco Ibáñez 15, 46010, Valencia, Spain
The feasibility and performance of telehealth are poorly tested in women's health settings.
•
Telehealth provides an alternative that reduces the risk of infection during the COVID-19 pandemic.
•
Women were less enthusiastic about telephone use during periods of low COVID prevalence compared with high prevalence.
•
In-person visits were preferred by the majority of women attending a women's health clinic.
Abstract
Background
Telehealth has emerged as an alternative to conventional, face-to-face visits, and the COVID pandemic has hastened its introduction. Telephone appointments make use of an easy-to-use and accessible technology.
Aim
To investigate the usability of telephone-based telehealth in a women's health outpatient clinic and whether this may be affected by the severity of the COVID pandemic.
Method
A telephone survey was prepared to explore two usability domains: interaction quality (4 items) and satisfaction, preference and future use (6 items). Women were selected from two periods during the COVID pandemic when the infection rates were high and low.
Results
The survey was completed by 106 women (60 when the prevalence of COVID was high, mean age 53.58 years, and 46 when it was low, mean age 48.59 years) out of the 153 women who had a telephone appointment. The severity of the COVID pandemic showed an effect on responses. Women were less enthusiastic about using the telephone during the period of low COVID prevalence, as shown by lower scores on 3 of the 4 items of the first domain [I had enough time; I would have understood better in person; I would have expressed myself better in person (p < 0.001 for comparison between groups on each of the 3 items)], and on 4 of the 6 items in the second domain [satisfied with quality of care (p < 0.001), or with the information received (p = 0.018); use of telephone in future (p < 0.001); preference to try other technologies in future (p < 0.001)]. Overall, women expressed a preference for in-person visits regardless of COVID prevalence rates.
Conclusion
Telephone calls were a feasible alternative to face-to-face visits in a women's health outpatient clinic, but the pandemic pressure modified usability parameters. Respondents preferred in-person visits at any pandemic stage.
Rapid advances in information and communication technologies (ICT) have resulted in systems which facilitate the virtual exchange of medical information [
]. Telehealth can facilitate patient access to care by circumventing certain obstacles such as the need to travel. Moreover, telehealth implementation is consistent with policies of institutions to extend the digital transformation of health care as a measure to build a healthier society. For example, the European Commission has issued the communication “Transformation of Health and Care in the Digital Single Market”, with the purpose of transforming the way citizens receive health and care services [
]. However, telehealth may also have limitations due to lacking the direct contact that is provided by face-to-face visits. These potential pros and cons may vary from one type of care to the other, and warrant investigation.
The coronavirus disease 2019 (COVID-19) pandemic has made implementing telehealth imperative [
]. The demands of accumulating numbers of infected patients imposed a burden on resources, resulting in reduced attention towards less urgent forms of care. Moreover, policies to reduce disease transmission discouraged visits from less urgent patients, who were advised to stay away from care centers. Given that women's health falls into this less urgent category, care provided in this context has lost priority during the pandemic on a worldwide level, making it a good candidate for implementing telehealth programs [
]; however, there is a lack of data on the subject.
In order to run smoothly, telehealth programs must be feasible, which depends on the complexities of the required care. Moreover, programs must be acceptable to users at both ends [
]. Acceptability is strongly influenced by usability factors, including areas such as comfortability of use and satisfaction.
Our study has two endpoints. First, we explored several patient usability factors of telephone-based virtual appointments in a midlife women's health outpatient clinic in a public health program of integrated care. Specifically, we included two domains: i) interaction quality and ii) satisfaction, preference and future use. Second, we investigated whether limited access to care due to the severity of the COVID-19 pandemic had an effect on the selected usability variables. To this end, women stratified into two groups by attendance during two different severity periods during the COVID-19 pandemic were directly interviewed by two specialized nurses using a structured questionnaire.
2. Material and methods
2.1 Participants and program
The women's health outpatient clinic at our center provides integrated care to midlife women in the health department covered by our hospital, a tertiary-level university center. Women are referred from primary care centers, where specialized nurses (midwives) and general practitioners share an integrated care protocol with the Obstetrics and Gynecology Department in the hospital. The current program includes symptomatic menopausal women, together with other subgroups potentially needing specialized care, including women surviving cancer, with postmenopausal osteoporosis, or with other noncommunicable chronic diseases, most of them women with diabetes.
During the course of the COVID-19 pandemic, our center took steps to transform programmed outpatient appointments into a telehealth format in winter 2021. The program was designed around a two-way audio connection. The decision to replace in-person visits with telehealth appointments was made by the physician in charge of patient schedules after reviewing electronic medical records (EMR) at least one week prior to the programmed appointment. The decision to maintain in-person visits was based primarily on the need to perform exploratory measures, such as ultrasound, etc., or other conditions judged by the physician in charge to require face-to-face attention. A short message service (SMS) was sent out to patients via telephone 2–3 days beforehand informing them of the change to telephone appointment and the approximate time of the call. Telehealth was used successfully to complete some of the outpatient appointments and to speed up patient wait lists, which validated the system as an established alternative of care in our health system.
We selected two different periods of the COVID pandemic in 2021, the first during the peak of the third, most serious pandemic wave, between January 21 and February 19, 2021, and the second, during an inter-peak period with a very low disease prevalence, from October 8 until November 19. Restrictions in attendance were substantially reduced during the second period, so the proportion of in-person visits increased. All women using the telehealth format during those two periods were considered candidates and invited to participate by telephone call.
2.2 Instruments
A survey was modelled on the Telehealth Usability Questionnaire (TUQ), a validated instrument for use in telemedicine [
]. Given that the TUQ is intended for use with various telehealth systems, often more sophisticated than the basic audio-only telephone, items were adapted to adequately assess two domains of applicable usability factors: 1) interaction quality, and 2) satisfaction, preference and future use. The responses were constructed as a 5-point Likert scale.
The interaction quality domain consisted of four items assessing practical conditions related to telehealth format, fluency of communication and duration of the appointment, and two items addressing the quality of bidirectional information transmission. Overall satisfaction and preferences, including the option of wider future use, were the subject of the second domain composed of 6 items.
A number of free text questions were also introduced to enhance understanding of the variables affecting survey outcomes. The list included questions about drug intake (3 or more), comorbidities, and sociodemographic items, including urban or semi-urban background, education level, financial status, and marital status.
The study was approved by the Research Ethics Committee (Approval code 2021/130). During each call, participants were informed about the purpose of the study and their verbal consent was required prior to initiating the interview. At the end of the interview, the answers to all questions were reviewed by the interviewer to obtain definitive confirmation from participants.
2.3 Statistics
Continuous demographic, clinical and laboratory variables were analyzed for normality using the Shapiro-Wilk test. Normally distributed data were reported as mean and standard deviation (SD) and non-normally distributed data were reported as median and interquartile range (IQR). Comparisons between peak and inter-peak periods were performed using t-tests and Mann-Whitney U for normally and non-normally distributed data. Categorical data were reported as number and percentage (%) and comparisons analyzed by Chi-squared (χ2) test.
The “interaction quality” and “satisfaction, preference and future use” domains were evaluated for internal consistency using Cronbach's alpha reliability test score.
Evaluation of time period and each TUQ question was carried out using Chi-squared test. Statistical significance was set at p < 0.05.
3. Results
The items included in each domain are presented in Table 1. Cronbach's alpha for the 4 items included in domain 1 was 0.79 (0.72–0.86). The items 3 (I felt that I had understood the doctor's explanations better in an in-person visit) and 4 (I felt that I had expressed myself better in an in-person visit) were re-ordered because they were presented in reverse order to the others. The Cronbach's alpha for the 6 items included in domain 2 was 0.79 (0.71–0.88).
Table 1Questionnaire items in the two domains: interaction quality (domain 1) and satisfaction, preference and future use (domain 2).
Component
Questionnaire items
Interaction quality
1
I could discuss all issues concerning me with my doctor
2
I had enough time to discuss everything I wanted
3
I felt that I had understood the doctor's explanations better in an in-person visit
4
I felt that I had expressed myself better in an in-person visit
Satisfaction, preference and future use
1
I felt comfortable when speaking with my doctor by telephone
2
I am satisfied with the quality of care received by telephone
3
I am satisfied with the information received by telephone
4
I prefer the telephone to in-person visits
5
I would like to use telephone appointments in the future
Details of the enrolment process presented in Fig. 1 show that 369 women were attended in the selected periods, 141 from the schedule corresponding to 21 January till 19 February (COVID-high), and 228 women corresponding to 8 October till 19 November 2021 (COVID-low). There were 92 (65.3 %) telehealth appointments in the first period and 61 (26.8 %) in the second period, of which 60 women responded to the questionnaire in the first period and 46 women in the second.
Fig. 1Flowchart of patient selection at both the peak and inter-peak COVID periods.
No prior sample size was calculated due to the exploratory nature of the study. Since 106 out of the 153 women attended by telephone fully completed the survey, the margin of error was 5.29 % assuming a 95 % confidence level.
Table 2 describes the clinical and sociodemographic characteristics of participants in each group. Both groups were composed of midlife women, with a mean age of 53.58 years in the COVID-high and 48.59 in the COVID-low group. The groups were comparable, but differed regarding economic status, with a higher representation of women of low financial means in the COVID-low group. Multimorbidity data was retrieved from EMR and verbally confirmed with patients during interviews. A total of 26 women (43.33 %) met multimorbidity criteria, defined as existence of two or more disorders, in the first period, and 20 women (43.48 %) in the second period. Educational level, as well as marital and financial status, were also similar between the two groups.
Table 2Participants' demographic and clinical characteristics.
COVID-HIGH
COVID-LOW
Differences
(n = 60)
(n = 46)
P value
Age (years)
53.58 (12.58)
48.59 (11.23)
F = 1.41; P = 0.245
Education level
None
18 (30.00)
12 (26.09)
n (%)
Primary
6 (10.00)
4 (8.69)
χ2 = 2.15; P = 0.905
Secondary
22 (36.67)
18 (39.13)
University
14 (23.33)
12 (26.09)
Marital status
Married/couple
25(41.67)
15 (32.61)
χ2 = 9.21; P = 0.162
n (%)
Separated
9 (15.00)
8 (17.39)
Single
22 (36.66)
20 (43.48)
Widow
4 (6.67)
3 (6.52)
Economic status
Low
5 (8.33)
14 (30.44)
χ2 = 53.12; P < 0.001
n (%)
Medium
30 (50.00)
16 (34.78)
High
25 (41.67)
16 (34.78)
Comorbidities
0 disease
18 (30.00)
12 (26.09)
χ2 = 9.30; P = 0.054
n (%)
1 disease
16 (26.67)
14 (30.43)
2 or more
26 (43.33)
20 (43.48)
Data are expressed as mean (standard deviation) or otherwise indicated.
Fig. 2 presents the results for items addressing the quality of interaction at each COVID-19 period. The data were mixed, since there was a similar result (women scoring 4 or 5) in the questions about ability to address all issues of concern (68.3 % vs 65.2 %), but the feeling of being short of time (56.6 % vs 30.4 %), a poorer understanding of physician explanations (15.0 % vs 56.5 %), and satisfactory expression of their own needs (5.0 % vs 43.4 %) were dramatically different during the second, less intense pandemic period.
Fig. 2Responses on the Likert scale for each of the 4 items related tfo quality of interaction (domain 1). The Y axis denotes the percentage of women who gave a valid answer to the survey. 1 = strongly disagree. 2 = disagree. 3 = neutral. 4 = agree. 5 = strongly agree.
The impact of COVID severity also emerged when analyzing answers given for several specific items: 2 (I had enough time), 3 (I had understood better in person), and 4 (I had expressed myself better in person) of the first domain, which showed between-group differences (p < 0.001 for each of the 3 items).
Fig. 3 presents the data obtained for items of domain 2: satisfaction, preference and future use. The results were also mixed, since there were similar results across periods (scores of 4 or 5) in comfortability (76.7 % vs. 63.0 %), but more dissimilarity was found in satisfaction with the information received (71.7 % vs. 47.8 %) and with the perceived quality of care received (80.0 % vs. 49.99 %).
Fig. 3Responses on the Likert scale for each of the 6 items related to satisfaction, preference and future use (domain 2). The Y axis denotes the percentage of women who gave a valid answer to the survey. 1 = strongly disagree. 2 = disagree. 3 = neutral. 4 = agree. 5 = strongly agree.
There were no differences as regards preferences, which seemed to be unanimously in favor of the in-person modality. The three items addressing the issue obtained similar results in both periods, with most women giving only 1 or 2 points in the Likert scale. The analysis of the impact of each time period in the profile of the answers obtained for each item confirmed that the two groups differed significantly in 4 out of the 6 items. This was the case of item 2 (satisfied with quality of care, p < 0.001), item 3 (satisfied with information, p = 0.018), item 5 (use of telephone appointments in the future, p < 0.001), and item 6 (preference to try other technological options in the future, p < 0.001).
4. Discussion
The need to limit face-to-face appointments, particularly during the pandemic, has increased interest in telehealth. Our study is of particular interest because we provide information about the usability of an audio-based telephone telehealth system in the particular context of women attending a midlife integrated care system. We explored two usability domains which are strong determinants of the success and long-term use of telehealth [
]. Moreover, we investigated two different cohorts to clarify how the usability parameters may be affected by external constraints, which in our particular case was represented by the severity of the COVID-19 pandemic. As the main finding, the overall results of our study support that the system performed acceptably in both COVID environments. As with any telehealth system, the advantages of the protocol include higher safety for end-users, who avoid risk of exposure during times of pandemic, and a likely cost-effectiveness, since the expenses of travelling or time lost at work were not incurred [
]. Moreover, unsurprisingly, there were no difficulties concerning access or familiarity with technological devices, as the telephone is routinely used and accessible in most environments. This was confirmed for both periods, as shown by the high numbers of women giving scores of 5 in the first item in domain 2 about comfortability with the system (Fig. 3).
More potential difficulties were found in the pivotal area of interaction quality. Under the severe conditions imposed by the peak of a pandemic wave, we found that interaction quality was acceptable (Fig. 2). A high proportion of women acknowledged that the system offered a good chance to communicate with the physician, giving scores of 4 or 5 in questions about whether they could discuss all their points of concern (68.3 %), or had enough time to do so (56.6 %). Moreover, the proportion of those scoring 4 or 5 when enquired about whether they had understood better the explanations of the doctor or had expressed themselves better if the visit had been in-person decreased, and the most frequent choice was neutrality. These answers were different, however, under the less urgent conditions of low COVID infection rates in 3 out of 4 items of the domain (items 2–4). Indeed, while the proportion of women allocating scores of 4 or 5 remained stable for the first item (65.2 %), it decreased for the second item (30.4 %) in which the availability of enough time to communicate was enquired. Furthermore, the proportion of women giving high scores on the item of whether they had understood better the explanations of the physician, or had expressed themselves better should the visit have been in person increased to 56.5 % and 43.5 %, respectively.
Analysis of the second domain, including satisfaction, preference and future use, also revealed the major influence of COVID pressure, with differences in answers attaining statistical significance in 4 out of 6 items (Fig. 3). In summary, apart from the aforementioned comfortability with the system which was consistently high, women were less satisfied with the quality of care and information provided by telephone during the inter-peak interval, and agreed with those surveyed during COVID-high in their aversion to continue using the telephone or trying other technological options in the future. Therefore, although a reasonable proportion of women accepted that the telephone addressed the main requirements of a medical appointment, in-person visits were seemingly preferred in both COVID circumstances. Furthermore, enthusiasm for virtual appointments apparently decreased in some specific items when accessibility to standard in-person care was normalized during the inter-pandemic interval. Women still accepted the comfortability and versatility of the system to address any topic of interest, both features scoring quite similarly between groups, but some interaction difficulties were also reported, such as feeling that the appointment was too short, and difficulties in communicating, including some trouble understanding the explanations of the physician or expressing their own needs. This could partly explain the reduced scores in satisfaction with both the information and the quality of care received.
Around 20–30 % of women gave high scores in the two items addressing preference for the telephone or for testing other technological alternatives in the future (Fig. 3). This was consistent despite COVID status. It seems, therefore, that the advantages provided by the telephone appointment were recognized by some women even in conditions where the in-person alternative was clearly accessible.
It is unclear whether preferences were mediated by patient characteristics: the groups consisted of women with a mean age around 50 years and with a high multimorbidity burden (43 %) compared with the population standards for their age [
]. Perhaps a younger group might have shown a more positive attitude to telehealth. However, a facilitator to the telephone in this group may have been the particular conditions imposed by the pandemic, even during the period of low infection rates, since safety was an important issue, and is protected by telehealth, as has been observed in other studies [
]. We cannot rule out that the dissimilar distribution of home income might have had an influence. Women in the covid-high group accumulated in the medium and high income level, while low-income was more prevalent in the covid-low group. Although sparsely investigated, it may be hypothesized that the acceptability of telehealth might be influenced, by social class. This has been reported, but it applies to more sophisticated technology and not when using the standard telephone [
Disparities in the uptake of telemedicine during the COVID-19 surge in a multidisciplinary head and neck cancer population by patient demographic characteristics and socioeconomic status.
One of the main strengths of our study is that the women's sociodemographic conditions are expected to have a limited impact on the results, because both groups were comparable in education level, marital status and co-morbidities, although women with low financial status were overrepresented in the COVID-low group. Another strength lies in the simplicity of the technology, the traditional telephone, which is widely accessible and provides equality of access to the service, and ease of use of technological devices has been highlighted as a component of satisfaction with telehealth [
M. Nguyen M. Waller A. Pandya et al A review of patient and provider satisfaction with Telemedicine. Curr. Allergy Asthma Rep. 20 72. doi:10.1007/s11882-020-00969-7.
]. Finally, the survey was performed by specialized health professionals, a more reliable procedure than alternatives such as internet-based surveys, a frequent option in some literature on this issue.
Certain limitations of the study should also be taken into account. The sample size may limit the scalability of the conclusions in terms of broader policy decisions. Another drawback was the number of women who did not reply or declined to participate (34.8 % in COVID-high and 24.6 % in COVID-low). This response failure rate is low compared to other studies [
], but it could be speculated that this group would be particularly unhappy with telephone appointments, which could potentially alter the results.
In conclusion, a bilateral audio telephone-based telehealth system was feasible in a women's health outpatient clinic and gave acceptable results in two different usability domains: interaction quality and the conglomerate of satisfaction, preference and future use. However, several items were sensitive to the degree of severity of the COVID-19 pandemic. Moreover, a preference for in-person visits was expressed regardless of the pressure of the COVID-19 conditions.
Contributors
Alicia García-Vigara participated in the collection of the data, the building of the database, and the literature search, and contributed to writing of the manuscript.
Víctor Martín-González participated in the analysis of the data and in the interpretation of the results, and added intellectual content by participating in the discussion of the manuscript.
Juan-Antonio Carbonell designed and completed the analysis of data, and participated in the interpretation of the results.
Celia Bauset-Castelló participated in the analysis of the data and in the interpretation of the results, and added intellectual content by participating in the discussion of the manuscript.
Ana Martínez-Aspas participated in the analysis of the data and in the interpretation of the results, and added intellectual content by participating in the discussion of the manuscript.
Aitana Monllor-Tormos participated in the analysis of the data and in the interpretation of the results, and added intellectual content by participating in the discussion of the manuscript.
Miguel-Ángel García-Pérez participated in the analysis of the data and in the interpretation of the results, and added intellectual content by participating in the discussion of the manuscript.
Juan J. Tarín participated in the analysis of the data and in the interpretation of the results, and added intellectual content by participating in the discussion of the manuscript.
Antonio Cano participated in the design of the study, the conception of the manuscript, in the literature search, was involved in the analysis of the paper's content, and wrote the draft of the manuscript.
All authors approved the version to be published.
Funding
Miss Alicia García-Vigara received funding from Instituto de Investigación Sanitaria-INCLIVA, Valencia, Spain.
Ethical approval
IRB (Comisión de Ensayos e Investigación Clínica- CEIM) Hospital Clínico Universitario-INCLIVA, Valencia, Spain.
Provenance and peer review
This article was not commissioned and was externally peer reviewed.
Research data (data sharing and collaboration)
There are no linked research data sets for this paper. Data will be made available on request.
Declaration of competing interest
The authors declare that they have no competing interest.
References
Cano A.
Traver V.
The health challenges at menopause and midlife: sustainability, ICT technology and patient empowerment.
Disparities in the uptake of telemedicine during the COVID-19 surge in a multidisciplinary head and neck cancer population by patient demographic characteristics and socioeconomic status.
M. Nguyen M. Waller A. Pandya et al A review of patient and provider satisfaction with Telemedicine. Curr. Allergy Asthma Rep. 20 72. doi:10.1007/s11882-020-00969-7.
00234-1&id=gr1.jpg){kind=link}
00234-1&id=gr2.jpg){kind=link}
00234-1&id=gr3.jpg){kind=link}