Study design

We carried out a non-experimental, cross-sectional, instrumental study ²² to develop, validate, and determine the psychometric properties of a PR scale for COVID-19 in the Peruvian population. It should be noted that, at the time of the study, Peru was at the beginning of the fourth wave of the COVID-19 pandemic ²³.

Instrument

The COVID-19 PR Scale (PR-COVID-19-PE) was developed in this study (Supplementary Material 1).

Procedures

The COSMIN Checklist (COnsensus-based Standards for the selection of health status Measurement) (https://www.cosmin.nl/) was used to verify the adequacy of the instrument’s validity analysis (Supplementary Material 2).

Our research was organized in three stages based on the ten steps proposed by Muñiz and Fonseca ²⁴. The first stage was oriented to designing the scale, for which we used an overall theoretical review of PR and a review of COVID-19 PR in order to stablish a theoretical framework and define the variable, its dimensions and indicators. Then, a panel of nine experts (authors of scientific articles on validation of PR scales before COVID-19 in Latin American countries or who conducted studies on COVID-19 prevention in Peru) validated these definitions and provided suggestions that were included in the following stages of the study.

Subsequently, for the construction of the items, we created six focus groups to explore the social representations on the COVID-19 PR held by different segments of the Peruvian population. The focus groups were made up of regular basic education teachers (primary and secondary level), higher education teachers, higher education students, parents and shopkeepers, who gave their informed consent prior to the session. The statements of the focus group participants were transcribed and coded using the Atlas.Ti v.8 program.

Next, we reviewed documents regarding PR scales before COVID-19 that were validated at the moment ^13,15,16,25. Data triangulation was carried out considering the statements of the participants in the focus groups, the indicators developed during the theoretical review and the information from the documents reviewed. As a result, we constructed an initial scale, composed of 26 items, its dimensions and response alternatives.

In the second stage of the study, we used experts’ judgements to estimate the content validity, (two psychometric psychologists, a neuroscientific psychologist and two public health research nurses); the documents containing the theoretical definitions and the proposed PR scale together with its rating scale were sent to the experts. The experts’ ratings were consolidated. Then, by using the Content Validity Coefficient concordance index (CVCic) by Hernández Nieto ²⁶, we evaluated three criteria: coherence, relevance and clarity.

Simultaneously, we carried out a pilot test to evaluate how to present the scale and to detect items that could be difficult to understand, as well as any type of error. The pilot test was conducted on 41 inhabitants of thirteen cities in Peru, who were invited by telephone to fill a virtual formulary designed in Google Forms that was distributed by WhatsApp. Subsequently, they were asked to complete an additional form to report any difficulty in understanding the items and the response alternatives.

Finally, the third stage was aimed at determining the psychometric properties of the instrument. We initially calculated a minimum sample size of 260, assuming the classic criterion of 10 subjects for each variable ²⁷; however, in order to have greater statistical power, we included all 678 participants who answered the survey. The sample was selected by non-random snowball sampling.

Peruvians over 18 years of age were invited to participate between July 5 and 21, 2021. Data collection was carried out with a virtual form designed on the ALLCOUNTED platform (https://www.allcounted.com), which was distributed through social networks and on the website of the National School of Public Health of the Ministry of Health.

Variable

PR to COVID-19 was defined as the subjective assessment of the probability of contagion and severity of the possible consequences of COVID-19 at the individual, family and global levels. This construct is made up of two dimensions, one cognitive and the other emotional. The definitions of the indicators are presented in Supplementary Material 3.

Statistical analysis

We evaluated the factor structure of two dimensions by means of a confirmatory factor analysis (CFA) in order to corroborate the hypothesized factor structure of the scale. An exploratory factor analysis was not carried out because a theoretical structure was defined from the conceptual framework and the qualitative analysis; therefore, the CFA was a better choice ^28,29. The least squares estimator of weighted means and variances was used due to the ordinal nature of the instrument. To identify the absolute fit of the model, we assessed whether the chi-square ratio between the degrees of freedom (X²/ gl) was less than 3. Similarly, to evaluate the relative fit, we assessed whether the Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI) indicators were greater than 0.95, as well as whether the Root Mean Square Error of Approximation (RMSEA) indicator was equal to or less than 0.05 and if the Standardized Root Mean-Square (SRMR) indicator was equal to or less than 0.08 ^30-32.We considered eliminating items that did not have a significant factor load or those with very low levels. Similarly, we performed an invariance analysis according to sex and educational level to compare the configural, metric and scalar invariance. For this criterion, changes less than or equal to 0.01 in CFI ³³, and changes less than or equal to 0.015 in RMSEA ³⁴ were considered to have reached the level of invariance.

Internal consistency was determined with the McDonald omega coefficient (ώ). To verify the criterion validity of the PR-COVID-19-PE, two questionnaires were used as contrast measures. First, it was contrasted with a validated questionnaire that measures PR before the COVID-19 ¹³, which was selected because it uses the theoretical model on which the scale was constructed ¹⁴, however, due to the low reliability (ώ=0.53), item 6 was eliminated as it showed no correlation with the other items of the instrument; thus, achieving an increase in the reliability index (ώ=0.68 of the survey). The second contrast with the COVID-19 fear questionnaire ³⁵ showed high reliability (ώ=0.93). Both instruments were correlated with PR-COVID-19-PE, by using Pearson’s r correlation coefficient. We applied the Little test, which contrasts the null hypothesis of missing values completely at random (MCR), and found insufficient evidence to reject that fact that the data loss was missing completely at random, so we applied multiple imputation with 20 databases and the Predictive Mean Matching (PMM) algorithm. We reviewed quantile-quantile graphs and Shapiro-Wilk tests to determine the level of normality of the variables, and found that the distribution of the data meets sufficient criteria to consider that they come from a normal distribution. Therefore, we used parametric tests to identify the associations of concurrent validity. All statistical analyses were performed in R v 4.2.1 software and its interface R Studio v 2021.0.

Ethical Aspects

The article is derived from the study entitled: “Percepción de riesgo y prácticas de medidas preventivas ante la COVID-19 en docentes de Educación Básica Regular, Perú, 2022: Un estudio mixto” which was approved by the Ethics and Research Committee of the María Auxiliadora University (Act n.° 004-2022) and registered (code EI00000002729) in the Platform for Health Research Projects (PRISA) of the National Institute of Health (INS). The participants gave their informed consent prior to participation All data were anonymized.

Scale validity

The CVCic obtained for each of the items evaluated by expert judgment can be found in Table 1, which shows that most of them obtained values greater than 0.90. Likewise, the CVCic obtained for the coherence, relevance and clarity criteria were greater than 0.90.

During the pilot test, we determined that the average time to apply the questionnaire was 12 min. Of the 41 participants in the pilot test, 34 agreed to complete the second form by means of an interview. At this stage, only 6 of the 34 reported having some difficulty in answering the items.

Finally, all the items that reported CVCic below 0.90 by the expert judgment and those that reported difficulties in the pilot test were evaluated. During this evaluation, we reformulated 9 items (2, 3, 7, 8, 9, 10, 16, 17 and 20), 3 items were changed to another dimension (17, 18 and 19) and 3 items were eliminated (4, 5 and 6); thus, the scale consisted of 23 items.

Confirmatory factor analysis

The virtual survey for the CFA was answered by 780 participants, of which five were excluded for quality control and 97 because they answered only the first part of the questionnaire. Of the total of 678 participants, most were women (67.0%), with an average age of 34.5 years (SD= 12.41). Of the sample, 50.8% reported having at least higher education, 24.3% reported having incomplete higher education, while the rest of the participants reported lower education levels. Regarding previous experience with COVID-19, 59.2% of the participants reported having been previously infected and 2.5% reported having a diagnosis of COVID-19 at the time of responding to the survey. On the other hand, 34.3% of the participants reported having lost a close person or family member to COVID-19.

Initially, when considering all items, we obtained the following fit indicators: X²/gl=3.61, CFI=0.92, TLI= 0.91, RMSEA=0.07 and SRMR=0.09. Then, we detected that the reverse coding item “If I get COVID-19 it will be like a simple flu or something temporary, then I can continue with my life” did not have a significant factor load that would allow an adequate adjustment of the hypothetical model used, so this item was eliminated from the model. Similarly, the residual correlations of the items with similar wording were left free (e.g., items 4, 5, 6 and 7; and items 19, 20, 21, 22). Finally, the fit indicators without considering the reverse coding item were: X²/gl=2.34, CFI=0.96, TLI=0.96, RMSEA=0.05 and SRMR = 0.07. The final factorial model tested is shown in Figure 1, which consisted of 22 items.

Table 2 shows the averages and standard deviations for each of the PR-COVID-19-PE items. Overall, item 17 obtained the highest mean score (M=4.21, SD=0.69), while item 4 obtained the lowest mean score (M=3.15, SD=0.90). Given that all items were measured on a scale of 1 to 5, we found that the averages of all items were higher than 3, this could indicate that the people who participated in the study had a risk perception above the average value of three.

Figure 1 Factor model for the COVID-19 risk perception scale. 

The factor structure resulting from the confirmatory model, the factor loadings and their respective significance are presented in Figure 1. All items of the cognitive dimension of PR-COVID-19-PE have statistically significant factor loadings (p<0.001), except for item 2 (p=0.001) and item 3 (p=0.003). However, it is important to highlight that items 1, 2 and 3 have factor loadings with low effect sizes (0.28, 0.16 and 0.15, respectively). This indicates that the three items have a significant loading, but are not as strongly correlated with the cognitive dimension; however, they were kept because they may explain some characteristics of the cognitive dimension that other items could not. All the other items of this dimension had factor loadings higher than 0.30, which shows that they have good correlation with the cognitive dimension factor. All the items of the emotional dimension have factor loadings greater than 0.30 and are statistically significant (p<0.001). Furthermore, a very strong correlation was found between both dimensions of the scale (r=0.87, p<0.001). Finally, we confirmed that PR-COVID-19-PE has an excellent internal consistency coefficient (ώ=0.88).

Table 3 presents the results of the invariance analyses according to sex and educational level. We compared subjects with less than higher education to subjects with higher educational level. The differences in CFI and RMSEA for the metric and scalar models are smaller than the criteria of ΔCFI≤0.01 and ΔRMSEA≤0.015, indicating that the instrument performs similarly in different demographic groups.

Analysis of criterion validity

Both dimensions of PR-COVID-19-PE and the total score were found to be positively, significantly and strongly related to the Dryhurst PR scale (p<0.001); but were positively, significantly and moderately related to the COVID-19 fear scale (p<0.001) (Table 4).