Volume 17, Issue 1 (2020)
ioh 2020, 17(1): 1098-1108 |
Back to browse issues page
Research code: 2968
Ethics code: IR.IUMS.FMD.REC 1396.9511139001
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Jafari H, Vosoughi S, Abolghasemi J, Ebrahimi H. Development and validation Checklist for the Health, Safety and Environment Management System Auditing in the Surface Mines. ioh 2020; 17 (1) :1098-1108
URL: http://ioh.iums.ac.ir/article-1-2959-en.html
URL: http://ioh.iums.ac.ir/article-1-2959-en.html
Iran university of medical sciences , hossein.ebrahimi@yahoo.com
Abstract: (2139 Views)
Background and aims: Mining has always been recognized as one of the most hazardous occupations, and mining environments have been discussed as one of the most dangerous work environments. Thus, occupational safety and health is a major issue for the 13 million people or miners in the world, many of whom are working in surface mines.
Considering the slogan "Healthy humans are at the center of sustainable development" and given the need for maintaining the health of the workforce and protecting the environment, HSE management systems have currently been extended to most companies, organizations, industries, and mines to ensure workforce health and environmental protection. On the other hand, any organization and industry needs monitoring and auditing to determine the status and effectiveness of its HSE management system in order to identify and improve the strengths, weaknesses, and non-compliances of the system. So far, no comprehensive and coherent tool or mechanism has been provided for monitoring and auditing HSE in surface mines.
HSE audit is a systematic process used to assess the compliance with the guidelines, regulations, rules, and standard executive procedures developed in the field of HSE. The existence of a comprehensive HSE audit system makes it possible to identify and control all system defects before they may cause irreparable damage. In addition, regarding the importance of HSE in mines and despite the existence of the ISRS[1] for safety auditing in mines, as this method only deals with safety and does not address health and environment as well as the main issues on mining, and due to the lack of a comprehensive and specific tool in Iran for auditing health, safety and environment, the present study was conducted in 2019 to provide a valid audit tool for HSE management system for surface mines.
Methods: This analytical cross-sectional study was conducted in 3 stages:
A: Designing a preliminary checklist based on the HSE management system dimensions: Having reviewed the texts, standards, and regulations related to the health, safety and environment of the mines and also the related audit methods, the researcher developed the items for each dimension.
B: Checklist validity: Given that this study aimed to design a specific checklist for mines, face and content validity was evaluated. The convenience sampling method was used to ensure face validity, and 15 HSE experts (working in surface mines) were selected and invited to cooperate. Finally, according to the expertschr('39') opinions, the items were reviewed and corrected and a new draft was prepared.
To evaluate the content validity, qualitative and quantitative methods were used based on the Lawshe model. In a qualitative assessment of content validity, 15 HSE mining experts with work, research, and educational experiences were invited to participate as a panel group. They were then asked to read the checklist carefully and consider the issues such as the use of appropriate words, grammar, placement of items in their right place, and proper scoring, and to submit their corrective views in writing. Once the groupchr('39')s comments were collected, necessary changes were applied to the checklist.
To quantitatively examine the content validity, two indicators were used (content validity ratio and content validity index). Thus, the CVR[2] and CVI[3] were respectively used to ensure the selection of the most important and correct content and to ensure that the checklist items were best designed to measure the intended content.
C: Checklist reliability: Reliability shows how the measurement tool gives the same results under the same conditions. To determine the external reliability, the checklist was completed by HSE experts in 20 surface mines and was assessed using the test-retest method and correlation coefficient at a two-week interval.
Furthermore, the interclass correlation coefficient (ICC) was used to calculate the internal reliability (agreement between evaluators) of the checklist items. To this end, the checklist was completed by three auditors in 8 surface mines, and the results were analyzed using the ICC method.
Results: The results of this research are presented in the following sections: determining the items and designing the preliminary checklist, and evaluating the validity and reliability of the checklist.
A: Determining the items and designing the preliminary checklist: Having reviewed various texts and considering the opinions of the research team and experts, the researcher used the HSE management system dimensions to design the audit checklist of the HSE management system of surface mines. The dimensions included leadership and commitment; policy and strategic goals; organization, resources, and documentation; risk assessment and management; planning, implementation, and monitoring; and audit and review. In the next step, the items were designed for each dimension.
B: Checklist Validity: 1- Face Validity: Once the experts examined the sentences and items of the checklist in terms of appearance, word choice, and comprehensibility, necessary changes were applied. 2- Qualitative evaluation of content validity: After reviewing the checklist in terms of grammar, insertion of items in their right place, and proper scoring by the panel group (15 experts), the expertschr('39') opinions were collected and necessary changes were applied to the checklist. 3- Quantitative evaluation of content validity: The results of examining the content validity ratio (CVR) showed that 27 items of the checklist had a content validity ratio of < 0.49, and were removed from the audit checklist. Also, examining the content validity index showed that 191 items received a CVI score of >0.79, two items had a score of <0.70 and were removed from the audit checklist, and 3 items received a score of 0.70 - 0.79, which were corrected and revised and were then provided to 6 experts from the panel group. They were kept in the checklist with a score of >0.79.
C: Checklist reliability: To ensure the external reliability (reliability over time), the health, safety and environmental experts completed the checklist twice at a two-week interval in 20 surface mines, and the data were analyzed using the SPSS software. The interclass correlation coefficient (reliability over time) of the dimensions and the total score of the audit checklist were favorable (> 0.8). Thus, the lowest and highest correlations after two weeks were estimated to be 0.970 and 0.995, respectively, which were associated to the policy and strategic goals and the risk assessment and management dimensions, respectively. To determine the internal reliability (agreement between evaluators), the audit checklist was completed by three auditors in 8 surface mines, and the collected data were entered into the SPSS software. The interclass correlation coefficient (agreement between evaluators) of the dimensions and the total score of the audit checklist were also favorable (> 0.8). The highest and lowest internal reliability were associated to leadership and commitment (0.806) and risk assessment and management (0.967), respectively.
Conclusion: According to the results of this study, the CVI and CVR values for the whole audit checklist were 0.95 and 0.84, respectively. In other words, the audit checklist had an acceptable and appropriate validity index. Also, studying the reliability over time and the agreement between evaluators showed that the interclass correlation coefficient of the dimensions and the total score of the audit checklist were greater than 0.8. In other words, the auditors had given approximately the same answers to the questions and thus the dimensions of the audit checklist, which in fact confirmed the validity of the researcher-made checklist. Finally, after making necessary changes and removing inappropriate questions from the preliminary checklist during validity and reliability assessments, 194 questions remained in the audit checklist.
One of the strengths of this research is the comprehensiveness of the audit tool, which not only included safety, but also health and environment. It is also a specific tool for auditing the HSE management system of surface mines. In addition, the tool has high validity and reliability. One weakness of the tool is that it can only be used for surface mines and is not suitable for underground ones. It is recommended to improve the checklist for auditing the HSE management system for underground mines.
According to the results of this study, the checklist designed for surface mines had proper reliability and validity and could be used to audit the HSE management system of surface mines. Finally, the audit output of the HSE management system for mines could be used to improve the HSE management system, identify the risks and non-compliances in the whole HSE management system, management interventions to eliminate defects, and achieve compliance with standards and regulations.
Considering the slogan "Healthy humans are at the center of sustainable development" and given the need for maintaining the health of the workforce and protecting the environment, HSE management systems have currently been extended to most companies, organizations, industries, and mines to ensure workforce health and environmental protection. On the other hand, any organization and industry needs monitoring and auditing to determine the status and effectiveness of its HSE management system in order to identify and improve the strengths, weaknesses, and non-compliances of the system. So far, no comprehensive and coherent tool or mechanism has been provided for monitoring and auditing HSE in surface mines.
HSE audit is a systematic process used to assess the compliance with the guidelines, regulations, rules, and standard executive procedures developed in the field of HSE. The existence of a comprehensive HSE audit system makes it possible to identify and control all system defects before they may cause irreparable damage. In addition, regarding the importance of HSE in mines and despite the existence of the ISRS[1] for safety auditing in mines, as this method only deals with safety and does not address health and environment as well as the main issues on mining, and due to the lack of a comprehensive and specific tool in Iran for auditing health, safety and environment, the present study was conducted in 2019 to provide a valid audit tool for HSE management system for surface mines.
Methods: This analytical cross-sectional study was conducted in 3 stages:
A: Designing a preliminary checklist based on the HSE management system dimensions: Having reviewed the texts, standards, and regulations related to the health, safety and environment of the mines and also the related audit methods, the researcher developed the items for each dimension.
B: Checklist validity: Given that this study aimed to design a specific checklist for mines, face and content validity was evaluated. The convenience sampling method was used to ensure face validity, and 15 HSE experts (working in surface mines) were selected and invited to cooperate. Finally, according to the expertschr('39') opinions, the items were reviewed and corrected and a new draft was prepared.
To evaluate the content validity, qualitative and quantitative methods were used based on the Lawshe model. In a qualitative assessment of content validity, 15 HSE mining experts with work, research, and educational experiences were invited to participate as a panel group. They were then asked to read the checklist carefully and consider the issues such as the use of appropriate words, grammar, placement of items in their right place, and proper scoring, and to submit their corrective views in writing. Once the groupchr('39')s comments were collected, necessary changes were applied to the checklist.
To quantitatively examine the content validity, two indicators were used (content validity ratio and content validity index). Thus, the CVR[2] and CVI[3] were respectively used to ensure the selection of the most important and correct content and to ensure that the checklist items were best designed to measure the intended content.
C: Checklist reliability: Reliability shows how the measurement tool gives the same results under the same conditions. To determine the external reliability, the checklist was completed by HSE experts in 20 surface mines and was assessed using the test-retest method and correlation coefficient at a two-week interval.
Furthermore, the interclass correlation coefficient (ICC) was used to calculate the internal reliability (agreement between evaluators) of the checklist items. To this end, the checklist was completed by three auditors in 8 surface mines, and the results were analyzed using the ICC method.
Results: The results of this research are presented in the following sections: determining the items and designing the preliminary checklist, and evaluating the validity and reliability of the checklist.
A: Determining the items and designing the preliminary checklist: Having reviewed various texts and considering the opinions of the research team and experts, the researcher used the HSE management system dimensions to design the audit checklist of the HSE management system of surface mines. The dimensions included leadership and commitment; policy and strategic goals; organization, resources, and documentation; risk assessment and management; planning, implementation, and monitoring; and audit and review. In the next step, the items were designed for each dimension.
B: Checklist Validity: 1- Face Validity: Once the experts examined the sentences and items of the checklist in terms of appearance, word choice, and comprehensibility, necessary changes were applied. 2- Qualitative evaluation of content validity: After reviewing the checklist in terms of grammar, insertion of items in their right place, and proper scoring by the panel group (15 experts), the expertschr('39') opinions were collected and necessary changes were applied to the checklist. 3- Quantitative evaluation of content validity: The results of examining the content validity ratio (CVR) showed that 27 items of the checklist had a content validity ratio of < 0.49, and were removed from the audit checklist. Also, examining the content validity index showed that 191 items received a CVI score of >0.79, two items had a score of <0.70 and were removed from the audit checklist, and 3 items received a score of 0.70 - 0.79, which were corrected and revised and were then provided to 6 experts from the panel group. They were kept in the checklist with a score of >0.79.
C: Checklist reliability: To ensure the external reliability (reliability over time), the health, safety and environmental experts completed the checklist twice at a two-week interval in 20 surface mines, and the data were analyzed using the SPSS software. The interclass correlation coefficient (reliability over time) of the dimensions and the total score of the audit checklist were favorable (> 0.8). Thus, the lowest and highest correlations after two weeks were estimated to be 0.970 and 0.995, respectively, which were associated to the policy and strategic goals and the risk assessment and management dimensions, respectively. To determine the internal reliability (agreement between evaluators), the audit checklist was completed by three auditors in 8 surface mines, and the collected data were entered into the SPSS software. The interclass correlation coefficient (agreement between evaluators) of the dimensions and the total score of the audit checklist were also favorable (> 0.8). The highest and lowest internal reliability were associated to leadership and commitment (0.806) and risk assessment and management (0.967), respectively.
Conclusion: According to the results of this study, the CVI and CVR values for the whole audit checklist were 0.95 and 0.84, respectively. In other words, the audit checklist had an acceptable and appropriate validity index. Also, studying the reliability over time and the agreement between evaluators showed that the interclass correlation coefficient of the dimensions and the total score of the audit checklist were greater than 0.8. In other words, the auditors had given approximately the same answers to the questions and thus the dimensions of the audit checklist, which in fact confirmed the validity of the researcher-made checklist. Finally, after making necessary changes and removing inappropriate questions from the preliminary checklist during validity and reliability assessments, 194 questions remained in the audit checklist.
One of the strengths of this research is the comprehensiveness of the audit tool, which not only included safety, but also health and environment. It is also a specific tool for auditing the HSE management system of surface mines. In addition, the tool has high validity and reliability. One weakness of the tool is that it can only be used for surface mines and is not suitable for underground ones. It is recommended to improve the checklist for auditing the HSE management system for underground mines.
According to the results of this study, the checklist designed for surface mines had proper reliability and validity and could be used to audit the HSE management system of surface mines. Finally, the audit output of the HSE management system for mines could be used to improve the HSE management system, identify the risks and non-compliances in the whole HSE management system, management interventions to eliminate defects, and achieve compliance with standards and regulations.
Type of Study: Research |
Subject:
HSE
Received: 2019/10/30 | Accepted: 2020/08/8 | Published: 2020/05/30
Received: 2019/10/30 | Accepted: 2020/08/8 | Published: 2020/05/30
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
