Identification of preoperative risk factors for postoperative pulmonary complications after thoracic and abdominal surgery

Received: 19-Apr-2018 Accepted Date: May 15, 2018; Published: 20-May-2018

Citation: Jensen JH, Steffensen BF, Brendstrup T, Petersen AK. Identification of preoperative risk factors for postoperative pulmonary complications after thoracic and abdominal surgery. Gen surg: Open Access. 2018;1(1):13-18.

This open-access article is distributed under the terms of the Creative Commons Attribution Non-Commercial License (CC BY-NC) (http://creativecommons.org/licenses/by-nc/4.0/), which permits reuse, distribution and reproduction of the article, provided that the original work is properly cited and the reuse is restricted to noncommercial purposes. For commercial reuse, contact reprints@pulsus.com

Abstract

Postoperative pulmonary complications (PPC) are well-known after major thoracic and abdominal surgery and are associated with increased mortality, morbidity and prolonged hospital stay. There is no consensus in the literature regarding risk factors for PPC.

To optimize care and prevent PPC, we aimed at identifying the most important preoperative risk factors.

The Delphi method was used as a consensus-seeking approach. A total of 22 health professionals with clinical experience within the field of thoracic or abdominal surgery were invited to participate in the panel. Risk factors identified in the literature were commented on and rated by the panel in a three-round Delphi-process.

The Delphi panel reached consensus on 20 preoperative risk factors for PPC for patients undergoing elective thoracic or abdominal surgery. The risk factors were divided into two categories: related to type of surgery or to the patients' preoperative status. Furthermore, they were categorized as high, moderate or low risk factors. This list of risk factors is considered a qualified starting point for development of a risk assessment tool.

A systematic pre-operative identification of patients at high risk of PPC could be useful to facilitate an early preventive preoperative and postoperative intervention and to allocate proper resources to high risk patients.

Introduction

Incidence of postoperative pulmonary complications (PPC) in patients undergoing thoracic and abdominal surgery remains high and the occurrence of these complications has enormous implications for the patient and the health care system [1,2]. Complications may be related to anesthesia, mechanical ventilation, tissue damage, immobilization and pain. These conditions cause decreased lung volumes and limited airway clearance, which can lead to PPC such as atelectasis, pneumonia and hypoxemia [3]. Health professionals involved in the management of patient undergoing surgery needs to be aware that postoperative pulmonary complications are a major cause of morbidity, mortality, prolonged hospital stay, and increased cost of care [1,2,4].

The incidence of PPC varies in relation to type of surgery. After thoraco-abdominal surgery like esophagectomy or thoraco-abdominal aorta aneurism repair, incidences between 15 and 32% have been reported [5-9]. Within open lung, cardiac or abdominal surgery, PPC is less frequent with an incidence between 1 and 20% [10-15]. In recent years, less invasive surgical techniques such as laparoscopy have become more widespread. Furthermore, anesthesiology and postoperative treatments involving analgesics and mobilization have been optimized [16]. This makes it possible for older and fragile patients with co-morbidities to be offered surgery [17]. Thus, prevention of PPC is therefore highly relevant.

Prophylactic physiotherapy, especially in the postoperative period, has traditionally been a part of the standard treatment after thoracic and abdominal surgery. Evidence is scarce on the effect of preventive measures of PPC such as respiratory physiotherapy techniques like incentive spirometry, continuous positive airway pressure (CPAP) and positive expiratory pressure (PEP) in this patient group of surgery patients [18-21]. Moreover, evidence for early mobilization protocols is inconsistent [22]. However, a few studies have found that high risk patients in particular benefit from specific respiratory physiotherapy techniques to reduce their risk of PPC [23,24].

Therefore, a precise and systematic preoperative assessment of patients´ risk of PPC could facilitate an early and optimized intervention to prevent development and progression of PPC. For instance, the time point for initiation of respiratory physiotherapy techniques in the early postoperative period could be accentuated in high risk patients.

Several risk factors for PPC after thoracic and abdominal surgery have been identified in the literature and specific risk prediction models have been developed to provide an estimate of each patient's risk of a PPC [2,6,13,25-29]. However, the literature is inconsistent and risk prediction models have missing or poor external validity [30-33]. This makes it difficult to apply results to clinical practice, thus limiting a systematic identification of patients at increased risk of PPC and an appropriate allocation of physiotherapy resources to high risk patients.

In recent years, the Delphi method has become a recognized method for generating hypotheses, building consensus and developing best practice recommendations [34,35]. We hypothesized that the most important and generic risk factors for PPC in patients undergoing elective thoracic and abdominal surgery could be identified and rated by Delphi method.

By consulting a Delphi expert panel with a consensus-seeking approach, the purpose of this study was to identify and categorize preoperative risk factors for PPC. This work is considered a starting point for future prospective clinical investigations of risk assessment of patients undergoing thoracic and abdominal surgery.

Methods

Study design

The study was carried out as a three-round Delphi process. The focus of the study was to identify and categorize preoperative risk factors for PPC. The Delphi process was carried out at Aarhus University Hospital in Denmark between September 2014 and June 2015.

Participants

Potential participants were considered eligible for the Delphi panel if they; 1) had a degree in medicine, physiotherapy or nursing, 2) had a solid clinical experience within the area of thoracic or abdominal surgery, and 3) could read and understand Danish. The Delphi panel was strategically selected by the investigators to ensure representation by all specialties and health professionals within the field of interest.

In total, 22 potential panelists were invited to participate in the Delphi process. Of those invited, 18 had more than 15 years of clinical experience and three had published relevant research.

Ethical considerations

The Central Denmark Region Committee on Health Research Ethics was notified about the study, but further approval was not considered necessary.

The panelists were informed about the Delphi process and that anonymity was ensured.

Delphi process - procedure

Prior to the Delphi process, the literature was reviewed to identify preoperative risk factors for PPC related to thoracic and abdominal surgery. A total of 21 risk factors were identified and included as a starting point for the first Delphi round. To obtain a common understanding of PPC, the panelists were introduced to The Melbourne Group Scale (MGS-2) [13]. As seen in Figure 1, the MGS-2 is designed to detect PPC amendable to physiotherapy interventions [13], which is the focus of this study.

Figure 1: The Melbourne Group Scale version 2 (MGS-2).

The timeframe for a PPC was defined as the early postoperative period (from the first to the eighth postoperative day).

E–mail was used for communication between the Delphi panel and the primary investigator during the entire process. Using e-mail enabled participation of experts without considering geography and provided anonymity of the respondents. To ensure a high response rate in every Delphi round, a reminder was e-mailed to panelists who had not responded within the defined time-frame. To increase the transparency of the Delphi process, the primary investigator made a written summary of the data analysis containing both quantitative results and qualitative responses on every risk factor. This summary was sent to the panelists as feedback allowing them to reassess their initial input when making recommendations in subsequent rounds. As recommended in the literature, the level of consensus was defined a priori as a minimum of 80% agreement in the phrasing and categorization of each risk factor. The agreement was constituted categorical as yes or no.

Three rounds of review and feedback were required to achieve consensus. The content of each phase of the Delphi process is described in Figure 2.

Figure 2: Description of the content of each phase of the three-round Delphi process.

At each round, the panelists were asked to rate their assessment of each of the suggested risk factors on a numeric scale between 0 and 10, where 0 was no risk of PPC and 10 was maximal risk of PPC. Furthermore, they were invited to comment on the presented risk factors and to suggest additional risk factors. Based on responses from the panelists, risk factors were added, maintained, rephrased or omitted during the process.

Data analysis

The qualitative responses from the panelists contributed to rephrasing and consolidating risk factors through the rounds. Descriptive statistics was used to analyze the rated responses. In the first two Delphi rounds, the median value was estimated for every risk factor and used to categorize risk factors into three categories: High risk factor (ratings between 8-10) moderate risk factor (ratings between 5-7) and low risk factor (ratings between 0-4). Risk factors with a median rating from 0-4 were excluded. The interquartile range was calculated for each risk factor and used as a measure of agreement among the panelists. Good agreement was defined as an interquartile range of <3. Poor agreement was defined as an interquartile range of >3. Risk factors with poor agreement were rephrased in accordance to comments from the panelists.

Results

In total, we invited 22 health care professionals to participate in the Delphi panel. Of these, four declined participation and three dropped out during the process. Completion of all three Delphi rounds was fulfilled by 15 panelists. Figure 3 shows the progress through the phases of the study.

Figure 3: Participant flow through the phases of the Delphi process.

As a starting point for the first round, the panelists were introduced to the 21 risk factors identified in the literature. Based on suggestions from the panelists during the first round, another 15 risk factors were added in the second round.

At the end of the Delphi process the panelists reached consensus on a total of 20 risk factors. These were divided into risk factors regarding type of surgery and preoperative risk factors related to the patient. Table 1 illustrates the adjustments of risk factors throughout the three rounds. As shown in Table 1, five risk factors were maintained, eleven were omitted, eight were consolidated into two risk factors and twelve risk factors were rephrased. One rephrased risk factor was separated in two factors.

Table 1: Consensus based adjustment of risk factors presented to the Delphi panel in round 1. Risk factors were maintained, omitted, consolidated or rephrased until consensus was achieved in round 3. BMI: Body Mass Index; CAS: Cumulated Ambulation Score (1); DLCO: Diffusion Capacity in Lungs for Carbon-Monoxide; ECOG PS: Eastern Cooperative Oncology Group Performance Status (2); FEV1: Forced Expiratory Volume in the first second; NYHA: New York Heart Association´s Classes of Heart Failure; Positive Cough Test (2).

As seen in table 1, risk factors presented in round one underwent several adjustments during rounds due to a variety of opinions and reflections among the panelists:

Duration of open surgery: The panelists were unable to achieve consensus about a cut off value for duration of open surgery. It was argued that the location, the size of incision and the extent of surgery were more important factors. Consequently, the panelists reached consensus about grading open surgery procedures into surgery involving both thorax and abdomen (thoraco-abdominal surgery) or single region surgery (open thoracic or abdominal surgery). Furthermore, they agreed about including surgery procedures with expected intubation time>24 h as high-risk surgery.

Dementia was a subject of disagreement, as the risk of PPC would be influenced by the severity of dementia and not all patients with dementia would be diagnosed. Different suggestions were consolidated and rephrased into limited self-care. However, limited self-care was argued to be too difficult to define and recognize objectively, and consensus was not achieved in the last round.

Alcohol intake>5 units/day: it was debated that it was not the alcohol intake itself, but the impact on a person´s self-care capacity that was associated with increased risk of PPC. For this reason, alcohol intake was consolidated with limited self-care in round two, but it was omitted in the last round due to lack of consensus.

Underweight was rephrased to unintended weight loss. A few panelists argued that the catabolic condition would predispose more to an increased risk than the actual body weight. In contrast, other panelists emphasized BMI as an important factor. The Delphi panel finally reached consensus about unintended weight loss graded by a BMI level above or under 18, respectively.

Respiratory symptoms and diseases were considered by the panelists to be expected risk factors. During the process, respiratory symptoms and respiratory diseases were consolidated with chronic obstructive pulmonary disease (COPD) and chronic productive cough into a risk factor termed pulmonary disease. Several panelists emphasized the importance of considering the severity of pulmonary disease and this risk factor was therefore divided into moderate and severe pulmonary disease based on FEV1% and symptoms.

Limited physical function graded by The Cumulated Ambulatory Score (CAS) 0-2 or 3-4 [36] was introduced to the Delphi panel as an objective measure of physical function. Exercise capacity was suggested by one panelist as a supplementary measure of physical function but was omitted since no cut off value or testing method was recommended. Another panelist suggested using the Eastern Cooperative Oncology Group (ECOG) to score performance status [37] instead of the CAS. The ECOG performance status describes a patient´s level of functioning in terms of self-care capacity, daily activity and physical ability. The Delphi panel was consulted and all panelists, except one, preferred the ECOG performance status. It was agreed that the ECOG performance status 3 and 4 were considered as risk factors.

In the third round, the Delhi panel was able to reach consensus about six risk factors related to type of surgery and fourteen patient-related risk factors. As seen in Table 2, risk factors are categorized into high, moderate and low risk of PPC. In perspective the assessment of a patients´ risk of PPC will systematically be based upon an operational rating of surgery type combined with preoperative patient-related risk factors.

Table 2: Risk factors for PPC after thoracic and abdominal surgery identified and categorized into high moderate or low risk in a three-round Delphi process; BMI: Body Mass Index; ECOG PS: Eastern Cooperative Oncology Group Performance Status (1); FEV1: Forced Expiratory Volume in the first second; NYHA: New York Heart Association´s classes of heart failure.

Discussion

In this study, we aimed at identifying and categorizing the most important preoperative risk factors for PPC in patients undergoing elective thoracic and/or abdominal surgery. Based on a three-round Delphi process consensus was achieved on 14 preoperative patient-related risk factors and six risk factors related to type of surgery, all categorized into high, moderate and low risk of PPC.

When the literature is inconsistent, Delphi panel recommendations can be valuable as they reflect consensus in a group of experts rather than individual opinions [38]. Based on the expert input it is assumed that the most generic and relevant risk factors have been identified. We consider this a solid foundation for future investigation of how these risk factors contribute to the development of PPC to develop a systematic risk assessment tool for identifying patients at increased risk of PPC.

The selection of participants for the Delphi panel is a critical element because it is directly related to the quality of the results generated [34]. The panelists in this study represented health professionals with an indepth knowledge in the field of thoracic or abdominal surgery. Recruitment in many Delphi studies is preferred among participants with research experience within the field of interest [35,38]. Only a few panelists in this Delphi process had done research on prevention of PPC, but the majority of panelists had a solid clinical experience. We considered that knowledge from clinical practice would be a valuable contribution to the knowledge retrieved from the literature to identify and categorize the most important risk factors.

Some of the identified risk factors in the literature did not reach consensus by the Delphi panel. Duration of surgery is an example of how the panelists were unable to agree about cut off values. Instead they agreed upon grading surgical procedures according to extent of surgery, size of incision and location. This grading is in accordance with Kehlet, reporting that the surgical stress response is directly related to the extent of tissue damage [16]. The different perspectives from health professionals could explain the broad range of opinions regarding duration of surgery e.g. surgeons have more in-depth knowledge about the impact of surgery procedures. It may have strengthened the study to include other aspects of risk factors for PPC if the Delphi panel had included specialists in pulmonary medicine.

Many of the panelists came from the same university hospital, which increased the risk of a culture-based dominance. Involving more international participants could perhaps have contributed with different aspects. However, a Danish language skill was an inclusion criterion for the panelists, which limited the inclusion of panelists.

Another consideration was that participation in a Delphi process is time consuming. This Delphi process lasted almost a year and there were only a few drop-outs during that period. The response rate at a minimum of 83% (15/18) in all three rounds was within the accepted level according to Delphi research guidelines [39].

Using e-mail correspondence during the process ensured anonymity, which reduces the potential of group pressure. On the other hand, a consensus-seeking process in a focus group setting could have clarified the modification and selection process of risk factors with other perspectives arising from discussions. Nevertheless, the controlled feedback after each Delphi round with both a qualitative and quantitative summary of data could provide the panelists with additional insight and ensure that all responses were well represented in all rounds.

Although differences in opinions among the panelists emerged during the Delphi process, panelists reached consensus on 20 risk factors. The level of consensus was defined a priori to 80% to strengthen the study. There are no standard requirements regarding level of consensus, but a level above 75% is recommended [40]. A few risk factors rated important by several panelists were omitted because they did not reach the desired level of consensus. Limited self-care was one of these; many panelists perceived that patients with limited self-care could find it difficult to follow preventive treatment recommendations and, in this way, increase their risk of complications. Other panelists argued that these socio-psychological issues were more complex and therefore difficult to recognize objectively. A few risk factors failed to reach the level of consensus and this underlines that not all conditions can be classified, and individual clinical judgment of the patients is always essential in clinical decision making.

Not only risk factors for PPC have been identified in studies, but also risk prediction tools have been developed to distinguish between patients at risk of PPC after surgery from patients not at risk. Canet et al. [27] and Gupta et al. [28], both developed a risk prediction tool by prospectively following a large cohort undergoing a variety of surgical procedures. Canet et al. identified age above 80 years, preoperative low saturation, respiratory infection, in the last month, preoperative anemia, intrathoracic or upper-abdominal surgical incision, duration of surgery >2 h and emergency procedures as independent risk factors for PPC [27]. We believe that low saturation would appear as a manifestation of pulmonary disease and anemia was not considered in any other study. Gupta et al. created a model with seven risk factors including age, American Society of Anesthesiologists (ASA) classification, chronic obstructive pulmonary disease, dependent functional status, preoperative sepsis, smoking before operation, and type of surgery [28]. We found that sepsis was irrelevant in this context, as this study concerns preoperative risk factors for patients undergoing elective surgery. The ASA classification was also suggested as a measure of co-morbidity by Smetana et al. [2]. However, the ASA classification has a low precision of classes [41], thus it was not presented to the panelists. Instead, co-morbidity was introduced as a single risk factor like COPD, heart failure and cancer.

Two meta-analyses concluded that both current smoking and alcohol intake are associated with an increased risk of PPC [42,43]. Both risk factors were presented to the panelists, but consensus was only reached for smoking. The fact that alcohol intake was considered not to be a risk factor according to the panelists is an example of how decision making should be informed by current available evidence, but always applied appropriately for the individual patients.

Conclusion

A novel approach for identification of preoperative risk factors for postoperative pulmonary complications after thoracic and abdominal surgery has been presented in this paper. The main contribution is in terms of the systematic preoperative identification of patients at high risk of PPC, which could be useful to facilitate an early preventive preoperative and postoperative intervention and to allocate proper resources to high risk patients. The list of risk factors is not yet operational for systematic risk assessment in clinical practice but is considered a starting point for further development of a manageable assessment tool.

Acknowledgements

We thank the Delphi panel participants who generously offered their time and expertise to contribute to this study.

We also thank the Aarhus University Hospital Foundation for financially supporting this work.

Conflict of Interest

The authors report no conflict of interest.

REFERENCES

1. Qaseem A, Snow V, Fitterman N, et al. Risk assessment for and strategies to reduce perioperative pulmonary complications for patients undergoing noncardiothoracic surgery: A guideline from the American College of Physicians. Ann Intern Med. 2006;144(8):575-80.
2. Smetana GW, Lawrence VA, Cornell JE. Preoperative pulmonary risk stratification for noncardiothoracic surgery: Systematic review for the American College of Physicians. Ann Intern Med. 2006;144(8):581-95.
3. Frownfelter DDE. Individuals with acute surgical conditions. Cardiovascular and Pulmonary Physical Therapy - Evidence to Practice 5th edition 2012; 488-98.
4. Neto AS, Hemmes SNT, Barbas CSV, et al. Incidence of mortality and morbidity related to postoperative lung injury in patients who have undergone abdominal or thoracic surgery: A systematic review and meta-analysis. Lancet Respir Med. 2014;2(12):1007-15.
5. Law S, Wong KH, Kwok KF, et al. Predictive factors for postoperative pulmonary complications and mortality after esophagectomy for cancer. Ann Surg. 2004;240(5):791-800.
6. Ferguson MK, Durkin AE. Preoperative prediction of the risk of pulmonary complications after esophagectomy for cancer. J Thorac Cardiovasc Surg. 2002;123(4):661-9.
7. Avendano CE, Flume PA, Silvestri GA, et al. Pulmonary complications after esophagectomy. Ann Thorac Surg. 2002;73(3):922-6.
8. Kinugasa S, Tachibana M, Yoshimura H, et al. Postoperative pulmonary complications are associated with worse short- and long-term outcomes after extended esophagectomy. J Surg Oncol. 2004;88(2):71-77.
9. Etz CD, Di Luozzo G, Bello R, et al. Pulmonary Complications After Descending Thoracic and Thoracoabdominal Aortic Aneurysm Repair: Predictors, Prevention, and Treatment. Ann Thorac Surg. 2007;83(2):S870-6.
10. Amar D, Munoz D, Shi W, et al. A clinical prediction rule for pulmonary complications after thoracic surgery for primary lung cancer. Anesth Analg 2010;110(5):1343-1348.
11. Riera M, Ibáñez J, Herrero J, et al. Respiratory tract infections after cardiac surgery: Impact on hospital morbidity and mortality. J Cardiovasc Surg. 2010;51(6):907-914.
12. Topal AE, Eren MN. Risk factors for the development of pneumonia post cardiac surgery. Cardiovasc J Afr. 2012;23(4):212-215.
13. Scholes RL, Browning L, Sztendur EM, et al. Duration of anaesthesia, type of surgery, respiratory co-morbidity, predicted VO2max and smoking predict postoperative pulmonary complications after upper abdominal surgery: An observational study. Aust J Physiother. 2009;55(3):191-198.
14. Boden I, Skinner EH, Browning L, et al. Preoperative physiotherapy for the prevention of respiratory complications after upper abdominal surgery: Pragmatic, double blinded, multicentre randomised controlled trial. BMJ. 2018;360.
15. Lugg ST, Agostini PJ, Tikka T, et al. Long-term impact of developing a postoperative pulmonary complication after lung surgery. Thorax. 2016;71(2):171-176.
16. Kehlet H, Wilmore DW. Evidence-based surgical care and the evolution of fast-track surgery. Ann Surg. 2008;248(2):189-198.
17. Hulzebos EHJ, Van Meeteren NLU, De Bie RA, et al. Prediction of postoperative pulmonary complications on the basis of preoperative risk factors in patients who had undergone coronary artery bypass graft surgery. Phys Ther. 2003;83(1):8-16.
18. Pasquina P, Tramèr MR, Walder B. Prophylactic respiratory physiotherapy after cardiac surgery: Systematic review. Br Med J. 2003;327(7428):1379-1381.
19. Pasquina P, Tramèr MR, Granier JM, et al. Respiratory physiotherapy to prevent pulmonary complications after abdominal surgery: A systematic review. Chest. 2006;130(6):1887-1899.
20. Ireland CJ, Chapman TM, Mathew SF, et al. Continuous positive airway pressure (CPAP) during the postoperative period for prevention of postoperative morbidity and mortality following major abdominal surgery. Cochrane Database Syst Rev. 2014;1(8).
21. Do Nascimento Junior P, Módolo NSP, Andrade S, et al. Incentive spirometry for prevention of postoperative pulmonary complications in upper abdominal surgery. Cochrane Database Syst Rev. 2014;8(2).
22. Castelino T, Fiore JF, Niculiseanu P, et al. The effect of early mobilization protocols on postoperative outcomes following abdominal and thoracic surgery: A systematic review. Surgery. 2016;159(4):991-1003.
23. Agostini P, Naidu B, Cieslik H, et al. Effectiveness of incentive spirometry in patients Following thoracotomy and lung resection including those at high risk for developing pulmonary complications. Thorax. 2013;68(6):580-585.
24. Hulzebos EHJ, Helders PJM, Favié NJ, et al. Preoperative intensive inspiratory muscle training to prevent postoperative pulmonary complications in high-risk patients undergoing CABG surgery: A randomized clinical trial. JAMA. 2006;296(15):1851-1857.
25. Kinlin LM, Kirchner C, Zhang H, et al. Derivation and validation of a clinical prediction rule for nosocomial pneumonia after coronary artery bypass graft surgery. Clin Infect Dis. 2010;50(4):493-501.
26. Brooks-Brunn JA. Predictors of postoperative pulmonary complications following abdominal surgery. Chest. 1997;111(3):564-571.
27. Canet J, Gallart L, Gomar C, et al. Prediction of postoperative pulmonary complications in a population-based surgical cohort. Anesthesiology. 2010;113(6):1338-1350.
28. Gupta H, Gupta PK, Schuller D, et al. Development and validation of a risk calculator for predicting postoperative pneumonia. Mayo Clin Proc. 2013;88(11):1241-1249.
29. Agostini P, Cieslik H, Rathinam S, et al. Postoperative pulmonary complications following thoracic surgery: Are there any modifiable risk factors? Thorax. 2010;65(9):815-818.
30. Parry S, Denehy L, Berney S, et al. Clinical application of the Melbourne risk prediction tool in a high-risk upper abdominal surgical population: An observational cohort study. Physiotherapy. 2014;100(1):47-53.
31. Brooks-Brunn JA. Validation of a predictive model for postoperative pulmonary complications. Heart Lung. 1998;27(3):151-158.
32. Yepes-Temiño MJ, Monedero P, Pérez-Valdivieso JR. Risk prediction model for respiratory complications after lung resection. Eur J Anaesthesiol. 2016;33(5):326-333.
33. Sanagou M, Wolfe R, Leder K, et al. External validation and updating of a prediction model for nosocomial pneumonia after coronary artery bypass graft surgery. Epidemiol Infect. 2014;142(3):540-544.
34. Hsu CC, Sandford BA. The Delphi technique: Making sense of consensus. Practical Assessment, Research and Evaluation. 2007;12(10):1-8.
35. Hanekom S, Gosselink R, Dean E, et al. The development of a clinical management algorithm for early physical activity and mobilization of critically ill patients: Synthesis of evidence and expert opinion and its translation into practice. Clin Rehabil. 2011;25(9):771-787.
36. Foss NB, Kristensen MT, Kehlet H. Prediction of postoperative morbidity, mortality and rehabilitation in hip fracture patients: The cumulated ambulation score. Clin Rehabil. 2006;20(8):701-708.
37. Oken MM, Creech RH, Davis TE Toxicology and response criteria of the Eastern Cooperative Oncology Group. Am  J Clin Oncol Cancer Clin Trials. 1982;5(6):649-655.
38. McGinnis PQ, Wainwright SF, Hack LM, et al. Use of a Delphi panel to establish consensus for recommended uses of selected balance assessment approaches. Physiother Theory Pract. 2010;26(6):358-373.
39. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000;32(4):1008-1015.
40. Keeney S, Hasson F, McKenna H. Consulting the oracle: Ten lessons from using the Delphi technique in nursing research. J Adv Nurs. 2006;53(2):205-212.
41. Castillo J, Canet J, Gomar C, et al. Imprecise status allocation by users of the American Society of Anesthesiologists classification system: survey of Catalan anesthesiologists. Rev Esp Anestesiol Reanim. 2007;54(7):394-398.
42. Eliasen M, Grønkjær M, Skov-Ettrup LS, et al. Preoperative alcohol consumption and postoperative complications: A systematic review and meta-analysis. Ann Surg. 2013;258(6):930-942.
43. Grønkjær M, Eliasen M, Skov-Ettrup LS, et al. Preoperative smoking status and postoperative complications: A systematic review and meta-analysis. Ann Surg. 2014;259(1):52-71.