Assessing Perioperative Risk.
N Shah, A Hegarty
a. Is risk free
b. Electively has a mortality in the UK of 1-2%
c. Emergently has a mortality in the UK of 5%
d. A small number of people suffer most of the complications
e. Cardiac surgery has a mortality of 9%
a. Is vanishingly rare
b. Is not important
c. Cannot be predicted
d. Cannot be reduced
e. Is purely due to physiological factors
a. 1 MET is equivalent to 3.5 mls/min/kg of O2 consumption
b. Exercise tolerance below 4 METS is associated with poorer outcomes
c. 4 METs of exercise tolerance is the equivalent of climbing 1 flight of stairs
d. We cannot measure a patient’s MET status in daily practice
e. The MET status of a patient does not affect the risk of postoperative morbidity
4. Risk Assessment
a. The ASA system has no relation to mortality and morbidity
b. Single organ risk scores don’t predict mortality
c. A patient with a mortality risk of 5% should have consultant delivered care
d. The APACHE score was created for perioperative prediction
e. POSSUM is not well validated
5. New scores:
a. ACS NSQIP is validated in the UK population
b. SORT predicts morbidity
c. SORT cannot be used in an emergency population
d. Neither score gives detailed morbidity predictions
e. Both scores compare well to the POSSUM score for predictive power
6. Functional Assessment
a. Only refers to CPET
b. Can only be tested, not inferred
c. CPET does not offer any additional information above the 6 minute walk test
d. There is no evidence base for functional assessment relating to mortality
e. The MET calculation is the only useful information to be derived from functional assessment
· Risk assessment and communication is essential to ensure informed consent
· There are many scoring systems available, with variable evidence base and validation
· Functional capacity assessment is increasingly comprehensive with a growing evidence base
· UK based and validated scoring systems are being developed (e.g. SORT and PQIP)
· Frailty scoring is also increasingly topical and provides a useful adjunct to formal organ based risk scoring
There are an estimated 313 million operations carried out worldwide every year, with over 4.2 million of these in England. For a variety of reasons, it is difficult to gain a precise estimation of perioperative mortality and morbidity. However, it has been suggested that this may occur in between 3 and 17% of operations, with total UK inpatient surgical mortality up to 3.6%. Emergency surgery has been shown to have significantly higher mortality, with rates exceeding 11% in certain patient groups. A small group of particularly high risk patients has been shown to be responsible for over 80% of deaths and prolonged hospitalisation, despite making up only 12.5 % of hospital admissions for surgery. Interestingly, although 90% of this high-risk group had emergency surgery, less than 15% of them were admitted directly to critical care following the operative procedure. This figure compares poorly to cardiac surgical patients. This group of high risk patients are routinely admitted to critical care postoperatively, and mortality is now generally accepted to be less than 2%.
The wide range in morbidity reported above is likely to be multi-factorial, involving variable reporting, disputed classification of complications and of variable source. The complications cover most organ systems, including cardiovascular, respiratory, gastrointestinal, renal and haematological. There is no clear agreement on a structured classification of postoperative morbidity which makes it very difficult to compare different predictive models for operative complications. Clavien et al proposed a model for this in 1992, but this had a slow uptake secondary to concerns regarding adequate validation of the model. This was overcome following an update to the model in 2004 which was evaluated in a large international cohort of patients. This new model enables broad characterisation of complications in different grades, regardless of the initial surgery. In order to accurately collect information about more specific complications, the Postoperative Morbidity Survey has been developed and well validated, both internationally and in the UK. With increasing use of the two systems described above, we are now starting to gather more accurate information on postoperative morbidity.
Taken from 22
As outlined above, there are clear potential physiological benefits to identifying and intervening in high risk patients. This is important; both to ensure appropriate perioperative care and to inform consent. Furthermore, good understanding of both the benefit and the risks involved in operative care is essential in the process of shared decision making. Finally, it is important to use the identification of the higher risk patient to guide interventions to reduce the risk. This may relate to changing patient factors, operative factors or perioperative care pathways. Ultimately, the primary aim of risk stratification is to enable us to target care to reduce potential morbidity and mortality.
Extensive work has been carried out over the last decade to produce guidance on the identification and care of the high-risk patient. This began with the standardisation of preoperative testing outlined in the guidelines issued by the National Institute for Clinical Excellence in 2003.
Both the American Heart Association and the European Society of Cardiology have published guidance on perioperative cardiovascular evaluation prior to noncardiac surgery. These have been further endorsed by the European Society of Anesthethesiology. A key feature of both sets of guidance is the use of metabolic equivalents (METs). 1 MET is referenced to the oxygen consumption of a 40-year-old man weighing 70 Kg and is approximated at 3.5ml/ minute/ Kg. There is some evidence demonstrating that patients who are unable to climb 2 flights of stairs (4 METs worth of work) are at greater risk of perioperative morbidity and mortality following major surgery.
This is incorporated to a significant degree in the Association of Anaesthetists of Great Britain and Ireland’s various guidelines published between 2010 and 2015. These documents encourage a formal perioperative assessment process with the identification of high risk patients both pre and postoperatively. In 2014, the Institute for Clinical Systems Improvement in the United States of America revised their guidance from 1997, further emphasising the importance of formal pre-operative assessment incorporating a level of functional assessment.
As part of this increased interest in identifying and improving the care of the high-risk patient, the Royal College of Surgeons of England and the Department of Health have also set up a Working Group on the Peri-operative Care of the Higher Risk General Surgical Patient. This group have also issued a set of guidelines on the care of the high-risk surgical patient. In addition to the preoperative guidance, this piece of work also describes the desired response to postoperative complications, including early involvement of intensive care.
To successfully identify a patient as high risk, it is essential to define what “high risk” means. The Royal College of Surgeon’s Working group has suggested the high-risk patient to be someone with an estimated mortality of greater than 5%. They go on to suggest that consultant presence should be mandated for patients with a mortality risk of greater than 10%, and that these patients should subsequently go on to Intensive Care postoperatively.
To enable us to estimate perioperative mortality, we need to make use of a predictive model. This model needs to consider both the patient’s physiology and the surgical insult that they are about to undergo. There are a number of different approaches to this that we will outline below.
The best-known risk score in common use that is based on estimations of population risk is the American Society of Anesthesiologists Physiological Status (ASA-PS) classification. This was originally created for statistical data collection and reporting, but has subsequently been used a risk scoring tool. There is evidence that the different classes have been shown to be good predictors of mortality, with some predictive value for morbidity as well.
It is a well-known scale that is easy to use with widespread recognition across specialties. Unfortunately, this also makes it less useful, as it is extremely subjective with poor individual or procedure specificity and hence, poor specificity or sensitivity for mortality or morbidity.
Another generic score that is well known is the Charlson Comorbidity Index. This score is used to predict survival based on the presence and severity of various comorbid diseases. As with the ASA, this is a relatively simple score to apply, but has them same limitations outlined above. Hence, it has not entered common daily clinical practice, but tends to remain a research tool.
Organ Specific Risk
Another way of assessing risk is to focus on the specific organs that are of interest in the individual.
A well-recognised and validated example of this approach is the Revised Cardiac Risk Index, published by Lee and colleagues. This revision of an older risk score is simple to use and also factors in the scale of surgery to be undertaken. The obvious constraint on wider application of the Index is its organ specific nature.
Physiological Risk Scores
The best described example of a physiological risk score is the Acute Physiology and Chronic Health Evaluation (APACHE) score. This was first introduced in 1981, and has since been updated with the APACHE II (the best known and widespread iteration) in 1985. Subsequent versions have been released, with APACHE IV released in 2006. APACHE II remains the best validated for the perioperative patient; both because of easy access, and because it is considerably simpler than the later models. APACHE II takes account of 12 physiological variables collected over the first 24 hours of the inpatient admission, with additional weighting provided by age and chronic health. However, as one might expect in a score initially designed for use in critical care, APACHE II does not account for the type of surgery undertaken. This, in combination with the need for the variables to be collected over 24 hours to get an accurate estimate of risk is the major reason that APACHE II is not used more regularly.
A derivation of the APACHE database that has also failed to gain great traction is the Simplified Acute Physiology Score. This score is now on its second iteration, but also requires the collection of a range of physiological variables over 24 hours, leading to poor uptake and use in perioperative scoring.
A third physiological scoring system that can be used perioperatively is the Sequential Organ Failure Assessment score (SOFA score). This is another physiological score that has limited application preoperatively, but can be used to predict perioperative course and postoperative mortality and morbidity.
The most commonly utilised perioperative scoring system is probably the Physiological and Operative Severity Score for Enumeration of Mortality and Morbidity (POSSUM). Unlike many of the systems described above, this tool was designed specifically for use as a perioperative risk prediction tool. As a result, it accounts for both individualised physiology and operative insult. This is carried out by consideration of 12 physiological and 6 operative variables. A logistic regression method is used following mathematical modelling to predict mortality and morbidity. However, a side effect of this approach is a tendency to over predict mortality and morbidity in lower risk patients. An attempt was made to rectify this in 1998 with the Portsmouth-POSSUM tool. This improved mortality scoring, but not morbidity scoring. Subsequently several surgery specific versions have been released, including Colorectal-POSSUM and Vascular-POSSUM. POSSUM, P-POSSUM and the various surgical iterations have all been well validated across a variety of specialties and health systems.
The Veterans Affairs (VA) Hospitals in the USA prospectively collected data on major operations at 44 VA hospitals for the National Veterans Affairs Surgical Risk Study. This led to the development of risk adjusted models for 30-day mortality and morbidity for a few surgical subspecialties. This led to the Veterans Affairs National Surgical Quality Improvement Programme (NSQIP) being set up in 1994 at all VA hospitals. This programme had the dual aim of data collection and quality improvement, leading to a 45% reduction in morbidity and 27% reduction in mortality. The American College of Surgeons (ACS) got involved with the programme through the Patient Safety in Surgery study from 2001 to 2004. Following a significant reduction in mortality through this study, the ACS-NSQIP was rolled out. This programme aimed to use hospitals with low rates of morbidity and mortality as benchmarks to identify adjustable factors in poor outcomes; by feeding these back into worse performing hospitals, the aim is to improve results. They collected data from over 2.7 million operations at 586 participating hospitals between 2010 and 2014 and used it to develop a new risk scoring tool- the ACS NSQIP Surgical Risk Calculator.
This tool is based on 20 patient parameters along with the surgery planned. As well as predicted 30-day mortality, it also provides the risk of up to 14 separate morbidity outcomes, with a comparison to an “average patient”. This makes this risk scoring tool particularly valuable when discussing procedural risk with patients, and attempting to gain truly informed consent. While this tool has been validated in for an American population through the ACS NSQIP programme, there is no data currently available on its applicability to a British population and health service. As a result, it is important to always qualify the use of this tool with patients.
In 2010, data was collected on 19,097 operations carried out in 326 British hospitals for publication in the 2011 National Confidential Enquiry into PeriOperative Death (NCEPOD) study (“Knowing the Risk”). Subsequently, this dataset was used to build and internally validate a new perioperative scoring system- the Surgical Outcome Risk Tool (SORT). SORT utilises 6 preoperative variables to provide a mortality risk that is validated for the UK population. Unfortunately, there is not a morbidity calculation, which remains the biggest drawback to this scoring system.
There have been a number of different forms of functional assessment used in the perioperative setting. Functional assessment has always been a central part of the pre-assessment process prior to the removal of organs (pulmonary testing prior to pneumonectomy and dimercaptosuccinic acid scans before nephrectomy). In keeping with this approach of organ specific testing, we have also traditionally made use of functional assessment to quantify dysfunction in patients with known organ disease. Examples of this approach include spirometry, coronary angiograms and stress echocardiography.
A central plank of functional capacity assessment is the concept of Metabolic Equivalents (METs). 1 MET is the equivalent of metabolic activity of rest, while 4 METs equates to climbing 2 flights of stairs uninterrupted. Numerous studies have demonstrated significantly higher mortality and morbidity in patients undergoing significant surgery if they are unable to perform 4 METs worth of exercise.
A more formal assessment of functional capacity can be carried out by either the 6 Minute Walk Test or the Shuttle Walk Test (SWT). There is evidence that these non-invasive tests correlate well with perioperative mortality and morbidity in major surgery. Despite this, they are fundamentally limited by the fact that they cannot diagnose the limitations in exercise tolerance that they may demonstrate. Moreover, both tests are limited by being dependant on participant effort. There is no way of ensuring that the exercise performed is maximal; however, they may be better measures of the normal functional level of participants.
The area of functional assessment that is currently receiving the most investigation is cardiopulmonary exercise testing (CPET). CPET involves incremental exercise, either using an exercise bicycle or a treadmill, up till the patient’s maximal exercise level. During this period of exercise the operator will measure the patients ventilatory effort, inspiratory gasses, expiratory gasses, blood pressure and electrocardiogram. These results are used to calculate the body’s maximal oxygen uptake (VO2 Max) and the point at which anaerobic metabolism exceeds aerobic metabolism (Anaerobic Threshold (AT)). These figures are then used to calculate the patients true MET status. Interestingly, the combination of AT and VO2 Max have demonstrated that the MET status as estimated by exercise tolerance or SWT/6MWT is often wrong. CPET has thus been used to identify patients at higher perioperative risk who would not have otherwise been noted to be high risk. The evidence base for CPET’s ability to predict postoperative complications was initially strongest in pulmonary surgery; however, there is now increasing evidence for its applicability in general surgery. However, gaps continue to exist in the evidence base in certain surgical sub-specialties.
An additional area of CPET testing that potentially exciting is the use of CPET to identify patients who will benefit from prehabilitation prior to surgery. The theory is that intervention to ensure a patient can achieve certain VO2 Max and AT figures prior to surgery will result in improved outcomes. However, there is no strong evidence to demonstrate benefit yet.
As the population gets older, the final area of risk assessment that has seen increasing development is the development of frailty scores. The concept of the frailty phenotype has been discussed for several years, resulting in work to try to quantify it. Various generic scores have been described in the literature, including the Canadian Study of Health and Ageing, the Edmonton Frailty Scale and the Modified Frailty Index. The Nottingham Hip Fracture Score attempts to use a frailty score to estimate mortality for a specific operation. In keeping with the other frailty scores this has been shown to correlate with mortality and morbidity, but there is not robust validation data yet.
There have been several major advances in risk assessment in the recent past. Work is underway to add a morbidity component to the SORT scoring system in the UK. The launch of the Perioperative Quality Improvement Programme (PQIP) in the UK is anticipated to lead to the creation of a larger UK database of patient outcomes, enabling the development of risk scores that are better validated in our populations. The ongoing work on functional and frailty assessment is likely to lead to better validation and application of both systems in our regular practice. Finally, the increasing focus on informed consent is likely to lead to formal risk assessment becoming part of routine perioperative practice for all patients soon.
 Weiser TG, Haynes AB, Molina G et al. Estimate of the global volume of surgery in 2012: an assessment supporting improved health outcomes. Lancet. 2015;385 Suppl 2:S11.
 Hospital Episode Data [www.hesonline.nhs.uk]
 Kable AK, Gibberd RW, Spigelman AD: Adverse events in surgical patients in Australia. Int J Qual Health Care 2002, 14:269-276.
 Gawande AA, Thomas EJ, Zinner MJ, Brennan TA: The incidence and nature of surgical adverse events in Colorado and Utah in 1992. Surgery 1999, 126:66-75.
 Pearse RM, Moreno RP, Bauer P et al. Mortality after surgery in Europe: a 7 day cohort study. Lancet. 2012;380:1059-1065.
 NELA Project Team. First patient report of the National Emergency Laparotomy Audit. 2015.
 Saunders DI, Murray D, Pichel AC, Varley S, Peden CJ. Variations in mortality after emergency laparotomy: the first report of the UK Emergency Laparotomy Network. Br J Anaesth. 2012;109:368-375.
 Pearse RM, Harrison DA, James P, Watson D, Hinds C, Rhodes A, Grounds RM, Bennettt ED: Identi cation and characterisation of the high-risk surgical population in the United Kingdom. Crit Care 2006, 10:R81.
 Survival Rates – Heart Surgery in United Kingdom [http://heartsurgery.cqc.org.uk/Survival.aspx]
 Clavien P, Sanabria J, Strasberg S: Proposed classi cation of complication of surgery with examples of utility in cholecystectomy. Surgery 1992, 111:518-526.
 Bennett-Guerrero E, Welsby I, Dunn TJ, Young LR, Wahl TA, Diers TL, Phillips- Bute BG, Newman MF, Mythen MG: The use of a postoperative morbidity survey to evaluate patients with prolonged hospitalization after routine, moderate-risk, elective surgery. Anesth Analg 1999, 89:514-519.
 Dindo D, Demartines N, Clavien PA: Classi cation of surgical complications. Ann Surg 2004, 240:205-213.
 Grocott MPW, Browne JP, Van der Meulen J, Matejowsky C, Mutch M, Hamilton MA, Levett DZH, Emberton M, Haddad FS, Mythen MG: The Postoperative Morbidity Survey was validated and used to describe morbidity after major surgery. J Clin Epidemiol 2007, 60:919-928.
 National Institute for Clinical Excellence: Pre-operative Tests, the Use of Routine Pre-operative Tests for Elective Surgery. London: NICE; 2003.
 Fleisher LA, Beckman JA, Brown KA, Calkins H, Chaikof EL, Fleischmann KE, Freeman WK, Froehlich JB, Kasper EK, Kersten JR, Riegel B, Robb JF, Smith SC Jr, Jacobs AK, Adams CD, Anderson JL, Antman EM, Buller CE, Creager MA, Ettinger SM, Faxon DP, Fuster V, Halperin JL, Hiratzka LF, Hunt SA, Lytle BW, Nishimura R, Ornato JP, Page RL, Riegel B, et al.: ACC/AHA 2007 guidelines on perioperative cardiovascular evaluation and care for noncardiac surgery: a report of the American College of Cardiology/ American Heart Association Task Force on Practice Guidelines (Writing Committee to Revise the 2002 Guidelines on Perioperative Cardiovascular Evaluation for Noncardiac Surgery). J Am Coll Cardiol 2007, 50:e159-e241.
 Fleisher LA, Beckman JA, Brown KA, Calkins H, Chaikof EL, Fleischmann KE, Freeman WK, Froehlich JB, Kasper EK, Kersten JR, Riegel B, Robb JF, American Society of Echocardiography; American Society of Nuclear Cardiology; Heart Rhythm Society; Society of Cardiovascular Anesthesiologists; Society for Cardiovascular Angiography and Interventions; Society for Vascular Medicine; Society for Vascular Surge: 2009 ACCF/AHA focused update on perioperative beta blockade incorporated into the ACC/AHA 2007 guidelines on perioperative cardiovascular evaluation and care for noncardiac surgery. J Am Coll Cardiol 2009, 54:e13-e118.
 Poldermans D, Bax JJ, Boersma E, De Hert S, Eeckhout E, Fowkes G, Gorenek B, Hennerici MG, Iung B, Kelm M, Kjeldsen KP, Kristensen SD, Lopez-Sendon J, Pelosi P, Philippe F, Pierard L, Ponikowski P, Schmid JP, Sellevold OF, Sicari R, Van den Berghe G, Vermassen: Guidelines for preoperative cardiac risk assessment and perioperative cardiac management in noncardiac surgery: the Task Force for Preoperative Cardiac Risk Assessment and Perioperative Cardiac Management in noncardiac Surgery of the European Society of Cardiology (ESC) and endorsed by the European Society of Anaesthesiology (ESA). Eur Heart J 2009, 30:2769-2812.
 Reilly DF, McNeely MJ, Doerner D, Greenberg DL, Staiger TO, Geist MJ, Vedovatti PA, Co ey JE, Mora MW,Johnson TR, Guray ED, Van Norman GA, Fihn SD: Self-reported exercise tolerance and the risk of serious perioperative complications. Arch Intern Med 1999, 159:2185-2192.
 Verma R, Wee MYK, Hartle A, Alladi V, Rollin AM, Meakin G, Struthers R, Carlisle J, Johnston P, Ricett K, Hurley C: Pre-operative Assessment and Patient Preparation. London: Association of Anaesthetists of Great Britain and Ireland; 2010.
 Card R, Sawyer M, Degnan B, Harder K, Kemper J, Marshall M, Matteson M, Roemer R, Schuller-Bebus G, Swanson C, Stultz J, Sypura W, Terrell C, Varela N. Institute for Clinical Systems Improvement. Perioperative Protocol. Updated March 2014.
 Anderson I, Eddleston J, Grocott M, Lees N, Lobo D, Loftus I, Markham N, Mitchell D, Pearse R, Peden C, Sayers R, Wigfull J: The Higher Risk General Surgical Patient: Towards Improved Care for a Forgotten Group. London: Royal College of Surgeons of England, Department of Health; 2011.
 Shah N, Hamilton M: Clinical Review: Can we predict which patients are at risk of complications following surgery? Critical Care 2013, 17:226
 Barnett S, Moonesinghe S: Clinical risk scores to guide peioperative management. Postgrad Med J 2011, 87:535-541.
 Rix T, Bates T: Pre-operative risk scores for the prediction of outcome in elderly people who require emergency surgery. World J Emerg Surg 2007, 2:16.
 Saklad M: Grading of patients for surgical procedures. Anesthesiology 1941, 2:281-284.
 Wolters U, Wolf T, Stutzer H, Schroder T: ASA classi cation and peri-operative variables as predictors of postoperative out-come. Br J Anaesth 1996, 77:217-222.
 Wolters U, Wolf T, Stutzer H, Schroder T, Pichlmaier H: Risk factors, complications, and outcome in surgery: a multivariate analysis. Eur J Surg 1997, 163:563-568.
 Akoh JA, Mathew AM, Chalmers JWT, Finlayson A, Auld GD: Audit of major gastrointestinal surgery in patients aged 80 years or over. J R Coll Surg Edin 1994, 39:208-213.
 Charlson ME, Pompei P, Ales KL, MacKenzie CR: A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987, 40:373-383.
 Moonesinghe S, Mythen M, Grocott M: High-risk surgery: epidemiology and outcomes. Anaesth Analg 2011, 112:891-901.
 Lee TH, Marcantonio ER, Mangione CM, Thomas EJ, Polanczyk CA, Cook EF, Sugarbaker DJ, Donaldson MC, Poss R, Ho KK, Ludwig LE, Pedan A, Goldman L: Derivation and prospective validation of a simple index for prediction of cardiac risk of major noncardiac surgery. Circulation 1999, 100:1043-1049.
 Knaus WA, Zimmerman JE, Wagner DP, Draper EA, Lawrence DE: APACHE– Acute Physiology and Chronic Health Evaluation: a physiologically based classi cation system. Crit Care Med 1981, 9:591-597.
 Knaus WA, Draper EA, Wagner DP, Zimmerman JE: APACHE II: a severity of disease classi cation system. Crit Care Med 1985, 13:818-829.
 Le Gall J-R, Lemeshow S, Saulnier F: A New Simpli ed Acute Physiology Score (SAPS II) based on a European/North American Multicenter Study. JAMA 1993, 270:2957-2963.
 Bota P, Lebuffe G, Robin E, Mayeur C, Fleyfel M, Vallet B: Comparison between POSSUM, ASA, SAPS II, and SOFA scores as predictors for hospital mortality in surgical patients admitted to postoperative acute care unit. EJA 2006, 23: p200
 Copeland GP, Jones D, Walters M: POSSUM: a scoring system for surgical audit. Br J Surg 1991, 78:355-360.
 Prytherch DR, Whiteley MS, Higgins B, Weaver PC, Prout WG, Powell SJ: POSSUM and Portsmouth POSSUM for predicting mortality. Physiological and Operative Severity Score for the enUmeration of mortality and morbidity. Br J Surg 1998, 85:1217-1220.
 Khuri SF, Daley J, Henderson WG, Barbour G, Lowry P, Irvin G: The National Veterans Administration Surgical Risk Study: risk adjustment for the comparative assessment of the quality of surgical care. J Am Coll Surg 1995, 180:519-531.
 Khuri SF, Daley J, Henderson WG, Hur K, Gibbs JO, Barbour G: Risk adjustment of the postoperative mortality rate for the comparative assessment of the quality of surgical care. J Am Coll Surg 1997, 185:315-327.
 Daley J, Khuri SF, Henderson WG, Hur K, Gibbs JO, Barbour G: Risk adjustment of the postoperative morbidity rate for the comparative assessment of the quality of surgical care. J Am Coll Surg 1997, 185:328-340.
 Khuri SF, Daley J, Henderson WG, Hur K, Hur K, Demakis J, Aust JB: The Department of Veterans A airs’ NSQIP. The rst national, validated, outcome-based, risk-adjusted, and peer-controlled programme for the measurement and enhancement of the quality of surgical care. Ann Surg 1998, 228:491-507.
 Khuri SF, Henderson WG, Daley J, Jonasson O, Jones RS, Campbell DA Jr: The Patient Safety in Surgery Study: background, study design, and patient populations. J Am Coll Surg 2007, 204:1089-1102.
 Henderson WG, Khuri SF, Mosca C, Fink AS, Hutler MM, Neumayer LA: Comparison of risk-adjusted 30-day postoperative mortality and morbidity in Department of Veterans A airs hospitals and selected university medical centres: general surgical operations in men. J Am Coll Surg 2007, 204:1103-1114.
 Birkmeyer JD, Shahian DM, Dimick JB, Finlayson SR, Flum DR, Ko CY, Hall BL: Blueprint for a new American College of Surgeons National Surgical Quality Improvement Program. J Am Coll Surg 2008, 207:777-782.
 Bilimoria, Karl Y. et al.Development and Evaluation of the Universal ACS NSQIP Surgical Risk Calculator: A Decision Aid and Informed Consent Tool for Patients and Surgeons. Journal of the American College of Surgeons , Volume 217 , Issue 5 , 833 - 842.e3
 Liu Y, Cohen ME, Hall BL, Ko CY, Bilimoria KY. Evaluation and enhancement of calibration in the ACS NSQIP Surgical Risk Calculator. Journal of the American College of Surgeon, (epub ahead of print).
 Findlay GP, Goodwin APL, Protopapa KL, Smith NCE, Mason M. Knowing the Risk: a Review of the Peri-Operative Care of Surgical Patients. National Confidential Enquiry into Patient Outcome and Death; London, 2011
 Protopapa KL, Simpson JC, Smith NCE, Moonesinghe SR. Development and validation of the Surgical Outcome Risk Tool (SORT). Br J Surg 2014. 101: 1774-1783.
 Older P, Hall A, Hader R. Cardiopulmonary exercise testing as a screening test for peri-operative management of major surgery in the elderly, Chest , 1999, vol. 116 (pg. 355-62)
 Murray P, Whiting P, Hutchinson SP, Ackroyd R, Stoddard CJ, Billings C. Pre-operative shuttle walking testing and outcome after oesophagogastrectomy, Br J Anaesth , 2007, vol. 99 (pg. 809-11)
 American Thoracic Society. ATS statement: guidelines for the six-minute walk test. Am J Respir Crit Care Med. 2002;166:111–7.
 Older P, Hall A, Hader R: Cardiopulmonary exercise testing as a screening test for perioperative management of major surgery in the elderly. Chest 1999, 116:355-362.
 Bayram AS, Candan T, Gebitekin C: Preoperative maximal exercise oxygen consumption test predicts postoperative pulmonary morbidity following major lung resection. Respirology 2007, 12:505-510.
 Bolliger CT, Jordan P, Solèr M, Stulz P, Grädel E, Skarvan K, Elsasser S, Gonon M, Wyser C, Tamm M, et al.: Exercise capacity as a predictor of postoperative complications in lung resection candidates. Am J Respir Crit Care Med 1995, 151:1472-1480.
 Carlisle J, Swart M: Mid-term survival after abdominal aortic aneurysm surgery predicted by cardiopulmonary exercise testing. Br J Surg 2007, 94:966-969.
 Epstein SK, Freeman RB, Khayat A, Unterborn JN, Pratt DS, Kaplan MM: Aerobic capacity is associated with 100-day outcome after hepatic transplantation. Liver Transplant 2004, 10:418-424.
 Nagamatsu Y, Yamana H, Fujita H, Hiraki H, Matsuo T, Mitsuoka M, Hayashi A, Kakegawa T: The simultaneous evaluation of preoperative cardiopulmonary functions of oesophageal cancer patients in the analysis of expired gas with exercise testing. Nippon Kyobu Geka Gakkai Zasshi 1994, 42:2037-2040.
 Smith TB, Stonell C, Purkayastha S, Paraskevas P: Cardiopulmonary exercise testing as a risk assessment method in non cardio-pulmonary surgery:a systematic review. Anaesthesia 2009, 64:883-893.
 Young EL, Karthikesalingam A, Huddart S, Pearse RM, Hinchliffe RJ, Loftus IM, Thompson MM, Holt PJ: A systematic review of the role of cardiopulmonary exercise testing in vascular surgery. Eur J Vasc Endovasc Surg 2012, 44:64-71.
 Levett D, Growcott M. Cardiopulmonary exercise testing, prehabilitation, and Enhanced Recovery After Surgery (ERAS). Can J Anaesth. 2015; 62: 131-142
 Amrock L, Deiner S. The implication of Frailty on Preoperative Risk Assessment. Curr Opin Anaesthesiol. 2014 Jun; 27 (3): 330-335.
 Rockwood K, Song X, Macknight C, Bergman H, Hogan D, McDowell I, Mitnitski A. A global clinical measure of fitness and frailty in elderly people. CMAJ. 2005 Aug; 173 (5): 489-495.
 Robinson T, Wu D, Pointer L, Dunn C, Cleveland J, Moss M. Simple Frailty Score Predicts Post-Operative Complications Across Surgical Specialties. Am J Surg. 2013 Oct; 206 (4): 544-550.
 Moppett I, Parker M, Griffiths R, Bowers T, White S, Moran C. Nottingham Hip Fracture Score: longitudinal and multi-centre assessment. BJA. 2012; 109 (4): 546-550.