Patient Survey Score

Methodology, rigour and how we calculate the patient experience scores

The Patient Experience Survey has 30 questions in total, of which 22 questions are used to determine our Patient Survey Score. Five of the 22 questions are used as indicator questions that determine whether other questions are applicable to the person completing the survey. An example of this is Question: 12 which asks if pain medication was required. The answer to this question allows us to determine whether Questions 13 and 14 apply to that individual. The other 17 questions are grouped into 8 categories that each address a particular aspect of care in hospital.The remaining questions provide information about patient demographics, and are used to make valid analytical comparisons. The survey provides the opportunity for each patient to recognise individual nurses who stood out for that patient, through the survey to offer comments, and where permission is given, and where these comments relate directly to care in hospital, they are shared with the hospital.

The survey is not sent to:

  • Patients under 18 years of age
  • Patients admitted for mental health problems
  • Patients who were surveyed within the past four months after an admission to the same hospital.

The Patient Survey Score (PASS) is only calculated for hospitals that have 50 or more responses in a year. This minimum number of responses has been determined using statistical methods.

We currently only calculate scores for private hospitals. Day, state, sub-acute and rehabilitation facilities are excluded.

We also collect and review surveys from specialist hospitals that focus only on specific kinds of problems or procedures (e.g. eye surgery). We do not report their patient experience scores publicly.

The Discovery Patient Experience Survey is based on the HCAHPs survey, developed by the US Agency for Healthcare Research and Quality (AHRQ), and with their permission. In most of its text, the survey is almost a verbatim copy of the original, with the following changes.

  • Question 9 asks about the quietness of the patient’s room at night. A response option has been added:  “I was not in the hospital overnight”. This is because day case encounters are included in our survey sample but not in the US survey sample.
  • Response options to the question about the respondent’s highest level of education, changed to reflect the educational levels and qualifications in South Africa
  • We do not include a question about Spanish, Hispanic or Latino descent or about race.
  • We do included a question about home language
  • As noted above, we have included a section allowing the member to name a nurse who provided particularly good care to that person. This forms the initial criterion for selection of nurses considered for the Discovery Nursing Excellence Awards (DEAN).
  • Finally we also include a “comments” field to allow for free text feedback from our members about their experience in hospital.       

Using the “top box” (best possible) method and how we calculate the results

Each hospital’s patient experience score is calculated as a single number, that is, the proportion of “top box” (best possible) responses to the survey questions. The results from each hospital can be compared with benchmarks on the performance of hospitals across the whole country, the province in which the hospital is located, or with hospitals of the same size.

The top box method means that we show the percentage of people who give the best possible score. For example, we ask how often the area around a hospital room is quiet. The options in this case are “always”, “usually”, “sometimes” and “never”. We score hospitals based on the percentage of “always” responses.

Top Box relates to the most positive response.
Example of answers to survey questions:

The patient experience score we calculate is a single number – a proportion of “top box” (best possible) responses. Using a survey as the source of the data, we only have a sample of all the patients who visited that hospital. This experience score is representative of the sample.

The true patient experience we would obtain if we were to receive all the responses from the survey, lies somewhere in a range – this range is known as a confidence interval. The confidence interval is based on the hospital’s score variability as a result of low responses or true variability, as determined by the data.

When we make comparisons, we use statistical techniques to ensure that if a patient experience score for a hospital is higher or lower than the benchmark, the result is statistically significant. If the result does not meet the analytical criteria for statistical significance it is shown on the website by the use of an exclamation mark (!). These scores are still analytically valid, however the scores cannot be compared with certainty to the benchmark (i.e. higher or lower than the benchmark), due to the scores having a wide range as noted below.

The figure below is an example of a hospital score within its confidence interval:

In scoring the patient experience at hospitals, we follow a global methodology

  • We score each category in the survey separately (for example, nursing, doctors, discharge information, etc.) and combine the results to get an overall or summary score for the hospital.
  • When calculating an overall patient experience score for each hospital, we use statistical methods that emphasise the categories that differ the most between different hospitals. This weighting is meant to ensure the most important quality issues are recognised.
  • We adjust the results to reflect characteristics of patients responding to the survey. This ensures that when hospital results are compared, these comparisons take into account factors that may differ between hospitals that the hospitals themselves have no control over but do influence how hospitals are rated by patients. For example: the type of medical care needed (medical, surgical or obstetrics), demographics (age, gender, education level, language), longer (overnight) against day admissions, and response time after discharge from the hospital.

In addition to the “best possible” scoring, we provide an alternative view of Survey Question 21, which is the overall rating that a patient gives a hospital on a scale of 0 to 10. While the “best possible” score for this question analyses only responses of 9 and 10, the alternate view provides the average score of all responses for this question i.e. the score computed is an average of all responses from 0 to 10. The scores are shown as percentages. This alternate view is referred to as the “average score” for Survey Question 21 on the website and it is shown at a national level across all survey hospitals, and at an individual hospital level.

The methodology for analysing patient experience scores is in line with methods used internationally and has been reviewed by external, independent experts.

We will also acknowledge efforts that hospitals make to improve patient experience scores publically. Data updates will pay close attention to improvements and recognise hospitals with improved scores. Such improvements will occur as a result of many factors including national standards, creative ideas of empowered hospital staff and management, and quality improvement programmes such as Best Care Always.

About quality of care

Everyone wants, and expects, high quality healthcare – whether it is for a simple cold when you visit your family practitioner or for something more serious when you are admitted to hospital.

Quality care means doing the right thing, at the right time, in the right way

The influential US Institute of Medicine (IOM) defines quality of care as “the degree to which health services for individual populations increase the likelihood of desired health outcomes and care consistent with current professional knowledge1”. The IOM divides healthcare quality into six useful dimensions that can be translated into the aims of a high-quality healthcare system:

This correlates how quality is defined by the South African National Core Standards that includes patient rights, patient safety and care amongst other measurement aligned to the broader healthcare system.

The healthcare system is complex, so quality can be perceived differently and measurement depends on many factors. These include, for example, where the care takes place, the healthcare professionals involved, and the type of medical problems treated. There is sometimes variation in the interpretation and adoption of best practice which may impact quality of care. This variation needs to be understood as it can identify improvement opportunities that can be addressed through re-engineering ways of working, training and improving communication.

1 Berwick D. Institute of Healthcare Improvement.

Improving quality of care is everyone’s responsibility

Measuring patient experience can highlight opportunities where improvements can be made. Through the Patient Experience Survey, Discovery Health aims to improve the visibility of concerns raised by members of schemes under our management. Working with healthcare professionals, professional groups, hospitals, patients and families, we aim to participate in solving concerns to improve the patient experience and so improve the overall quality of care.

How does Your Hospital Score by Patient Experience?

Find out about Your Individual Hospital's Patient Survey Score

Patient Survey Summary Score

The Patient Survey Summary Score is a best possible or top box score.

Top Box relates to the most positive response as illustrated below.

Survey questions can be answered as follows:

Examples of answers to survey questions:

Top box methodology calculates the percentage of responses that are the best positive response for all questions surveyed.

By measuring the top box(e.g. ‘always’), and not responses such as ‘usually’ or ‘sometimes’, we acknowledge our commitment to achieving the highest levels of excellence.