Metrics

This is an explanation of the metrics I use in various Zorgi scores, their relative importance, in my opinion, and if necessary, how they’re calculated. I also include common metrics used in various tables and charts in this site.

INDEX
  1. Percentage change daily cases
  2. Percentage change doubling days
  3. Prevalence index
  4. Percentage change daily fatalities
  5. Percentage change fatality doubling days
  6. Percentage change daily patients
  7. Percentage change daily ICU patients
  8. Percentage change hospital utilization rate
  9. Percentage change ICU utilization rate
  10. Availability of testing results
  11. Percentage change daily testing
  12. Positivity rate end of period
  13. Tests per case
  14. Availability of hospital data
  15. Availability of ICU data
METRIC #1
Percentage change in daily cases
Cases typically lead the news headlines. They are important, since without a case, there is no measureable impact of COVID-19 until someone ends up in the hospital or dies. However, I do not weight cases as heavily as fatalities or big increases in HUR or IUR. I also think increases in daily fatalities rise in significance as the absolute number grows.
Relative Importance (Scale A,B,C): B
Used in County & State Zorgi Scores
Calculation: first, cases are smoothed using a 7 day rolling average. The change is the current day’s case count, minus the previous period’s count, divided by the previous period’s count.
METRIC #2
Percentage Change in Case Doubling Days
In the early days of the pandemic, doubling days are typically very small, since it’s much easier to go from 5 cases to 10 cases than it is to go from 10,000 to 20,000. As the case count grows, doubling days should increase as well, even if the change in daily cases is large. If doubling days start to plateau, or worse, drop, then that indicates some serious trouble may be ahead. The absolute number of cases at the beginning of the period modifies the calculation, since it is easier to be doubled.
Relative Importance (Scale A,B,C): A
Used in County & State Zorgi Scores
Calculation: The program that calculates the doubling day number starts with the current day, and goes backward until it finds a day where the number was half or less than the current number. That is the doubling day. This differs from doubling day formulas used in financial forecasting, as this is not a future prediction, but rather an accurate picture of how long it took for that day’s number to be doubled.

The doubling day number (DDN) from the current period is then compared to the previous period’s DDN:
(current DDN – prev. period DDN )/ prev. period DDN
METRIC #3
Prevalence Index
This metric is important because it indicates how large a statistical group you would have to assemble in order to find one person who had direct experience with COVID-19, either through testing positive, getting sick, being a patient, recovering, or dying. I you think of a typical extended family as having around 50 members, that meanse a prevalence index of < 50 indicates a very high percentage of the population has had direct contact with COVID-19.
Relative Importance (Scale A,B,C): C
Used in County & State Zorgi Scores
Calculation: The population of a locality is divided by the cumulative number of cases: population / cases.
METRIC #4
Percentage Change in Daily Fatalities
Death is the bottom line with COVID-19; if there were not fatalities, it would serious, but probably not a crisis requiring lockdown of the entire economy. Large changes in daily fatalities indicate a deteriorating situation. It also indicates that not just young people are getting the virus, but they are spreading it to older and vulnerable people.
Relative Importance (Scale A,B,C): A
Used in County & State Zorgi Scores
Calculation: Daily fatalities are the difference between the current day’s cumulative fatality count and the previous periods’s fatality count. A daily fatality number is based on a seven day rolling average of each day’s count of fatalities.
METRIC #5
Percentage Change in Fatality Doubling Days
See comments on #2 above. Fatality doubling days are treated more heavily because of the reasons stated in #4 above. If doubling days go below 15, that is a matter of urgent concern, as it could indicate that the hospital system has been overwhelmed.
Relative Importance (Scale A,B,C): A
Used in County & State Zorgi Scores
Calculation: See calculation of case doubling days above.
METRIC #6
Percentage Change in Daily Patients, Positive + Suspected
One of the main goals of the lockdown in every state was to preserve the functionality of the hospital system. Fortunately, the state of CA provides daily data showing the number of hospital beds available as well as daily patients. This is a lagging indicator, but very important, as it indicates the number of people who are symptomatic, as opposed to those who are presymptomatic or asymptomatic.
Relative Importance (Scale A,B,C): B
Used in CA County Zorgi Scores
Calculation: This is one of the few metrics that is not smoothed using a 7 day rolling average, at least for now.
(Today’s number – prev. period number) / prev. period number
METRIC #7
Percentage Change in Daily ICU Patients
See #6 above. This metric is even more important that #6, because such a high percentage of COVID patients in ICU status become fatalities. Also, ICU patients require a higher level of care, more medical equipment, more PPE, and pose a much higher risk to medical personnel. Higher ICU utilization also leads to stress and burnout amongst healthcare workers.
Relative Importance (Scale A,B,C): A
Used in CA County Zorgi Scores
Calculation: the daily ICU numbers are smoothed using a 7 day rolling average.
(Today’s number – prev. number) / prev. number
METRIC #8
Percentage Change in Hospital Utilization Rate (HUR)
The HUR & IUR are calculated from the total bed count and the ICU bed counts provided by the state. The HUR is simply the number of patients divided by the number of available beds. Note that when the HUR and IUR climb above normal levels, this places stress on staffing, PPE, and other functions of the hospital.
Relative Importance (Scale A,B,C): B
Used in CA County Zorgi Scores
Calculation: Since the hospital patient number isn’t using a 7 day moving average, this metric doesn’t either. The HUR is the number of patients, positive and suspected for COVID-19, divided by the total number of hospital beds. The percentage change is the difference between today’s number and the previous period’s number.
(Today’s HUR – prev. HUR) / prev. HUR
METRIC #9
Percentage Change in ICU Utilization Rate (IUR)
See #8 above. An increase in IUR is scored more heavily than HUR, for the simple reason that these patients are in a high degree of stress and may be on the verge of dying. If a hospital system gets stressed, it is typically the IUR that indicates that first.
Relative Importance (Scale A,B,C): A
Used in CA County Zorgi Scores
Calculation: ICU numbers are smoothed with a 7 day rolling average because there’s too much noise in the daily numbers. The total for a day combines both suspected and positively identified COVID patients in the ICU. The utilization rate is calculated differently from the HUR, because the state database doesn’t show total ICU beds; it only shows available beds. So the total bed count is derived from the sum of ICU beds available, plus the current number of ICU patients. The IUR is current ICU patients / (current ICU patients + ICU beds available). The percentage change is: (Today’s IUR – prev. IUR) / prev. IUR
METRIC #10
Testing Results Publicly Available?
There are two reasons why counties that provide testing data get 2 points deducted from their score. First, they might have them added right back if they’re not doing well on testing or their positivity rates go up. Second, I have found that localities that aren’t transparent with testing statistics in general aren’t taking the pandemic as seriously as the localities that are.
Relative Importance (Scale A,B,C): A
Used in County & State Zorgi Scores
Calculation: counties that provide an easily accessible, public interface to their data, on a historical basis, and make it available through an API or some other download mechanism.
METRIC #11
Percentage Change in Daily Testing
The ability to open up the economy is based on having an infrastructure of “test-trace-isolate”. If a locality isn’t testing, one leg of that three-legged stool doesn’t exist. If a locality has largely contained the pandemic, it’s testing rate may go way down, but it’s positivity rate will also generally fall.
Relative Importance (Scale A,B,C): C
Used in County & State Zorgi Scores
Calculation: The daily testing number (DTN) comes from a 7 day rolling average. The percentage change is:
(current DTN – prev. period DTN) / prev. period DTN
METRIC #12
Positivity Rate at End of Period
Even though the positivity rate is based on some “fuzzy” numbers, that should drive down the rate overall. If the rate is climbing, that can indicate some serious spread of the disease.

Another way of looking at this is Tests per Case, below.
Relative Importance (Scale A,B,C): A
Used in County & State Zorgi Scores
Calculation: Daily tests (DT) are smoothed using a 7 day rolling average. Same for daily cases (DC), so DC / DT.
Some localities provide positive and negative numbers for each day, based on the tests themselves; others do not. So to be consistent, I use the above calculation.
METRIC #13
Tests per Case
The more a country gets the pandemic under control, the higher this number should go. This is a good indication of whether a locality is doing the proper amount of testing. For example, at the beginning of July, the number for Brazil, which had skyrocketing cases and hardly any testing, was 2.7. The number of the US was 12.6. The number for Germany, which had the pandemic under control, was 29.7. See Our World in Data for more.

Note that this is just the positivity ratio in reverse. But it may be easier to comprehend, because as testing increases, positivity goes toward zero, whereas tests per case rises in whole numbers.
Relative Importance (Scale A,B,C): A
Used in All Zorgi Scores
Calculation: Cumulative Tests divided by cumulative cases.
METRIC #14
Hospital Data Publicly Available?
The publication of hospital data is a critical tool in evaluating the progress of a locality in containing the virus. In particular, states that practice this sort of data opacity have often been accused of hiding the seriousness of their situation from reporters, Florida and Georgia being good examples of that.
Relative Importance (Scale A,B,C): A
Used in Zorgi Scores for counties and states
Calculation: Localities like states and counties that should be publishing hospital data, specifically how many patients and how many beds they have, are penalized in two ways. First every locality that does publish their data has two points deducted from their score, partially to compensate for the possible addition of points when evaluating hospital metrics. If they only publish general hospital data, but not ICU data, they have one point deducted. Second, localities that don’t publish hospital data also have two points added to their score.
METRIC #15
ICU Data Publicly Available?
See #14 above. ICU Data in particular can help the public evaluate whether a crisis point has been reached in the hospital system. Also, when ICU’s are beyond capacity, there is a much higher possibility that people will die because they’re unable to get critical care for non-COVID problems.
Relative Importance (Scale A,B,C): A
Used in Zorgi Scores for counties and states
Calculation: Localities like states and counties that should be publishing hospital data, specifically how many patients and how many beds they have, are penalized in two ways. First every locality that does publish their data has two points deducted from their score, partially to compensate for the possible addition of points when evaluating hospital metrics. Second, localities that don’t publish hospital data also have one point added to their score.

Other metrics used in tables and charts