Structured Numeric Support

From Humanitarian FOSS Summer Institute 2008

Overview

While numeric results for a clinical information system are either stored as whole integers or fractions, there are a number of numeric values that have more of a structured form. For example, ranges such as "5-10", "<=5" or ">10"; ratios such as "1:2"; and qualitative responses mixed in with numeric responses such as "too many to count". The project involves incorporating these type of numeric values (structured numeric) in OpenMRS system.

Workflow

Admin Page

• The user should be able to select structured numeric datatype while adding a concept and also additional metadata such as units and high-low ranges should be able to be specified

Add Observation

• On the Add Observation page, if a user selects a concept which involves a Structured Numeric datatype an input field with additional metadata (only if selected when concept was created) should appear and the user should be able to enter a structured numeric value as a text

Cohort

• When the user selects Cohort Builder, a user should be able to make queries with observations which involves structured numeric

Requirements

• Add structured numeric data type at the API level (obs and Concept) within OpenMRS
• Facilitate the use of numeric database queries on structured numerics

Specifications

• Requirement: Add structured numeric data type at the API level (obs and Concept) within OpenMRS
• Specification: Identify the parts of the API (model, view and controller) and make appropriate changes to accomodate structured numerics
  
  

• Requirement: Facilitate the use of numeric database queries on structured numerics
• Specification: Make changes in the cohortForm.jsp to allow a search for structured numerics and also make query modifications in HibernatePatientSetDAO to allow queries for structured numerics
  
  

Design Options

Introducing a new data type will involve adding a data type at the concept level and and a way to store it in the database. Following design options are proposed:

• One of the ways in which structured numeric data type can be implemented in the database level is by using value_modifier column which can store character of length 2 in the obs table to store comparator(<, <=, >, >=) and separator(-, :) and use value_numeric column to store numerics. One of the advantages of this approach is that the query can be run at the SQL level but the problem with this approach is that in the case of separators (eg. 2-3 or 2:3) two value_numeric columns will be required.The possible solution to this problem will be adding another value_numeric column and some form of coded abbreviation for qualitative responses (eg. too many to count). The obs table for some sample data such as >2, 1:2, 2-3 and too many to count will look like:
  

Another limitation of this approach will be to specify the type of ranges (inclusive, exclusive, inclusive-exclusive and exclusive-inclusive). For example, a range 10-20 can be inclusive on only one side, like [10, 20). But after doing some more research on HL7 side, they assume ranges to be inclusive on both sides.

• Another proposed solution is using the value_text column to store the structured numeric value as it will help to resolve the issue of qualitative responses and also it will avoid adding extra columns to the database. To store a structured numeric following way is recommended by HL7 which involves adding a carat (^) or hat sign.
  
  <comparator (ST)> ^ <num1 (NM)> ^ <separator/suffix (ST)> ^ <num2 (NM)>
  
  For example: 100-200 will be stored as ^100^-^200 and 2+ can be stored as ^2^+
  The only problem with this approach is to run queries at the database level. But, using the idea of pattern matching using regular expressions or string functions in MySQL, it could be done. This approach can be costly in terms of processing time because even in the case of displaying an observation, parsing might be involved.

Current Design

Database

After lot of brainstorming, the first option with some modifications was implemented. Two columns value_numeric_min and value_numeric max were added to the obs table. Also, the value_modifier column was modified to store a string up to 3 characters. The obs table for some sample data such as >2, <=2, 1:2, 2-3, [2,3), 12.1 and too many to count looks like this:

One could notice that you could specify the type of range (inclusive, exclusive, inclusive-exclusive and exclusive-inclusive) using appropriate use of parenthesis.

Validation

Right now, validation is done before parsing at the ObsValidator and basically it checks, if the structured numeric entered is in correct format. The rules right now is:

• If the value contains a comparator (<,>,<=,>=), it must start with it.
• The value after a comparator must be a numeric.
• The values before a separator (-, :) must be numerics and in case of range, can start with "(" or "[" and end with ")" or "]".

Parsing of Input

Parsing of input is done at the ObsFormController with the help of onBindAndValidate(override) method. Modifications are made in the Obs.java to set valueNumericMin and valueNumericMax.

Query Handling

HibernatePatientSetDAO.java was modified to allow queries for structured numeric. So, if a concept with structured numeric data type is selected, one can run queries. A simple query was written to allow queries to find all observations <, >, <=, >= and = value a user wants to. For example: if a user run queries for all obs with value <10, value such as <3, <=5, 2:3 and 4-8 will display. The query considers if the range is inclusive or exclusive. For example, in this case (4-10] won't be displayed.

Rules for Query:

• If the modifier of the query is < , and if it is a comparator, the modifier of the comparator needs to be either < or <=.
• If the modifier of the query is < , and if it is a range, the higher end of the range needs to be less than or equal to the value entered depending on the inclusiveness of the range.
• If the modifier of the query is < , and if it is a proportion or a numeric value, it is treated similar to a numeric query.

Similarly, the other modifiers (>, <=, >=, and =) are implemented.

Proposed Design (After developer's call)

Database

My understanding after the developer's call was to use value_modifier (which will be renamed to value_structured_numeric) to store the structured numeric in natural way and use value_numeric to store the numeric value. In case of ranges, value_numeric will store the average and in proportions, the exact value. The obs table for some sample data such as >2, <=2, 1:2, 2-3, [2,3), 12.1 and too many to count would look like this:

ConceptStructuredNumeric class

Also, we decided that we need a ConceptStructuredNumeric to support structured numerics. I could think of following methods:

getValueModifier(): string
setValueModifier(String svm): void
getValueNumeric(): double
setValueNumeric(double svn): void
isInequality(): boolean
getInequalitySign(): string
setInequalitySign(str sis): void
getNumericAfterInequality(): string
setInequalityAfterInequality(str siai): void
isProportion(): boolean
getProportionLow(): double
setProportionLow(double spl): void
getProportionHigh(): double
setProportionHigh(double sph): void
isRange(): boolean
getRangeLow(): double
setRangeLow(double srl): void
getRangeHigh(): double
setRangeHigh(double srh): void
parse (string sn): void
toString(): string

But, I still need to know about where this class will be written and if it will extend any class.

Query Handling

I guess query handling in this case would need using regexp to parse the value_modifier to know what type or structured numeric data type it is and get appropriate numeric values in there for query.

Developer

Vinit Agrawal
Email: vinit.agrawal@trincoll.edu

Mentor

Darius Jazayeri
Burke Mamlin