Segment Rule Engine

Rule Evaluation Model

Segments use a declarative rule engine that translates JSONB rule definitions into SQL queries. Rules are evaluated against three data sources:

1. Forms — forms.values (JSONB)
2. Profile — custom_field_values.value (TEXT)
3. Appointments — appointments table (aggregates)

Rule Structure

Basic Rule Format

{
  "source": "form|profile|appointments",
  "op": "eq|neq|contains|in|gt|lt|gte|lte|exists|empty",
  "value": <any>
}

Source-Specific Fields

Form Rules

{
  "source": "form",
  "template_id": 5,
  "custom_field_id": 11,
  "op": "eq",
  "value": "Big pain"
}

Required:

• template_id — which form template to query
• custom_field_id — which custom field in forms.values JSONB (values keyed as field_{id})

Evaluation:

SELECT 1 FROM forms f
WHERE f.patient_person_id = (SELECT patient_person_id FROM patients WHERE id = :patient_id)
  AND f.organization_id = :org_id
  AND f.form_template_id = :template_id
  AND f.status IN ('completed', 'signed')
  AND f.values->>CONCAT('field_', :custom_field_id) <operator> :value
ORDER BY f.updated_at DESC
LIMIT 1

Multi-instance resolution: When a patient has multiple forms of the same template, the latest completed/signed form is used. pending and in_progress forms are ignored.

Profile Rules

{
  "source": "profile",
  "custom_field_id": 10,
  "op": "eq",
  "value": "Bucharest"
}

Required:

• custom_field_id — which custom field to query

Evaluation:

SELECT 1 FROM custom_field_values cfv
WHERE cfv.entity_type = 'patient'
  AND cfv.entity_id = :patient_id
  AND cfv.custom_field_id = :custom_field_id
  AND cfv.organization_id = :org_id
  AND cfv.value <operator> :value

Appointment Rules

Appointment rules don't use field_key. They aggregate the appointments table.

{
  "source": "appointments",
  "metric": "count",
  "op": "gte",
  "value": 2,
  "filters": {
    "status": "done"
  }
}

Metrics:

• count — number of appointments matching filters
• last_date — most recent appointment date matching filters

Filters:

• status — appointment status (done, upcoming, cancelled, etc.)
• template_id — specific appointment template
• after — started_at >= date
• before — started_at <= date

Evaluation (count):

SELECT COUNT(*) FROM appointments a
WHERE a.patient_person_id = (SELECT patient_person_id FROM patients WHERE id = :patient_id)
  AND a.organization_id = :org_id
  AND a.status = :filter_status
  <additional filter clauses>
HAVING COUNT(*) <operator> :value

Evaluation (last_date):

SELECT MAX(a.started_at) FROM appointments a
WHERE a.patient_person_id = (SELECT patient_person_id FROM patients WHERE id = :patient_id)
  AND a.organization_id = :org_id
  AND a.status = :filter_status
HAVING MAX(a.started_at) <operator> :value

---

Match Modes

match_mode: "all" (AND Logic)

Patient must match every rule.

{
  "match_mode": "all",
  "rules": [
    {"source": "profile", "custom_field_id": 10, "op": "eq", "value": "Bucharest"},
    {"source": "form", "template_id": 5, "custom_field_id": 11, "op": "eq", "value": "Big pain"}
  ]
}

SQL:

SELECT p.id FROM patients p
WHERE p.organization_id = :org_id
  AND <rule_1_condition>
  AND <rule_2_condition>

match_mode: "any" (OR Logic)

Patient must match at least one rule.

{
  "match_mode": "any",
  "rules": [
    {"source": "profile", "custom_field_id": 10, "op": "eq", "value": "Bucharest"},
    {"source": "profile", "custom_field_id": 10, "op": "eq", "value": "Cluj-Napoca"}
  ]
}

SQL:

SELECT p.id FROM patients p
WHERE p.organization_id = :org_id
  AND (<rule_1_condition> OR <rule_2_condition>)

---

Nested Rule Groups

For complex logic like "A AND (B OR C)", use nested groups:

{
  "match_mode": "all",
  "rules": [
    {
      "source": "profile",
      "custom_field_id": 10,
      "op": "eq",
      "value": "Bucharest"
    },
    {
      "group": true,
      "match_mode": "any",
      "rules": [
        {"source": "form", "template_id": 5, "custom_field_id": 11, "op": "eq", "value": "Big pain"},
        {"source": "form", "template_id": 5, "custom_field_id": 11, "op": "eq", "value": "Extreme pain"}
      ]
    }
  ]
}

Means: city = Bucharest AND (pain_level = "Big pain" OR pain_level = "Extreme pain")

SQL:

SELECT p.id FROM patients p
WHERE p.organization_id = :org_id
  AND <city_condition>
  AND (
    <pain_big_condition> OR <pain_extreme_condition>
  )

Nesting limit: Maximum 3 levels deep. Deeper nesting is rejected at validation time.

---

Multi-Instance Form Resolution

Problem

A patient can have multiple forms of the same template (e.g., multiple appointments each generating a "Pain Assessment" form). When evaluating a form rule, which instance is used?

Solution: Latest Completed/Signed Form Wins

func (e *ruleEvaluator) resolveFormValue(
    ctx context.Context,
    patientID int64,
    templateID int64,
    customFieldID int64,
) (*string, error) {
    var value *string
    fieldKey := fmt.Sprintf("field_%d", customFieldID)
    err := e.db.QueryRow(ctx, `
        SELECT values->>$1
        FROM forms
        WHERE patient_person_id = (SELECT patient_person_id FROM patients WHERE id = $2)
          AND form_template_id = $3
          AND organization_id = current_app_org_id()
          AND status IN ('completed', 'signed')
        ORDER BY updated_at DESC
        LIMIT 1
    `, fieldKey, patientID, templateID).Scan(&value)

    if errors.Is(err, pgx.ErrNoRows) {
        return nil, nil // No matching form found
    }
    return value, err
}

Behavior:

• Only completed and signed forms are considered (incomplete forms are ignored)
• Most recent form (by updated_at) represents the patient's current state
• If all forms are unsigned/incomplete, the patient doesn't match that rule
• Historical forms are preserved but don't affect current segment membership

Index support:

CREATE INDEX idx_forms_segment_lookup
    ON forms(user_id, form_template_id, status, updated_at DESC)
    WHERE status IN ('completed', 'signed');

Edge cases:

Scenario	Behavior
Patient has 3 completed forms of template 5	Use value from most recent
Patient has 1 pending, 1 completed	Use value from completed form
Patient has 2 pending forms, 0 completed	Rule doesn't match (no value found)
Form field doesn't exist in latest form	Rule doesn't match (NULL value)
Latest form has empty value for field	Depends on operator (see operators.md)

---

Relative Date Operators

Instead of hardcoded dates, use relative date syntax:

{"source": "appointments", "metric": "last_date", "op": "gte", "value": "now-30d"}

Supported tokens:

Token	Meaning
now	Current timestamp
now-Nd	N days ago
now-NM	N months ago
now-Ny	N years ago
now+Nd	N days from now

Examples:

// Patients with appointments in last 30 days
{"source": "appointments", "metric": "last_date", "op": "gte", "value": "now-30d"}

// Patients with appointments in last 6 months
{"source": "appointments", "metric": "last_date", "op": "gte", "value": "now-6M"}

// Patients with no appointments in last year
{"source": "appointments", "metric": "last_date", "op": "lt", "value": "now-1y"}

// Profile field: registered in last 90 days
{"source": "profile", "custom_field_id": 15, "op": "gte", "value": "now-90d"}

Resolution:

Relative dates are resolved at evaluation time, not at rule creation time. This means:

• "Last 30 days" is always relative to now
• Segments automatically stay current without rule updates
• matched_at timestamp shows when the patient was last confirmed to match

Implementation:

var relativeDateRe = regexp.MustCompile(`^now([+-])(\d+)([dMy])$`)

func resolveValue(raw any) (any, error) {
    str, ok := raw.(string)
    if !ok || !strings.HasPrefix(str, "now") {
        return raw, nil
    }

    if str == "now" {
        return time.Now().UTC(), nil
    }

    matches := relativeDateRe.FindStringSubmatch(str)
    if matches == nil {
        return nil, fmt.Errorf("invalid relative date: %s", str)
    }

    n, _ := strconv.Atoi(matches[2])
    if matches[1] == "-" {
        n = -n
    }

    now := time.Now().UTC()
    switch matches[3] {
    case "d":
        return now.AddDate(0, 0, n), nil
    case "M":
        return now.AddDate(0, n, 0), nil
    case "y":
        return now.AddDate(n, 0, 0), nil
    }
}

---

Rule Validation

When creating or updating a segment, rules are validated:

Validation Checks

1. Source exists and is supported (form, profile, appointments)
2. Operator is supported for that source (see operators.md)
3. Required fields present:
   • Form rules: template_id, custom_field_id
   • Profile rules: custom_field_id
   • Appointment rules: metric, optional filters
4. Referenced entities exist:
   • Form template exists in organization
   • Custom field exists in organization (for form and profile rules)
   • Custom field matches entity type (patient for patient segments)
5. Value type matches operator:
   • in operator: value must be array
   • gt, lt, gte, lte: value must be number or date
   • exists, empty: value is ignored
6. Nesting depth ≤ 3 levels (for nested groups)

Validation Example

func (v *ruleValidator) validateRule(ctx context.Context, rule RuleNode, depth int) error {
    if depth > 3 {
        return fmt.Errorf("nesting depth exceeds maximum of 3")
    }

    if rule.Group {
        for _, child := range rule.Rules {
            if err := v.validateRule(ctx, child, depth+1); err != nil {
                return err
            }
        }
        return nil
    }

    // Validate leaf rule
    switch rule.Source {
    case "form":
        if rule.TemplateID == nil {
            return fmt.Errorf("form rule missing template_id")
        }
        template, err := v.templateRepo.Get(ctx, *rule.TemplateID)
        if err != nil {
            return fmt.Errorf("form template %d not found", *rule.TemplateID)
        }
        if !hasField(template.Fields, rule.FieldKey) {
            return fmt.Errorf("field %q not found in template %d", rule.FieldKey, *rule.TemplateID)
        }

    case "profile":
        field, err := v.customFieldRepo.FindByKey(ctx, rule.FieldKey, "patient")
        if err != nil {
            return fmt.Errorf("custom field %q not found", rule.FieldKey)
        }

    case "appointments":
        if rule.Metric != "count" && rule.Metric != "last_date" {
            return fmt.Errorf("invalid appointment metric: %s", rule.Metric)
        }

    default:
        return fmt.Errorf("unsupported rule source: %s", rule.Source)
    }

    if !isValidOperator(rule.Op) {
        return fmt.Errorf("unsupported operator: %s", rule.Op)
    }

    return nil
}

---

Rule Evaluation Flow

Per-Patient Evaluation (Tier 1/2)

Used when a patient's data changes (form save, profile update):

func (e *ruleEvaluator) EvaluateForPatient(ctx context.Context, patientID int64, segment Segment) (bool, error) {
    return e.evaluateNode(ctx, RuleNode{
        Group:     true,
        MatchMode: segment.MatchMode,
        Rules:     segment.Rules,
    }, patientID)
}

func (e *ruleEvaluator) evaluateNode(ctx context.Context, node RuleNode, patientID int64) (bool, error) {
    if node.Group {
        return e.evaluateGroup(ctx, node.MatchMode, node.Rules, patientID)
    }
    return e.evaluateLeaf(ctx, node, patientID)
}

func (e *ruleEvaluator) evaluateGroup(ctx context.Context, mode string, rules []RuleNode, patientID int64) (bool, error) {
    for _, rule := range rules {
        matched, err := e.evaluateNode(ctx, rule, patientID)
        if err != nil {
            return false, err
        }
        if mode == "any" && matched {
            return true, nil // Short-circuit OR
        }
        if mode == "all" && !matched {
            return false, nil // Short-circuit AND
        }
    }
    return mode == "all", nil
}

Bulk Evaluation (Tier 3)

Used for full segment rebuilds:

func (e *ruleEvaluator) BuildBulkQuery(rules []RuleNode, matchMode string, orgID int64) (string, []any) {
    query := "SELECT p.id FROM patients p WHERE p.organization_id = $1"
    args := []any{orgID}

    conditions := e.buildConditions(rules, matchMode, &args)
    query += " AND " + conditions

    return query, args
}

Generated SQL example:

SELECT p.id
FROM patients p
WHERE p.organization_id = $1

  -- Profile rule: city = Bucharest
  AND EXISTS (
    SELECT 1 FROM custom_field_values cfv
    JOIN custom_fields cf ON cf.id = cfv.custom_field_id
    WHERE cfv.entity_type = 'patient' AND cfv.entity_id = p.id
      AND cf.key = $2 AND cfv.value = $3
  )

  -- Form rule: pain_level = Big pain
  AND EXISTS (
    SELECT 1 FROM forms f
    WHERE f.patient_person_id = p.patient_person_id
      AND f.organization_id = $1
      AND f.form_template_id = $4
      AND f.status IN ('completed', 'signed')
      AND f.values->>$5 = $6
    ORDER BY f.updated_at DESC
    LIMIT 1
  )

  -- Appointment rule: count >= 2, status = done
  AND (
    SELECT COUNT(*) FROM appointments a
    WHERE a.patient_person_id = p.patient_person_id
      AND a.organization_id = $1
      AND a.status = $7
  ) >= $8

---

Error Handling

Evaluation Errors

When rule evaluation fails for a patient:

matched, err := evaluator.EvaluateForPatient(ctx, patientID, segment)
if err != nil {
    logger.Error("segment evaluation failed",
        "segment_id", segment.ID,
        "patient_id", patientID,
        "error", err,
    )

    // Don't fail the parent operation (e.g., form save)
    // Segment membership will be stale but data is saved
    // Enqueue for retry or manual investigation
    return nil // Continue processing
}

Principle: Segment evaluation failures should never block data writes (form saves, profile updates). Stale segment membership is acceptable; data loss is not.

Partial Evaluation

When evaluating multiple segments, failures are isolated:

for _, segment := range segments {
    matched, err := evaluator.EvaluateForPatient(ctx, patientID, segment)
    if err != nil {
        logger.Error("segment eval failed, skipping", "segment_id", segment.ID, "error", err)
        continue // Skip this segment, evaluate others
    }
    updateMembership(ctx, segment.ID, patientID, matched)
}

Result: Some segments update successfully, others remain stale. Monitoring alerts on evaluation errors trigger investigation.

---

Testing Strategy

Unit Tests: Operators

Test each operator in isolation:

func TestOperatorEq(t *testing.T) {
    rule := RuleNode{Source: "profile", FieldKey: "city", Op: "eq", Value: "Bucharest"}
    matched := evaluator.evaluateLeaf(ctx, rule, patientWithCityBucharest)
    assert.True(t, matched)
}

Integration Tests: Multi-Source Rules

Test rules that combine forms + profile + appointments:

func TestComplexSegment(t *testing.T) {
    segment := createSegment(t, []RuleNode{
        {Source: "profile", FieldKey: "city", Op: "eq", Value: "Bucharest"},
        {Source: "form", TemplateID: ptr(5), FieldKey: "pain_level", Op: "eq", Value: "Big pain"},
        {Source: "appointments", Metric: "count", Op: "gte", Value: 2},
    })

    patient := createPatientWithData(t, /* ... */)
    matched := evaluator.EvaluateForPatient(ctx, patient.ID, segment)
    assert.True(t, matched)
}

Performance Tests: Bulk Evaluation

Benchmark full segment rebuild at scale:

func BenchmarkBulkEvaluation(b *testing.B) {
    segment := createComplexSegment()
    patients := seedPatients(10000)

    b.ResetTimer()
    for i := 0; i < b.N; i++ {
        evaluator.BuildBulkQuery(segment.Rules, segment.MatchMode, orgID)
    }
}