Question

I am building a system in Clojure that consumes events in real time and acts on them according to how many similar messages were received recently. I would like to implement this using a recency score based on Newtonian cooling.

In other words, when an event arrives, I want to be able to assign it a score between 1.0 (never happened before, or “ambient temperature” in Newton’s equation) and 10.0 (hot hot hot, has occurred several times in the past minute).

I have a vague idea of what this data structure looks like – every “event type” is a map key, and every map value should contain some set of timestamps for previous events, and perhaps a running average of the current “heat” for that event type, but I can’t quite figure out how to start implementing beyond that. Specifically I’m having trouble figuring out how to go from Newton’s actual equation, which is very generic, and apply it to this specific scenario.

Does anyone have any pointers? Could someone suggest a simpler “recency score algorithm” to get me started, that could be replaced by Newtonian cooling down the road?

EDIT: Here is some clojure code! It refers to the events as letters but could obviously be repurposed to take any other sort of object.

(ns heater.core
    (:require [clojure.contrib.generic.math-functions :as math]))

(def letter-recency-map (ref {}))

(def MIN-TEMP 1.0)
(def MAX-TEMP 10.0)
;; Cooling time is 15 seconds
(def COOLING-TIME 15000)
;; Events required to reach max heat
(def EVENTS-TO-HEAT 5.0)

(defn temp-since [t since now]
    (+
        MIN-TEMP
        (*
            (math/exp (/
                (- (- now since))
                COOLING-TIME))
            (- t MIN-TEMP))))

(defn temp-post-event [temp-pre-event]
    (+ temp-pre-event
        (/
            (- MAX-TEMP temp-pre-event)
            EVENTS-TO-HEAT)))

(defn get-letter-heat [letter]
        (dosync
            (let [heat-record (get (ensure letter-recency-map) letter)]
            (if (= heat-record nil)
                (do
                (alter letter-recency-map conj {letter {:time (System/currentTimeMillis) :heat 1.0}})
                MIN-TEMP)
                (let [now (System/currentTimeMillis)
                     new-temp-cooled (temp-since (:heat heat-record) (:time heat-record) now)
                     new-temp-event (temp-post-event new-temp-cooled)]
                     (alter letter-recency-map conj {letter {:time now :heat new-temp-event}})
                    new-temp-event)))))

Answer 1

In absence of any events, the solution to the cooling equation is exponential decay. Say T_0 is the temperature at the beginning of a cooling period, and dt is the timestep (computed from system time or whatever) since you evaluated the temperature to be T_0:

T_no_events(dt) = T_min + (T_0 - T_min)*exp(- dt / t_cooling)

Since your events are discrete impulses, and you have a maximum temperature, you want a given ratio per event:

T_post_event = T_pre_event + (T_max - T_pre_event) / num_events_to_heat

Some notes:

• t_cooling is the time for things to cool off by a factor of 1/e = 1/(2.718...).

• num_events_to_heat is the number of events required to have an effect comparable to T_max. It should probably be a moderately large positive value (say 5.0 or more?). Note that if num_events_to_heat==1.0, every event will reset the temperature to T_max, which isn’t very interesting, so the value should at the very least be greater than one.

• Theoretically, both heating and cooling should never quite reach the maximum and minimum temperatures, respectively (assuming parameters are set as above, and that you’re starting somewhere in between). In practice, though, the exponential nature of the process should get close enough as makes no difference…

• To implement this, you only need to store the timestamp and temperature of the last update. When you receive an event, do a cooling step, then a heating event, and update with the new temperature and timestamp.

• Note that read-only queries don’t require an update: you can just compute the cooling since the last update.