Lab note #059 Effects type checking
With the release of OpenAI's Deep Research and its testimonies on Twitter, I finally slapped down $200/month for Pro subscription. Deep research is what I do all the time by hand anyway, I figured I'd get a lot of mileage out of it. The second report I had it generate was comparing and contrasting algebraic effect handler lookup in various different languages that supported it natively.
I was pretty satisfied with the quality of the report. Reading it through took a while, but it gave me a good sense of how each language did things. I spot checked what it wrote here and there, and it all seemed to be on point.
Hence, I had meant to read more of the effects papers, but I went ahead with an implementation first. I'm mildly satisfied with the progress with this week. Python isn't the best language for a full-blown algebraic effect, since it lacks delimited continuations. However, it's the language of choice for AI pipelines, so I'm stuck with it for the time being.
I was pleasantly surprised that Python had type checking as an optional library. It's a bit clunky, but it's dragging mainstream programmers halfway to Haskell-like type syntax. With mypy type checking, it feels on par to having Typescript for dev-time type checking, albeit the syntax is better than Typescript's typing.
It took a bit of help from GPT to get up to speed on the type checking syntax in Python, but I was able to get it so that for any effectful generator function, you can type annotate it, to make sure that you don't raise any effects you didn't intend to. However, I can't check that all intended effects were raised, since that would be a runtime check.
On the handler side, I was able to leverage Protocols to make sure that we had all the handlers implemented as indicated in the trait/interface which Unison called an ability. I've written the drivers in the middle of the reactive decorators, and am in the process of implementing nesting handlers.
Here is an example of a StateAbility with its handler, StateHandler.
@dataclass
class Get(Effect):
pass
@dataclass
class Put[T](Effect):
new_state: T
# An ability is the trait or interface for handlers
class StateAbility[T](Ability, Protocol):
def get(self, effect: Get) -> T: ...
def put(self, effect: Put[T]) -> None: ...
class StateHandler[T]():
state: T
def __init__(self, initial: T):
self.state = initial
def get(self, _effect: Get) -> T:
return self.state
def put(self, effect: Put[T]) -> None:
self.state = effect.new_stateThe reactive sources of state is defined as:
class MyState:
@track
def a(self):
return 2
@track
def b(self):
return 3And the reactive computational graph can be an effectful node. With it, you can raise two effects, Get and Put.
self.state = MyState()
@reactive
def get_and_put_multiplier(self) -> EffectfulGen[Get | Put, int]:
multiplier: int = yield Get()
yield Put(multiplier + 2)
multiplier = yield Get()
return multiplier * self.state.aAnd finally, the handler can be used to execute the reactive node's effects, when we wrap it in the handler's block.
def test_get_and_put_effect(self) -> None:
handler: StateAbility = StateHandler(3)
with useHandler(handler):
res = self.get_and_put_multiplier()
self.assertEqual(res, 10)It's a little awkward to typecheck that the handler has all the necessary methods implemented in the ability by casting it to a StateAbility before putting it into useHandler. But it'll suffice for now. In addition the useHandler might be extraneous. I might be able to make the handler a ContextDecorator itself, but I'm not sure what the consequences are yet.
So far, I'm pleased with the progress, especially when I had to restart the whole thing at the end of December. In hindsight, one decision that overly complicated the implementation was trying to keep the runtime in Javascript and the rest in Python in anticipation of supporting multiple languages. It's much much easier when there's no explicit boundary between two parts of a system.