When I started building agents in Elixir, I tried some of the frameworks that were already available. They were useful for getting a first agent running quickly, but when I tried to use the same ideas inside real applications, I started finding pieces that I wanted to control more directly.
I needed routing that could combine normal rules, embeddings, local classifiers, semantic cache, and an LLM when necessary. I wanted tool selection to stay close to the Elixir functions that would eventually be called. I also needed memory with a real lifecycle, browser interaction, plans that could change while a task was running, and a clear way for agents to communicate without tying them to one transport.
This is how Spectre gradually became a family of libraries rather than one large framework. The libraries can work together, but they do not all need to be installed and used at the same time. Spectre itself is the conversational runtime. Kinetic, Lens, Mnemonic, Directive, and Pulse each solve a different problem around it.
I have already written separate articles about Spectre Kinetic, Spectre Lens, and Spectre Mnemonic. In this article I want to explain how the pieces now fit together and why I chose this structure.
Spectre Is the Runtime Around One Agent
Spectre is the core library. Its job is to describe an agent and handle one conversation turn in a predictable way.
An Agent module can declare input processing, routes, prompts, actions, and policies through a small DSL. The application still provides the actual model client, storage, permissions, and business functions. I wanted the DSL to make the structure of an agent easy to read without moving normal application logic into a framework-specific world.
When an input arrives, Spectre normalizes it, restores the relevant state and memory, checks whether a policy is already waiting for an answer, and then routes the input. Routing can use several sources of evidence. A simple route might use a regular expression, while another application may also use embeddings, a local classifier, a semantic cache, or an LLM classifier. Spectre collects that evidence and chooses one route.
The result of a turn tells the host application what happened:
{:ok, turn} =
Spectre.turn(
MyApp.Agent,
input,
conversation_id: conversation_id,
turn_id: message_id
)
case turn.decision do
{:awaiting, awaitable, result} ->
present_question(awaitable, result)
{:needs, effect, result} ->
persist_and_execute(effect, result)
{:completed, completion, result} ->
handle_completion(completion, result)
{:reply, result} ->
send_reply(result.reply_text)
{:no_response, _result} ->
:ok
endThe important point is that routing and execution are separate. A route or a model can propose an action, but that does not mean the action has already happened.
If an action is protected, Spectre opens a deterministic policy and waits for an accepted or rejected answer. Approval changes the state of the effect, but it still does not execute it. The host first persists that state and then calls the real application function. This gives the application a normal place to apply authorization, transactions, audit rules, retries, and idempotency.
I built Spectre around OTP and ordinary Elixir modules because an agent inside a production application is not only a model call. It has conversations that can continue after a process restarts, replies that may arrive twice, actions that may fail after creating an external side effect, and policies that must not be bypassed by probabilistic routing.
Why I Use Kinetic Instead of MCP Inside the Application
One of the first decisions behind Spectre was not to use MCP as the main abstraction for local tools.
This does not mean that MCP has no use. MCP is useful when an external client needs a standard way to discover and call tools. For example, exposing part of an application to ChatGPT, an IDE, or another remote system is a valid use of MCP.
Inside the application, however, I already have the Elixir modules, functions, types, permissions, and supervision tree. I did not want the tool logic to move into a separate server or to send a large catalogue of schemas to the model whenever the agent needed to decide what to do.
Spectre Kinetic takes a different approach. Functions remain normal Elixir functions, and Action Language examples can live beside their documentation and typespecs. Kinetic extracts this information into a registry and uses it to map a compact action expression to a real function and its arguments.
For example, a model can produce:
SEND MAIL TO="ops@example.com" BODY="The deployment failed"
Kinetic can return a structured plan containing the selected function, mapped arguments, missing values, scores, and classifier results. It does not execute the function.
When Kinetic is used with Spectre, the flow is straightforward:
user input
-> Spectre chooses the appropriate flow and prompt
-> the model may produce Action Language
-> Kinetic maps it to an action candidate
-> Spectre creates an effect and applies its policy
-> the application decides when and how to execute it
This keeps the model useful for interpreting language while keeping execution inside the application. It also means Spectre is not dependent on Kinetic. Kinetic is used heavily when Action Language and local tool planning are useful, but Spectre can also stage actions directly from deterministic routes or work as a conversational runtime without it.
MCP and Kinetic can therefore exist in the same system. I see MCP as a possible external interface and Kinetic as an internal planning mechanism. They solve different problems.
The Libraries Around Spectre
The other libraries grew from problems that did not belong inside the core runtime.
| Library | Main responsibility |
|---|---|
| Spectre | Conversation routing, state, policies, effects, and execution boundaries |
| Spectre Kinetic | Local tool selection and Action Language planning |
| Spectre Lens | Browser perception and interaction |
| Spectre Mnemonic | Active and durable agent memory |
| Spectre Directive | Long-running missions and plans that can change |
| Spectre Pulse | Communication between agents over different transports |
Lens: browser access in a form an agent can use
Spectre Lens controls a browser and turns a page into a more useful representation for an agent: readable content, links, forms, interactive elements, page outlines, and focused sections.
It also treats web content as untrusted. Lens can provide context to an agent, but the decision to perform an important browser action still belongs to the application and can pass through a Spectre action or policy.
I kept Lens separate because browser state has its own lifecycle and failure modes. It should be possible to use Lens without Spectre, and Spectre should not need to become a browser framework.
Mnemonic: memory is more than similarity search
Spectre Mnemonic manages working and durable memory. Recent information can remain in ETS, while important records can be persisted and recalled later.
The part I care about most is that memory has scope, provenance, time, and state. A fact can become stale or contradicted. A memory can be pinned or forgotten. Information from one tenant or project should not appear in another. Mnemonic can also build observations and stable mental models from source-linked evidence.
Spectre can use Mnemonic through its memory boundary, but Mnemonic remains a separate engine. It is not meant to replace the application's database, and it does not decide how a conversation should be routed.
Directive: a plan should change when the agent learns something
Spectre Directive is for work that cannot be represented as one turn or one tool call.
My main requirement for Directive was that the plan must remain alive. After each step, the agent may have new information. It should be able to check whether the remaining plan is still aligned with the mission, remove work that is no longer useful, add a new step, or ask the user before continuing.
For example, an agent researching potential clients may initially plan to inspect several sources in detail. If the first steps already show that a company does not fit the requested technology or location, continuing the full review is a waste of time. The plan should adapt to what has been learned.
Directive supports both plans written in Elixir through its DSL and plans created dynamically by an agent. Both use the same mission model and transition rules. Directive owns the mission, the current plan, collected information, and the pending request. It does not own the model provider, memory engine, tools, or application policy.
This is also why a chain of Kinetic actions does not automatically become a workflow. Kinetic plans individual actions. Directive controls the larger mission and decides what should happen next.
Pulse: a simple communication protocol between agents
Spectre Pulse came from another practical need: I wanted specialized agents to exchange information and ask each other to perform work.
I did not want Pulse to become a shared task manager or another orchestration framework. Pulse should remain deliberately simple. It defines the message, logical sender and recipient, message type, correlation, contacts, and reachability. The same protocol can then be implemented over local PubSub, WebSocket, REST, distributed Erlang, TCP, gRPC, or another transport.
An agent can know a contact such as :tao without knowing whether Tao is a local process or a remote service. The Agent DSL uses pulsing to define its identity, contacts, and advertised capabilities, while pulse sends an information message, query, or request.
Pulse does not own the session, memory, workflow, or retry policy. If Agent A asks Agent B to perform research, Agent A remains responsible for remembering that request and deciding what to do with the response. Pulse only provides a common communication envelope and delivery boundary.
This is simpler than introducing a shared room or task process between every pair of agents, and it fits naturally with the BEAM, where processes already communicate through messages.
How the Pieces Can Work Together
Consider an agent that searches for potential business leads.
Spectre can handle the conversation with the user, classify the initial request, and collect required constraints such as location and business type. Directive can create and maintain the research plan. Lens can inspect relevant websites. Mnemonic can remember accepted examples, rejected patterns, and project-specific preferences.
If part of the research should be delegated to a specialized agent, Pulse can send that request without coupling the two agents to a specific transport. When the system later needs to save a lead, prepare a report, or send a message, Kinetic can map Action Language to the appropriate application function. Spectre still applies the required policy and returns the effect to the host for execution.
The Journal in Spectre can record why a route was selected, how a policy was resolved, and how an effect changed state. It is different from conversation history: its purpose is to explain the agent's decisions without storing full messages, action arguments, or private results by default.
Not every application needs this complete combination. A small support agent may use only Spectre. A browser task may use Lens directly. Another application may combine Spectre, Mnemonic, and Directive without using Pulse. Keeping the libraries separate lets the application choose the pieces it actually needs.
The Direction I Want for Spectre
My goal is not to hide the whole application behind an autonomous agent abstraction. I want to use models where they are useful: for understanding language, selecting a possible route, proposing actions, and revising a plan, while keeping important application decisions visible in Elixir.
The BEAM already gives me processes, supervision, message passing, isolation, and recovery. Spectre should use those strengths instead of recreating a separate runtime around the model.
The core should remain responsible for a clear conversation lifecycle. Kinetic should remain focused on local planning. Mnemonic should govern memory. Lens should provide perception. Directive should control missions that evolve over time. Pulse should stay a small protocol that lets independent agents communicate.
Together these libraries provide the parts I was missing when I first tried to build a production agent in Elixir. More importantly, they let me see where a decision was made, which component owns the state, and where an external effect is finally executed. That control is the main reason Spectre is structured this way.
Send via.chat
Receive form leads, send login codes, and route important alerts through WhatsApp or Telegram.
Get in Touch
Have a question or want to work together? Drop a message below.