NeuroTrace

A Reproducible Analytics Framework for Cognitive AI

NeuroTrace is a deterministic, file-first analytics framework I built to investigate how personalization emerges in memory-enabled AI systems. It transforms raw conversational logs into structured, analyzable data, allowing for a graph-based exploration of an AI’s cognitive growth and behavioral patterns over time. The entire framework was designed to serve as the technical engine for my research on AI personalization.
A key visual from the research, illustrating the conceptual evolution of an AI's cognitive functions over time, from baseline Q&A to memory synthesis.
The conceptual model of cognitive zones (Planning, Feedback, Reflection, Memory) used to abstract and interpret assistant behaviors during analysis.

The Problem: Understanding AI’s Cognitive Growth

While memory-enabled AI systems can adapt to users, there is limited understanding of how this personalization actually unfolds over time. Most AI evaluations focus on performance benchmarks at discrete points but do not examine the system’s cognitive growth trajectory. I built NeuroTrace to address that gap by providing a tool to model and measure this evolution structurally.

How It Works: A Deterministic Analytics Pipeline

NeuroTrace is a CLI tool that runs a complete, one-shot pipeline: it ingests raw JSON conversation logs, computes stable derived artifacts, and generates visualizations and reports. The framework is built on two core principles:

Modular Analytics

A suite of computational modules processes the ingested data to analyze distinct cognitive dimensions. The key modules include:
  • Tagging: Assigns cognitive zones (e.g., Planning, Reflection, Feedback, Memory) to assistant messages.
  • Zone Transitions: Builds transition matrices to model how behaviors shift across conversations.
  • Response Depth: Measures the structural complexity of assistant outputs over time.
  • Semantic Memory: Identifies memory-like events and reactivation of past behaviors across sessions.

Guaranteed Reproducibility

For research to be credible, its methods must be reproducible. NeuroTrace is designed to be fully deterministic:
  • File-First: The pipeline ingests raw data and writes a series of stable JSON artifacts to disk. The API is strictly read-only and serves precomputed data.
  • Deterministic by Design: The framework guarantees that the same input will always produce the exact same output. It uses SHA-256 content hashing, sorted keys in JSON, and a frozen configuration with no environment variables.
Zone Distribution for a Single Session (67af2b): Illustrates the dominant cognitive roles within one user interaction. Planning accounts for 55.3% of assistant behaviors, indicating a highly structured, goal-oriented dialogue.
Behavioral Reinforcement: Zone Transition Matrix: A core metric showing the frequency of behavioral shifts. The dark blue cells (e.g., other → other, action → other) confirm strong self-loops and iterative refinement within specific functional roles.
Longitudinal Comparison of Cognitive Load: Compares the distribution of zones (Planning, Feedback, Memory, Reflection) across four distinct sessions. The variance suggests personalization leads to specialization, with some sessions heavily skewed toward Planning and Feedback.

Key Findings & Impact

The application of NeuroTrace resulted in the formal research paper, “Personalization in Memory-Enabled AI Systems.” The analysis of 835 conversations (~37,000 messages) provided measurable, data-driven insights into AI personalization:
This project serves as a complete loop from theory to tool to insight. It provided the technical means to conduct novel research and produced a reusable framework for future studies in human-AI interaction.
Read the Full Research PaperView on GitHub
Back to Featured Projects