Deep Research with Neon + Inngest Durable Endpoints

A recursive AI research workflow powered by Claude, Exa search, and Inngest durable step functions. Research sessions persist in Neon (PostgreSQL) and new research builds on past findings using pgvector semantic search.

Prerequisites

• Node.js 18+
• A Neon account
• API keys for: Anthropic, Exa, OpenAI

Setup

1. Install dependencies

npm install

2. Create a Neon database

Create a new project at console.neon.tech and copy your connection string.

3. Run the schema

Execute schema.sql against your Neon database. You can do this from the Neon SQL Editor in the console, or via psql:

psql "postgresql://user:password@your-host.neon.tech/neondb" -f schema.sql

4. Configure environment variables

Update .env.local with your actual values:

EXA_API_KEY=your-exa-api-key
ANTHROPIC_API_KEY=your-anthropic-api-key
DATABASE_URL=postgresql://user:password@your-host.neon.tech/neondb
OPENAI_API_KEY=your-openai-api-key

• DATABASE_URL - from your Neon project dashboard
• OPENAI_API_KEY - used for text-embedding-3-small embeddings (pgvector semantic search)

5. Run the dev server

npm run dev

Open http://localhost:3000.

How it works

1. Enter a research topic and answer clarification questions
2. The system recalls relevant findings from past research sessions (semantic search via pgvector)
3. Inngest durable steps execute recursive parallel searches (Claude + Exa)
4. Sources and learnings are persisted to Neon with embeddings after each depth level
5. A final report is generated with citations
6. Past sessions appear on the home screen and can be revisited

Project structure

src/
  app/
    page.tsx                          Main UI
    api/research/
      route.ts                        Durable research endpoint
      events/route.ts                 Polling endpoint for progress events
      clarify/route.ts                Clarification questions endpoint
      history/route.ts                Past sessions endpoint
  inngest/
    event-store.ts                    Neon-backed event store (replaces in-memory)
    deep-research.ts                  Recursive research algorithm with persist steps
    llm.ts                            Claude integration (queries, learnings, report)
    search.ts                         Exa search integration
    types.ts                          Shared type definitions
  lib/
    db.ts                             Neon client
    embeddings.ts                     OpenAI embedding utilities
  hooks/
    useResearch.ts                    Client state management and polling
  components/
    TopicInput.tsx                    Topic form + past sessions list
    ResearchProgress.tsx              Progress view with prior knowledge indicator
    ExecutionLog.tsx                  Real-time execution log
    ...
schema.sql                           Database schema (sessions, events, sources, learnings)