Webhook Delivery: Fire and Forget Research

The Polling Problem

Traditional API requests are synchronous:

// Client must wait for response
const response = await fetch('/api/research', {
  method: 'POST',
  body: JSON.stringify({ query: "..." })
})

// Blocks for 30-40 seconds
const result = await response.json()

Problems:

• Client connection held open
• Timeout handling complexity
• Scaling challenges (many concurrent waits)
• Poor user experience (loading spinners)

The Webhook Solution

Deep Research API supports asynchronous webhook delivery:

How It Works

┌─────────────┐
│   Your      │
│   Agent     │
└──────┬──────┘
       │
       ▼ POST
┌─────────────┐
│  Deep       │
│  Research   │
│    API      │
└──────┬──────┘
       │
       │ 30-40s
       ▼
┌─────────────┐
│   Your      │
│  Webhook    │
│  Endpoint   │
└─────────────┘
       │
       ▼ POST
┌─────────────┐
│   Agent      │
│  Continues  │
└─────────────┘

Implementation

Register Webhook

const response = await deepResearch({
  query: "Your research query",
  webhook: "https://your-agent.com/api/research-callback",
  webhookSecret: "your_webhook_secret"
})

// Returns immediately with job ID
const { jobId } = response
console.log("Research started:", jobId)

// Agent continues without waiting

Handle Webhook

// Your webhook endpoint
app.post('/api/research-callback', async (req, res) => {
  const { jobId, result, status } = req.body
  
  // Verify webhook secret
  if (req.headers['x-webhook-secret'] !== webhookSecret) {
    return res.status(401).send('Invalid secret')
  }
  
  if (status === 'completed') {
    // Process research result
    console.log("Research complete:", result)
    
    // Update agent state
    await updateAgentState({ jobId, result })
  }
  
  res.status(200).send('OK')
})

Poll for Status (Optional)

// If webhook fails, you can poll
const status = await deepResearch.getStatus(jobId)

if (status === 'completed') {
  const result = await deepResearch.getResult(jobId)
  console.log("Result:", result)
}

Benefits

Non-Blocking

Agent doesn't wait for research:

// Agent can process other tasks
async function handleUserQuery(query: string) {
  const research = await deepResearch({
    query,
    webhook: "https://your-agent.com/callback"
  })
  
  // Agent continues immediately
  console.log("Research started:", research.jobId)
  
  // User gets immediate feedback
  return { status: "processing", jobId: research.jobId }
}

Better UX

Webhook enables real-time updates:

// Frontend receives updates via WebSocket
socket.on('research-progress', (data) => {
  if (data.status === 'completed') {
    displayResult(data.result)
  } else {
    updateProgress(data.progress)
  }
})

Scalability

No connection limits:

// Can start 1000s of research jobs
for (let i = 0; i < 1000; i++) {
  await deepResearch({
    query: `Query ${i}`,
    webhook: "https://your-agent.com/callback"
  })
}

// All start immediately
// Webhooks handle completion

Best Practices

Retry Logic

Handle webhook failures:

let retryCount = 0

app.post('/api/research-callback', async (req, res) => {
  try {
    await processWebhook(req.body)
    res.status(200).send('OK')
  } catch (error) {
    retryCount++
    
    if (retryCount < 3) {
      // Retry webhook delivery
      await retryWebhook(req.body, retryCount)
    }
    
    res.status(500).send('Error')
  }
})

Security

Verify webhook authenticity:

// Generate unique secret per job
const webhookSecret = crypto.randomBytes(32).toString('hex')

await deepResearch({
  query: "...",
  webhook: "https://your-agent.com/callback",
  webhookSecret
})

// Verify in webhook handler
if (req.headers['x-webhook-secret'] !== webhookSecret) {
  return res.status(401).send('Unauthorized')
}

Monitoring

Track webhook delivery:

// Log all webhook events
app.post('/api/research-callback', async (req, res) => {
  const { jobId, status, timestamp } = req.body
  
  await logWebhookEvent({
    jobId,
    status,
    timestamp,
    receivedAt: new Date()
  })
  
  res.status(200).send('OK')
})

Get Started

Eliminate polling with webhooks.

Get Your API Key

Read Webhook Documentation