Custom Enrichment Sources

Calseta’s enrichment system is runtime-configurable — you can add any HTTP-accessible service as an enrichment provider without writing code. This makes it straightforward to use Azure Logic Apps, AWS Lambda functions, or internal APIs as custom enrichment sources.

The Pattern

Build an HTTP endpoint (Logic App, Lambda, internal API) that accepts an indicator and returns structured data
Register it as a provider in Calseta via the REST API
Configure field extractions to tell Calseta which fields to surface to agents
Configure malice rules to map response fields to verdicts

Your endpoint does the heavy lifting — querying internal databases, calling proprietary APIs, running custom logic. Calseta handles the orchestration: calling your endpoint during enrichment, caching results, extracting fields, and setting verdicts.

Example: Internal Threat Intel via Logic App

Suppose you have an internal threat intel database exposed through an Azure Logic App. The Logic App accepts an IP address and returns context from your internal systems.

Step 1: Build the Logic App

Your Logic App accepts a POST request with the indicator value and returns JSON: Request:

POST https://your-logic-app.azurewebsites.net/api/lookup
{
  "indicator_type": "ip",
  "value": "185.220.101.47"
}

Response:

{
  "found": true,
  "risk_score": 85,
  "internal_hits": 12,
  "first_seen_internal": "2024-11-03T14:22:00Z",
  "tags": ["known-scanner", "blocklist-candidate"],
  "source": "internal-threatintel-db"
}

Step 2: Register the Provider

curl -X POST http://localhost:8000/v1/enrichment-providers \
  -H "Authorization: Bearer cai_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "provider_name": "internal_threat_intel",
    "display_name": "Internal Threat Intel",
    "supported_indicator_types": ["ip", "domain"],
    "http_config": {
      "steps": [{
        "method": "POST",
        "url": "https://your-logic-app.azurewebsites.net/api/lookup",
        "headers": {
          "Content-Type": "application/json",
          "x-api-key": "{{auth_token}}"
        },
        "body": {
          "indicator_type": "{{type}}",
          "value": "{{value}}"
        }
      }]
    },
    "auth_config": {
      "token": "your-logic-app-api-key"
    },
    "malice_rules": {
      "conditions": [
        { "field": "risk_score", "operator": "gte", "value": 80, "verdict": "Malicious" },
        { "field": "risk_score", "operator": "gte", "value": 40, "verdict": "Suspicious" },
        { "field": "found", "operator": "eq", "value": false, "verdict": "Benign" }
      ],
      "default_verdict": "Benign"
    },
    "cache_ttl_by_type": {
      "ip": 1800,
      "domain": 3600
    }
  }'

Step 3: Configure Field Extractions

Tell Calseta which fields from your Logic App’s response to surface to agents:

curl -X POST http://localhost:8000/v1/enrichment-field-extractions \
  -H "Authorization: Bearer cai_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "provider_name": "internal_threat_intel",
    "indicator_type": "ip",
    "source_path": "risk_score",
    "target_key": "risk_score",
    "value_type": "integer",
    "description": "Internal risk score (0-100)"
  }'

For multiple extractions, use the bulk create endpoint to add them all at once:

curl -X POST http://localhost:8000/v1/enrichment-field-extractions/bulk \
  -H "Authorization: Bearer cai_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "extractions": [
      {
        "provider_name": "internal_threat_intel",
        "indicator_type": "ip",
        "source_path": "internal_hits",
        "target_key": "internal_hits",
        "value_type": "int",
        "description": "Number of times seen in internal logs"
      },
      {
        "provider_name": "internal_threat_intel",
        "indicator_type": "ip",
        "source_path": "tags",
        "target_key": "tags",
        "value_type": "list",
        "description": "Internal classification tags"
      }
    ]
  }'

You can also manage extractions after creation — list, update, or delete them:

# List all extractions for a provider
curl http://localhost:8000/v1/enrichment-field-extractions?provider_name=internal_threat_intel \
  -H "Authorization: Bearer cai_your_api_key"

# Toggle an extraction off
curl -X PATCH http://localhost:8000/v1/enrichment-field-extractions/{uuid} \
  -H "Authorization: Bearer cai_your_api_key" \
  -d '{"is_active": false}'

# Delete a custom extraction
curl -X DELETE http://localhost:8000/v1/enrichment-field-extractions/{uuid} \
  -H "Authorization: Bearer cai_your_api_key"

Builtin providers ship with system extractions that cannot be deleted — only toggled on/off. Custom extractions you create can be fully edited or removed. When you delete a provider, all its field extractions are automatically cleaned up.

Step 4: Test It

Trigger on-demand enrichment:

curl -X POST http://localhost:8000/v1/enrichments \
  -H "Authorization: Bearer cai_your_api_key" \
  -d '{"type": "ip", "value": "185.220.101.47"}'

Check the results:

curl http://localhost:8000/v1/enrichments/ip/185.220.101.47 \
  -H "Authorization: Bearer cai_your_api_key"

The response includes your custom provider alongside builtins:

{
  "enrichment_results": {
    "virustotal": {
      "extracted": { "malicious_count": 14 },
      "success": true
    },
    "internal_threat_intel": {
      "extracted": {
        "risk_score": 85,
        "internal_hits": 12,
        "tags": ["known-scanner", "blocklist-candidate"]
      },
      "success": true
    }
  }
}

Example: AWS Lambda for CMDB Lookup

An AWS Lambda that checks your CMDB for asset context on IP addresses:

curl -X POST http://localhost:8000/v1/enrichment-providers \
  -H "Authorization: Bearer cai_your_api_key" \
  -d '{
    "provider_name": "cmdb_lookup",
    "display_name": "CMDB Asset Lookup",
    "supported_indicator_types": ["ip"],
    "http_config": {
      "steps": [{
        "method": "GET",
        "url": "https://abc123.execute-api.us-east-1.amazonaws.com/prod/lookup?ip={{value}}",
        "headers": {
          "x-api-key": "{{auth_token}}"
        }
      }]
    },
    "auth_config": {
      "token": "your-api-gateway-key"
    },
    "malice_rules": {
      "conditions": [],
      "default_verdict": "Benign"
    },
    "cache_ttl_by_type": { "ip": 3600 }
  }'

If your custom source provides context rather than threat intelligence (like a CMDB), set default_verdict to "Benign" with no conditions. The provider adds context without affecting malice verdicts.

Response Format Requirements

Your endpoint must return valid JSON. Calseta’s field extraction system uses dot-notation paths (data.results.score) to extract values from the response, so you can nest data however you like. If your endpoint returns an error (non-2xx status code), Calseta records success: false for that provider and moves on — it never blocks other providers.

What Agents See

Agents receive the extracted fields from all providers — builtin and custom — in the same structure. Your internal threat intel data appears alongside VirusTotal and AbuseIPDB results, giving agents a complete picture without custom integration code.

Get Started

Concepts

Development

Integrations

Operations

Contributing

Custom Enrichment Sources

The Pattern

Example: Internal Threat Intel via Logic App

Step 1: Build the Logic App

Step 2: Register the Provider

Step 3: Configure Field Extractions

Step 4: Test It

Example: AWS Lambda for CMDB Lookup

Response Format Requirements

What Agents See

​The Pattern

​Example: Internal Threat Intel via Logic App

​Step 1: Build the Logic App

​Step 2: Register the Provider

​Step 3: Configure Field Extractions

​Step 4: Test It

​Example: AWS Lambda for CMDB Lookup

​Response Format Requirements

​What Agents See

The Pattern

Example: Internal Threat Intel via Logic App

Step 1: Build the Logic App

Step 2: Register the Provider

Step 3: Configure Field Extractions

Step 4: Test It

Example: AWS Lambda for CMDB Lookup

Response Format Requirements

What Agents See