Feed API Documentation

What is Feed API?

When synchronizing two systems, you have two primary approaches:

card: Classical API
icon: refresh-cw
Read data via API, detect changes locally, and push only modifications. Requires change tracking and state management on the client side.
**Best for:**
- Real-time synchronization
- Frequent small updates
- Fine-grained control over changes
card: Feed API
icon: zap
Send one large JSON (even gigabytes) with all values. The system efficiently compares and updates only real changes.
**Best for:**
- Bulk data synchronization
- Periodic full syncs
- Simpler integration logic

How Feed API Works

1. Efficient Preload

The system efficiently preloads existing data for comparison using optimized database queries.

2. Intelligent Matching

Records are matched using lookups (code:, id:) or unique identifiers. Even items without unique IDs (like line items) are matched using a probability mechanism.

3. Change Detection

The system compares all data and identifies actual changes, eliminating false differences.

4. Selective Updates

Only real changes are written to the database, minimizing overhead and maintaining audit trails.

Core Components

Feed API leverages two powerful tools: Mapper and Importer

card: Mapper
icon: settings
Transforms external data formats (XML, JSON, CSV) into the internal format. Handles field mapping, data transformations, and type conversions.
**Key Features:**
- JSONPath and XPath support
- Expression evaluation
- Lookup resolution
- Nested entity mapping
[navigate:/devdoc/mapper|View Docs|book-open]
card: Importer
icon: upload
Processes mapped data and intelligently updates the database. Handles upserts, deletions, and relationship management.
**Key Features:**
- Upsert operations
- Lookup-based matching
- Probabilistic matching
- Batch processing
[navigate:/devdoc/imports|View Docs|book-open]

Key Directives

Essential directives for Feed API operations

Lookups (code:, id:, etc.)

Identify records using various lookup types instead of database IDs.

"category": "code:ELECTRONICS"
"manufacturer": "id:12345"
"customer": "email:john@example.com"

@removeAll

Controls whether existing items not in the feed should be removed.

"items": [
  { "code": "ITEM-1", "quantity": 10 }
],
"items@removeAll": true  // Remove items not in feed

Default: true for array/object mode, false for merge mode

@operation

Explicitly control the operation type for each record.

{ "@operation": "upsert", "code": "PROD-1", "name": "Product 1" }
{ "@operation": "delete", "code": "PROD-OLD" }
{ "@operation": "create", "name": "New Product" }

Values: create, update, upsert, delete

@where

Mass operations using where clauses for bulk updates or deletions.

{
  "@operation": "update",
  "@where": "category = 'code:OLD_CAT'",
  "category": "code:NEW_CAT"
}

@atomic

Controls transaction behavior and error handling.

"@atomic": "batch"  // Recommended for large feeds
"@atomic": true     // All-or-nothing
"@atomic": false    // Independent records

Example: Products Feed

Complete example of synchronizing products via Feed API

1. Source Data (External System)

{
  "products": [
    {
      "productCode": "WIDGET-001",
      "productName": "Super Widget",
      "price": 99.99,
      "categoryCode": "ELECTRONICS",
      "variants": [
        {
          "sku": "WIDGET-001-RED",
          "color": "Red",
          "stock": 50
        },
        {
          "sku": "WIDGET-001-BLUE",
          "color": "Blue",
          "stock": 30
        }
      ]
    },
    {
      "productCode": "GADGET-002",
      "productName": "Mega Gadget",
      "price": 149.99,
      "categoryCode": "ELECTRONICS"
    }
  ]
}

2. Mapper Configuration

type JSON

@atomic = "batch"
@language = "en"

products = '$.products[*]' {
    id[code] = 'productCode'
    name = 'productName'
    price = 'price'
    category[code] = 'categoryCode'
    
    variants = 'variants[*]' {
        code = 'sku'
        color = 'color'
        stockQuantity = 'stock'
        @removeAll = "true"
    }
}

3. Resulting Import JSON

{
  "@atomic": "batch",
  "products": [
    {
      "id": "code:WIDGET-001",
      "name": "Super Widget",
      "price": 99.99,
      "category": "code:ELECTRONICS",
      "variants": [
        {
          "code": "WIDGET-001-RED",
          "color": "Red",
          "stockQuantity": 50
        },
        {
          "code": "WIDGET-001-BLUE",
          "color": "Blue",
          "stockQuantity": 30
        }
      ],
      "variants@removeAll": true
    },
    {
      "id": "code:GADGET-002",
      "name": "Mega Gadget",
      "price": 149.99,
      "category": "code:ELECTRONICS"
    }
  ]
}

**What Happens:**
- Products are matched by code (`code:WIDGET-001`)
- Category is resolved via lookup (`code:ELECTRONICS`)
- Existing variants not in the feed are removed (`@removeAll: true`)
- New variants are created, existing ones are updated
- Only actual changes are written to database
- Processing is done in batches for optimal performance

Example: Invoices Feed

Synchronizing invoices with line items

{
  "@atomic": "batch",
  "invoices": [
    {
      "id": "code:INV-2024-001",
      "customer": "code:CUSTOMER-123",
      "issueDate": "2024-01-15",
      "dueDate": "2024-02-15",
      "items": [
        {
          "product": "code:WIDGET-001",
          "quantity": 5,
          "unitPrice": 99.99,
          "description": "Super Widget - Red"
        },
        {
          "product": "code:GADGET-002",
          "quantity": 2,
          "unitPrice": 149.99,
          "description": "Mega Gadget"
        }
      ],
      "items@removeAll": true
    }
  ]
}

**Probabilistic Matching:**
Even though invoice line items typically don't have unique identifiers, the system uses a probability mechanism to match them based on product, quantity, and other fields. This ensures accurate updates without duplicate line items.

Generating Feeds

Feed API can also generate data exports in various formats

You can use the Feed API not just for imports, but also to generate data exports:

Query-Based Export

Use the query language to extract data and export it in various formats.

GET /api/products?format=json&export=true

Reverse Mapper

Use mappers in reverse to transform internal data back to external formats.

Forward Mapper

Apply forward mappers to generate specialized export formats.

Supported Export Formats:

JSON, XML, CSV, Excel, PDF, HTML, Custom

[navigate:/devdoc/exports|View Export Documentation|file-text]

Best Practices

Use batch mode for large feeds — Set @atomic: "batch" for optimal performance with large datasets.
Use lookups instead of database IDs — Prefer code:, email:, or other business keys for better portability.
Leverage @removeAll for complete synchronization — Use @removeAll: true to ensure deleted items are removed from the target system.
Test with small datasets first — Validate your mapper and import logic with a small subset before processing large feeds.
Monitor import results — Check import logs and error reports to identify and fix data issues.

Testing Tools

Interactive tools to test and validate your Feed API setup

[navigate:/devdoc/test-mapper|Test Mapper|settings] [navigate:/devdoc/test-import|Test Import|upload]

Test your mapper configuration with sample data and test import operations