Data Mapping Documentation

ParseMapper format for mapping and transforming data from various sources

[navigate:/devdoc/test-mapper|Test Mapper|play|right]

Overview

The ParseMapper ‘import’ format is a configuration syntax for mapping and transforming data from various structured sources (such as XML, JSON, CSV, etc.) into a normalized internal representation. This format is used to define how fields from the source data are mapped, transformed, and related to target entities.

Key Components

Variable Definitions Optional variable declarations using @variable directive

Type Declaration Specifies the source data type (XML, JSON, CSV, etc.)

Global Directives Optional settings that affect parsing behavior

Entity Mappings Definitions for extracting and mapping entities and fields

File Structure

A typical import mapping file consists of variable definitions, type declaration, global directives, and entity mappings:

@variable default_vat_rate = {
    name: "Default VAT Rate"
    type: "number"
    defaultValue: "21"
}

type XML

@atomic = "false"
@strict = false
@language = "en"

products = '$.SHOP.SHOPITEM[*]' {
    id[code] = 'CODE'
    name = 'NAME'
    // ... more field mappings ...
    variants = 'VARIANTS.VARIANT' {
        code = 'CODE'
        // ... nested field mappings ...
        @removeAll = "true"
    }
    // ... more nested entities ...
}

Path Syntax

Simple Dotted Path

Straightforward dot notation without leading $ or @.

LOGISTIC.WEIGHT

Used for direct, static field access

JSONPath

Powerful syntax for navigating JSON structures. Denoted by leading $ or @.

$.SHOP.SHOPITEM[*]

Supports advanced selectors, wildcards, and array access

Parent Reference ($parent) [badge:New|secondary|right]

Access fields from the parent object in nested structures using the $parent prefix.

$parent.orderId                 // Access parent field
$parent.customer.name           // Access nested parent field
$parent["field-name"]           // With special characters

Example usage:

orders = '$.orders[*]' {
    orderId = '$.id'
    items = '$.items[*]' {
        productName = '$.name'
        orderId = '$parent.id'
        orderDate = '$parent.date'
    }
}

Useful when child records need to reference parent-level information

Multiple Path Alternatives

Use the || operator to specify fallback paths. The first path that yields a value will be used.

name = '$.NAME' || '$.ALT_NAME' || "STATIC"

Syntax Reference

1. Type Declaration

Defines the source data type:

type XML

Supported types: XML, JSON, CSV, YAML, XLSX, XLS

2. Global Directives

Control parser behavior:

@atomic = "false"
@strict = false
@language = "en"

• @atomic: Whether the import should be atomic
• @strict: Whether to enforce strict mapping
• @language: Default language for imports

3. Variable Definitions

Define variables for configurable import parameters:

@variable default_vat_rate = {
    name: "Default VAT Rate"
    description: "Variable description"
    type: "number"
    defaultValue: "21"
}

Variable Types

• “number” - Numeric values
• “string” - Text values
• “boolean” - True/false
• “relation” - Entity references

Translatable Fields

name:
  - en: "English Name"
  - cs: "Czech Name"

4. Entity Mapping

Define how to extract entities from source data:

entityName = 'jsonpath' {
    field1 = 'SOURCE_FIELD'
    field2 = 'NESTED.PATH'
    manufacturer[code] = 'MANUFACTURER'
    status = "active"
    // ... more fields ...
}

Special Features

• Lookup fields: field[key] = ‘SOURCE’
• Static values: field = “static”
• Nested entities with @removeAll
• Field replacement tables
• Path-specific replacement tables

Expressions

total = ${price + vat}

Examples

E-shop XML Import

Complex XML import with nested entities and lookup fields:

type XML

@atomic = "false"
@strict = false

products = '$.SHOP.SHOPITEM[*]' {
    id[code] = 'CODE'
    name = 'NAME'
    description = 'DESCRIPTION'
    unit[name] = 'UNIT'
    manufacturer[code] = 'MANUFACTURER'
    variants = 'VARIANTS.VARIANT' {
        code = 'CODE'
        priceWithVat = 'PRICE_VAT'
        @removeAll = "true"
    }
    images = 'IMAGES.IMAGE' {
        url = '_'
        description = '$["$"].description'
        @removeAll = "true"
    }
    categories = 'CATEGORIES' {
        category[id] = 'CATEGORY.$.id'
        category[name] = 'CATEGORY._'
        @removeAll = "true"
    }
}

Field Replacement Example

Map specific source values to different target values:

taxRateLevel = 'TAX_RATE_LEVEL' ['high' = 'standard', 'low' = 'reduced']
status = 'state' ['active' = 'enabled', 'inactive' = 'disabled']

Path-Specific Replacement Tables [badge:New|secondary|right]

Apply replacement tables to individual paths in multipath expressions:

// Mixed: No replacement on first path, replacement on second path
visibility = 'VISIBILITY' || ('VISIBLE' ['1' = 'visible', '0' = 'hidden'])

// Individual: Different replacement tables for each path
status = ('STATUS' ['A' = 'active', 'I' = 'inactive']) || ('STATE' ['1' = 'enabled', '0' = 'disabled'])

// Backward compatible: Field-level replacement table (existing behavior)
type = 'TYPE' || 'TYPE_CODE' ['premium' = 'PREMIUM', 'basic' = 'STANDARD']

Mixed Global/Local Apply replacement only to specific paths while leaving others unchanged

Individual Path Tables Each path can have its own transformation rules

Backward Compatible Existing field-level syntax continues to work

Translatable Fields with Dynamic Language Keys [badge:New|secondary|right]

Map multi-language content with dynamic language key resolution

Translatable fields can determine the target language dynamically from the source data, enabling efficient multi-language imports from single source records.

Simple Path Language Key

name['LANG'] = 'TEXT'

Language field in the same record determines the language

JSONPath Language Key

name['$.language.code'] = 'TEXT'

Use complex path expressions to locate language identifier

Static Language Key

name["en"] = 'NAME_EN' || 'NAME'

Explicitly map to specific language with fallback

Expression Language Key

name[${LOWER(langCode)}] = 'TEXT'

Computed language key using expressions

Complete Example

products = '$.items[*]' {
    code = 'CODE'
    
    // Dynamic language keys from source data
    name['LANG'] = 'TEXT' @loop '$.TEXTS[*]'
    description['LANG'] = 'DESCRIPTION' @loop '$.TEXTS[*]'
    
    // Static language keys with fallback
    shortName["en"] = 'SHORT_NAME_EN' || 'NAME'
    shortName["cs"] = 'SHORT_NAME_CS' || 'NAME'
}

@loop Directive [badge:New|secondary|right]

Iterate over arrays to create multiple records or translations

The @loop directive allows you to iterate over arrays or collections in the source data, creating multiple records, translations, or field values.

Basic Syntax

fieldName = 'PATH' @loop 'ITERATION_PATH'

With Translatable Fields

name['lang'] = 'text' @loop '$.translations[*]'

Create multiple language translations from array

Array Creation

tags = 'tag' @loop '$.tags[*]'

Create array field from source array

Nested Relations with @loop

images = '$.images[*]' @loop '$.images[*]' {
    url = 'url'
    description['lang'] = 'alt' @loop '$.alts[*]'
    sortOrder = 'position'
}

Loop over images array to create multiple image records, each with translated descriptions

Complete Example

products = '$.products[*]' {
    code = 'sku'
    
    // Multi-language from translations array
    name['languageCode'] = 'title' @loop '$.translations[*]'
    description['languageCode'] = 'desc' @loop '$.translations[*]'
    
    // Simple array creation
    tags = 'tag' @loop '$.tags[*]'
    
    // Nested relations
    variants = '$.options[*]' @loop '$.options[*]' {
        code = 'optionSku'
        name['lang'] = 'optionName' @loop '$.optionTranslations[*]'
        priceWithVat = 'optionPrice'
    }
}

Translatable Field Output Formats [badge:Important|secondary|right]

Two supported formats for translatable field output

The mapper supports two output formats for translatable fields, depending on how you structure your mapping.

Object Format (with @loop) [badge:Recommended|default|right]

// Mapper syntax
name['LANG'] = 'TEXT' @loop 'TEXTS.ITEM'

// Output
{
  "name": {
    "en": "English Name",
    "cs": "České jméno",
    "de": "Deutscher Name"
  }
}

✅ Clean, flat structure ✅ Better performance for lookups ✅ Easier to work with in UI ✅ Recommended for most use cases

Array Format (without @loop) [badge:Alternative|outline|right]

// Mapper syntax
name = 'TEXTS.ITEM' {
    language = 'LANG'
    value = 'TEXT'
}

// Output
{
  "name": [
    {"language": "en", "value": "English Name"},
    {"language": "cs", "value": "České jméno"},
    {"language": "de", "value": "Deutscher Name"}
  ]
}

• Structured metadata format • Can include additional fields • Used for array-based systems • Legacy compatibility

Complete Example: Both Formats

type XML

products = '$.SHOP.PRODUCT[*]' {
    code = 'CODE'
    
    // Object format - recommended
    name['$.language'] = 'title' @loop 'DESCRIPTIONS.DESC'
    description['$.language'] = 'text' @loop 'DESCRIPTIONS.DESC'
    
    // Array format - alternative
    alternativeName = 'DESCRIPTIONS.DESC' {
        language = '$.language'
        value = 'title'
    }
}

When to Use Each Format

Use Object Format (@loop):

• Most common scenario - clean translations
• Better UI integration and performance
• Standard approach for new projects

Use Array Format (nested):

• Need to store metadata with translations
• Migrating from array-based systems
• Additional fields beyond language/value

@if Directive [badge:New|secondary|right]

Conditionally include fields based on runtime conditions

The @if directive allows you to conditionally include fields or relations based on runtime conditions evaluated from the source data.

Basic Syntax

fieldName = 'PATH' @if ${condition}

Simple Condition

discount = 'DISCOUNT' @if ${hasDiscount == 'true'}

Only include discount if product has one

Complex Condition

price = 'BULK_PRICE' @if ${isB2B == 'true' && price > 100}

Multiple conditions with AND/OR operators

Supported Operators

Comparison ==, !=, >, <, >=, <=

Logical && (AND), || (OR)

Types strings, numbers, booleans

Conditional Nested Relations

// Only create images if product has them
images = '$.images[*]' @loop '$.images[*]' @if ${hasImages == 'true'} {
    url = 'url'
    description = 'alt'
    sortOrder = 'position'
}

Complete Example

products = '$.products[*]' {
    code = 'sku'
    name = 'title'
    basePrice = 'price'
    
    // Conditional fields
    discount = 'discountPercent' @if ${hasDiscount == 'true'}
    weight = 'weightKg' @if ${isPhysical == 'true'}
    downloadUrl = 'fileUrl' @if ${isDigital == 'true'}
    
    // Extended description only for premium products
    extendedDesc = 'detailedText' @if ${productType === 'premium'}
    
    // Conditional relations
    variants = '$.options[*]' @loop '$.options[*]' @if ${isConfigurable == 'true'} {
        code = 'optionSku'
        priceWithVat = 'optionPrice'
    }
}

Combined @loop and @if [badge:Advanced|secondary|right]

Combine directives for powerful conditional array processing

You can combine @loop and @if directives for sophisticated conditional array processing and filtering.

Conditional Multi-Language Import

products = '$.items[*]' {
    code = 'CODE'
    
    // Import translations only if available
    name['LANG'] = 'TEXT' @loop '$.TEXTS[*]' @if ${HAS_TRANSLATIONS == 'true'}
    description['LANG'] = 'DESC' @loop '$.TEXTS[*]' @if ${HAS_TRANSLATIONS == 'true'}
    
    // Fallback if no translations
    name = 'NAME' @if ${HAS_TRANSLATIONS != 'true'}
    description = 'DESCRIPTION' @if ${HAS_TRANSLATIONS != 'true'}
}

Filtered Array Processing

products = '$.products[*]' {
    code = 'sku'
    name['lang'] = 'title' @loop '$.translations[*]'
    
    // Only import images if product has them
    images = '$.images[*]' @loop '$.images[*]' @if ${hasImages == 'true'} {
        url = 'url'
        description['lang'] = 'alt' @loop '$.alts[*]'
        sortOrder = 'position'
    }
    
    // Specifications only for electronics
    specifications = '$.specs[*]' @loop '$.specs[*]' @if ${category === 'electronics'} {
        name['lang'] = 'specName' @loop '$.nameTranslations[*]'
        value = 'specValue'
        unit = 'specUnit'
    }
}

Best Practices

• Use @if before @loop when possible to filter early
• Keep conditions simple and testable
• Test with representative sample data
• Avoid deeply nested loops (max 2-3 levels)
• Document complex business logic with comments

Array Syntax for Collection Fields [badge:New|secondary|right]

Append multiple values to collection fields from independent sources

The array syntax (fieldName[]) allows you to append multiple values to the same collection field, rather than replacing it. This is particularly useful when you need to populate a field from multiple independent sources or conditions.

Basic Syntax

fieldName[] = 'PATH' {
    // field mappings
}

When you use the [] suffix on a field name, each occurrence appends to the collection rather than replacing previous values.

Problem: Multiple Assignments

Without array syntax, you cannot assign to the same field multiple times:

❌ Not Supported - Parser Error

flags = 'LABELS.LABEL' {
    flag[code, name] = 'NAME'
}
flags = {
    flag[code] = 'new'
} @if ${NEW_YN === '1'}
flags = {
    flag[code] = 'special'
} @if ${SPECIAL_YN === '1'}

Solution: Array Syntax

✅ Correct - Appends to Collection

// First clear existing flags
flags = 'LABELS.LABEL' {
    flag[code, name] = 'NAME'
    @removeAll = "true"
}

// Then append conditionally
flags[] = {
    flag[code] = 'new'
} @if ${NEW_YN === '1'}

flags[] = {
    flag[code] = 'special'
} @if ${SPECIAL_YN === '1'}

flags[] = {
    flag[code] = 'sellout'
} @if ${SELLOUT_YN === '1'}

Complete Example with @removeAll

A common pattern is to clear existing data first, then append new items:

products = '$.SHOP.SHOPITEM[*]' {
    code = 'CODE'
    
    // First, clear all existing flags and import from primary source
    flags = 'LABELS.LABEL' {
        flag[code, name] = 'NAME'
        @removeAll = "true"
    }
    
    // Then append additional flags based on conditions
    flags[] = {
        flag[code] = 'new'
        flag[name] = 'New Product'
    } @if ${NEW_PRODUCT === 'true'}
    
    flags[] = {
        flag[code] = 'clearance'
        flag[name] = 'Clearance Sale'
    } @if ${CLEARANCE === 'true'}
}

Array Syntax with Simple Values

products = '$.products[*]' {
    code = 'id'
    
    // Base tags from primary field
    tags[] = 'primaryTag'
    
    // Add category-based tags
    tags[] = 'electronics' @if ${category === 'ELECTRONICS'}
    tags[] = 'apparel' @if ${category === 'CLOTHING'}
    
    // Add promotional tags
    tags[] = 'featured' @if ${isFeatured == 'true'}
    tags[] = 'bestseller' @if ${salesRank <= 10}
}

When to Use Array Syntax

✅ Use Array Syntax When:

• Multiple independent sources
• Conditional additions
• Incremental building
• Mixed static and dynamic data

❌ Don’t Use When:

• Single source (use @loop)
• Want to replace rather than append
• Simple single assignment

Best Practices

• Clear First, Then Append: Use @removeAll on the first assignment, then use [] for subsequent additions
• Use @if for Conditional Items: Combine with @if directive to add items only when conditions are met
• Maintain Sort Order: Consider adding explicit sort order fields when using array syntax with multiple sources
• Test Thoroughly: Verify that all array items are created correctly with sample data

Lookup Field Directives [badge:New|secondary|right]

Control behavior and capture missing data when resolving lookup fields

When mapping lookup fields (relations), these directives control what happens when a reference cannot be resolved and allow you to collect data for creating missing records.

@if-not-found Directive

Controls the behavior when a lookup field reference cannot be found in the database.

skip Skip the entire record (default)

create Collect data for creating missing records

ignore Leave field null, continue processing

error Throw error, halt import

products = '$.SHOPITEM[*]' {
    code = 'CODE'
    name = 'NAME'
    
    // Skip entire product if manufacturer not found (default)
    manufacturer[code] = 'MANUFACTURER_CODE'
    
    // Collect data to create missing categories
    category[code]@if-not-found = 'create' = 'CATEGORY_ID'
    
    // Continue even if unit not found (field will be null)
    unit[name]@if-not-found = 'ignore' = 'UNIT_NAME'
}

@create-as Directive

When using @if-not-found='create', this directive specifies what data to collect for creating missing lookup records.

products = '$.SHOPITEM[*]' {
    code = 'CODE'
    name = 'NAME'
    
    // Collect category data for creation if not found
    category[code]@if-not-found = 'create' = 'CATEGORY_ID'
    category[code]@create-as = {
        code = 'CATEGORY_ID'
        name = 'CATEGORY_TITLE'
        description = 'CATEGORY_DESC' || 'CATEGORY_TITLE'
        sortOrder = 'CATEGORY_ORDER'
    }
}

Advanced Examples

Alternative Paths and Replace Tables

category[code]@if-not-found = 'create' = 'CATEGORY_ID' || 'CATEGORY_CODE'
category[code]@create-as = {
    code = 'CATEGORY_ID' || 'CATEGORY_CODE'
    name = 'CATEGORY_NAME' || 'CATEGORY_TITLE' || 'CATEGORY_ID'
    type = 'CATEGORY_TYPE' ['MAIN' = 'primary', 'SUB' = 'secondary']
}

Static Values

manufacturer[code]@if-not-found = 'create' = 'MANUFACTURER_CODE'
manufacturer[code]@create-as = {
    code = 'MANUFACTURER_CODE'
    name = 'MANUFACTURER_NAME'
    country = "CZ"
    isActive = "true"
}

Nested Relations

products = '$.SHOPITEM[*]' {
    code = 'CODE'
    name = 'NAME'
    
    category[code]@if-not-found = 'create' = 'CATEGORY_ID'
    category[code]@create-as = {
        code = 'CATEGORY_ID'
        name = 'CATEGORY_NAME'
        
        // Nested relations in create-as block
        productPrices = 'PRICES.PRICE' {
            priceWithVat = 'PRICE_VAT'
            priceWithoutVat = 'PRICE_BASE'
            validFrom = 'DATE_FROM'
            @removeAll = "true"
        }
    }
}

How It Works

1. During import, the system tries to resolve each lookup field reference
2. If not found and @if-not-found='create', data is collected
3. Collected data is stored with key fieldName@create-as
4. The import job contains both imported records and creation data
5. You can then create missing records and re-run the import

Best Practices

• Use create for critical references like categories, manufacturers
• Use ignore for optional fields that can be filled later
• Use skip when missing references indicate invalid data
• Use error when data quality must be guaranteed
• Provide alternative paths and fallbacks in @create-as blocks
• Include enough fields to create valid records (check metamodel requirements)
• Test with data containing missing references

Parent Reference ($parent) [badge:New|secondary|right]

Access parent object fields in nested structures

When working with nested entities, you can reference fields from the parent object using the $parent prefix. This is particularly useful when child records need to include information from their parent level.

Syntax

$parent.field                   // Direct parent field
$parent.nested.field            // Nested parent field
$parent["field-name"]           // With special chars

Example: Order Items with Parent Order Info

type JSON

orders = '$.orders[*]' {
    orderId = '$.id'
    orderDate = '$.date'
    customer = '$.customer'
    
    items = '$.items[*]' {
        productName = '$.name'
        price = '$.price'
        quantity = '$.quantity'
        // Reference parent order fields
        orderId = '$parent.id'
        orderDate = '$parent.date'
        orderCustomer = '$parent.customer'
    }
}

Each item will include orderId, orderDate, and orderCustomer from the parent order object.

Example: Product Variants with Parent Product Info

products = '$.products[*]' {
    productId = '$.id'
    productName = '$.name'
    category = '$.category'
    brand = '$.brand'
    
    variants = '$.variants[*]' {
        variantCode = '$.sku'
        variantSize = '$.size'
        variantColor = '$.color'
        stockLevel = '$.stock'
        // Include parent product information
        productId = '$parent.id'
        productName = '$parent.name'
        category = '$parent.category'
        brand = '$parent.brand'
    }
}

Example: Invoice Line Items

invoices = '$.invoices[*]' {
    invoiceNumber = '$.number'
    invoiceDate = '$.date'
    currency = '$.currency'
    customer[code] = '$.customer.code'
    
    lineItems = '$.lines[*]' {
        itemCode = '$.code'
        description = '$.description'
        amount = '$.amount'
        quantity = '$.qty'
        // Reference parent invoice data
        invoiceNumber = '$parent.number'
        invoiceDate = '$parent.date'
        currency = '$parent.currency'
        customerCode = '$parent.customer.code'
    }
}

Important Notes

• $parent refers to the immediate parent object in the mapping hierarchy
• Parent context is only available in nested structures (objects with nested field mappings)
• If used at the root level where no parent exists, the field will be undefined
• In @loop directives, the loop item becomes the current source, and the original source becomes the parent
• You can access nested parent fields: $parent.customer.name

Use Cases

Denormalization Include parent information in child records for easier querying and reporting

Data Enrichment Add context from parent entities to make child records self-contained

Reference Tracking Maintain references between related records for data integrity

Audit Trail Include parent-level timestamps and identifiers in child records

Tips and Best Practices

Syntax Tips

• Use single quotes for source paths
• Use double quotes for static values
• Use nested blocks for arrays/objects
• Add @removeAll for nested entities
• Use JSONPath syntax ($.path[*]) in @loop
• Use $parent to reference parent fields in nested structures
• Keep @if conditions simple

Performance

• Test with large datasets
• Use lookup fields for relations
• Plan for error handling
• Document complex transformations
• Use @if to filter early in processing
• Minimize nesting depth for loops