> INTEGRATION_PATTERNS

Proper EDI transaction set design is critical for ensuring interoperability between trading partners. Here are proven strategies for building robust EDI implementations.

1. Standardize Over Customize

While X12 and EDIFACT standards allow for optional segments and elements, resist the temptation to create heavily customized transaction sets. Stick to industry-standard implementation guides (like VICS for retail or CIDX for chemical industry) whenever possible. Custom fields should be documented in detailed implementation guides and shared with all trading partners.

2. Mandatory vs. Optional Elements

• ISA/IEA Headers: Always include full qualifier sets (ISA05, ISA07) for proper routing
• Functional Groups: Use GS/GE envelopes even for single-transaction batches
• Control Numbers: Implement sequential numbering with proper rollover logic (ISA13, GS06, ST02)

3. Validation Layers

// Multi-layer EDI validation approach
1. Syntax Validation → Check segment structure, element count
2. Semantic Validation → Verify business logic (dates, amounts)
3. Partner Validation → Confirm trading partner agreements
4. Data Validation → Validate against internal business rules

4. Error Handling Best Practices

Always generate 997 Functional Acknowledgments for X12 or CONTRL for EDIFACT. Include detailed error codes (AK403, AK502) to help partners diagnose issues quickly. Implement retry logic with exponential backoff for transient failures.

Modern integration platforms require well-designed APIs that are intuitive, scalable, and maintainable. These patterns have proven effective across enterprise integrations.

Resource-Oriented Design

// Good: Resource-based URIs
GET    /api/v1/orders/{orderId}
POST   /api/v1/orders
PUT    /api/v1/orders/{orderId}/status
DELETE /api/v1/orders/{orderId}

// Avoid: Action-based URIs
POST /api/v1/createOrder  ❌
GET  /api/v1/getOrder    ❌

Pagination & Filtering

For large datasets, implement cursor-based pagination for better performance than offset-based approaches:

GET /api/v1/transactions?limit=100&cursor=eyJ0aW1lc3...
Response:
{
  "data": [...],
  "pagination": {
    "next_cursor": "eyJpZCI6MTIzNDU2...",
    "has_more": true
  }
}

Idempotency Keys

For POST requests that create resources, support idempotency keys to prevent duplicate processing in case of network retries. This is critical for payment and order processing integrations.

POST /api/v1/payments
Headers:
  Idempotency-Key: unique-transaction-id-12345
  
// Subsequent requests with same key return same response
// without creating duplicate payment

Versioning Strategy

• URL Versioning: /api/v1/resource - Simple and explicit
• Header Versioning: Accept: application/vnd.myapi.v2+json - Cleaner URLs
• Backward Compatibility: Maintain previous versions for at least 12 months

Event-driven architectures enable loose coupling and real-time data synchronization across distributed systems. Understanding when to use publish/subscribe vs. message queues is crucial.

Pub/Sub Pattern (Many Consumers)

Use publish/subscribe when multiple independent services need to react to the same event:

Event: order.created
├─> Inventory Service (reduces stock)
├─> Notification Service (sends email)
├─> Analytics Service (tracks metrics)
└─> Accounting Service (creates invoice)

Message Queue Pattern (Single Consumer)

Use message queues when only one consumer should process each message, with guaranteed delivery and ordering:

Queue: payment_processing
Message: { orderId: 123, amount: 99.99 }
→ One worker processes payment
→ Acknowledgment after successful processing
→ Retry on failure (with dead letter queue)

Event Schema Design

• CloudEvents Standard: Use standardized envelope format
• Schema Registry: Version and validate event payloads
• Event Sourcing: Store events as source of truth for system state

Bridging legacy systems and modern APIs requires sophisticated data transformation logic. These strategies minimize technical debt while ensuring data integrity.

Field-Level Transformations

// Legacy System (Fixed-width)
CUST_NAME    : "SMITH    JOHN      "
ORDER_AMT    : "0000099950"       // Cents
SHIP_DATE    : "20231215"

// Modern API (JSON)
{
  "customer": {
    "firstName": "John",
    "lastName": "Smith"
  },
  "amount": 999.50,
  "shipmentDate": "2023-12-15T00:00:00Z"
}

Lookup Table Pattern

When codes differ between systems, maintain lookup tables for bidirectional translation:

• Product Codes: SAP Material # ↔ NetSuite Item ID
• Location Codes: Warehouse IDs ↔ EDI Location Qualifiers
• Status Codes: Internal workflow states ↔ Trading partner statuses

Canonical Data Model

For complex integrations involving multiple systems, define a canonical data model as an intermediary format. Each system's adapter transforms to/from this universal format, reducing the number of transformation maps from N² to 2N.

Production integrations must handle failures gracefully. A well-designed error handling strategy prevents data loss and enables automated recovery.

Circuit Breaker Pattern

Prevent cascading failures by implementing circuit breakers for external service calls:

States:
CLOSED  → Normal operation, requests pass through
OPEN    → Too many failures, reject requests immediately  
HALF-OPEN → Test if service recovered, limited requests

Thresholds:
- Failure rate: 50% over 10 requests
- Open duration: 30 seconds
- Half-open test: 3 requests

Dead Letter Queue (DLQ)

Route failed messages to a DLQ after retry exhaustion. Monitor DLQ depth as a critical metric:

• Retry Strategy: 3 attempts with exponential backoff (1s, 4s, 16s)
• DLQ Processing: Manual review or automated reprocessing after fix
• Alerting: Trigger PagerDuty when DLQ depth >100 messages

Monitoring & Observability

Key Metrics:
├─ Throughput: Messages/second processed
├─ Latency: p50, p95, p99 processing times  
├─ Error Rate: Failed transactions / Total
├─ Queue Depth: Backlog size
└─ Circuit Breaker State: Open/Closed/Half-Open

Logging Strategy:
- Structured JSON logs with correlation IDs
- Full payload logging for failed transactions
- Distributed tracing (OpenTelemetry)

Idempotent Processing

Design integrations to handle duplicate messages safely. Store processed message IDs in a cache/database and skip already-processed messages. This enables safe retries without data corruption.