Advanced Workflow Patterns

Error Handling Best Practices

• ✓ Set errorConfig to "continue" for non-critical steps
• ✓ Add retry logic with exponential backoff in script steps
• ✓ Use conditions to branch on success/failure
• ✓ Log errors to database for monitoring
• ✓ Alert team on critical failures
• ✓ Implement circuit breaker for external dependencies

Loop Processing Strategies

• ✓ Batch large datasets to avoid memory issues
• ✓ Set maxIterations limit to prevent runaway loops
• ✓ Use item_last variable to capture final iteration output
• ✓ Track progress in database for long-running loops
• ✓ Add break conditions to exit early if needed
• ✓ Collect errors during loop and handle separately

Tiered Alert System

Workflow: "Intelligent Alert Routing"

Step 1: Analyze Severity (Transform)
  severity = 'critical' if metrics.error_rate > 50 else \
             'warning' if metrics.error_rate > 20 else \
             'info'
  
  is_business_hours = 9 <= datetime.now().hour < 17
  is_weekend = datetime.now().weekday() >= 5
  
  result = {
    'severity': severity,
    'is_urgent': severity == 'critical' and is_business_hours,
    'recipients': {
      'critical': ['on_call@company.com', 'slack:#critical-alerts'],
      'warning': ['team@company.com'],
      'info': ['logs@company.com']
    }[severity]
  }

Step 2: Route Alerts (Condition)
  if severity == 'critical':
    Branch: CRITICAL
      - Send SMS to on-call engineer
      - Page via PagerDuty
      - Post to #critical-alerts Slack
      - Create incident ticket
  
  elif severity == 'warning':
    Branch: WARNING
      - Send email to team
      - Post to #warnings Slack channel
  
  else:
    Branch: INFO
      - Log to database only
      - Add to daily digest

Step 3: Log Alert (Database)
  INSERT INTO alerts (severity, message, recipients, created_at)
  VALUES ('{{alert_data.severity}}', '...', '...', NOW())

Nested Condition & Loop Example

Workflow: "Complex Order Processing"

Step 1: Validate Order (Script)
  Output: order_validation

Step 2: Main Condition: Order Valid? (Condition)
  Expression: order_validation.is_valid == True
  
  True Branch:
    
    Sub-Step 1: Check Stock (Database)
      SELECT * FROM inventory WHERE sku IN (...)
      Output: inventory_data
    
    Sub-Step 2: Stock Available? (Condition)
      Expression: inventory_data.available > 0
      
      True Branch:
        
        Sub-Step A: Process Payment (API)
        
        Sub-Step B: Payment Success? (Condition)
          If yes:
            - Deduct from inventory
            - Send confirmation email
            - Create shipment
          If no:
            - Refund customer
            - Log transaction
      
      False Branch:
        - Check backorder preference
        - If yes: Add to queue
        - If no: Cancel order
  
  False Branch:
    - Send validation error to customer
    - Do not process payment

Data Aggregation Workflow

Workflow: "Customer 360 View"

Step 1: Fetch from CRM API
  URL: crm.example.com/customers/{{webhook_payload.customer_id}}
  Output: crm_data

Step 2: Fetch from Database
  Query: SELECT * FROM purchases WHERE customer_id = {{webhook_payload.customer_id}}
  Output: purchase_history

Step 3: Fetch from Analytics API
  URL: analytics.example.com/customer/{{webhook_payload.customer_id}}/metrics
  Output: behavior_metrics

Step 4: Merge Data (Transform)
  customer = {
    'profile': crm_data,
    'purchases': purchase_history,
    'metrics': behavior_metrics,
    'lifetime_value': sum([p['total'] for p in purchase_history]),
    'engagement_score': behavior_metrics['score'],
    'risk_level': 'high' if behavior_metrics['churn_prob'] > 0.7 else 'low'
  }
  Output: customer_360

Step 5: Route by Risk (Condition)
  if customer_360.risk_level == 'high':
    - Alert retention team
    - Offer discount coupon
    - Schedule call
  else:
    - Update marketing profile
    - Continue normal campaign

Daily Export to S3 Example

Workflow: "Daily Sales Export"
Schedule: 0 2 * * * (2 AM daily)

Step 1: Query Data (Database)
  Query:
  SELECT order_id, customer_id, total, status, created_at
  FROM orders
  WHERE DATE(created_at) = CURRENT_DATE - 1
  Output: daily_orders

Step 2: Format as CSV (Transform)
  import json
  import csv
  from io import StringIO
  
  output = StringIO()
  writer = csv.DictWriter(output, fieldnames=['order_id', 'customer_id', 'total', 'status'])
  writer.writeheader()
  for row in daily_orders:
      writer.writerow(row)
  
  csv_content = output.getvalue()
  
  result = {
    'csv': csv_content,
    'filename': f"sales_{date.today().isoformat()}.csv",
    'row_count': len(daily_orders)
  }

Step 3: Upload to S3 (API)
  URL: s3.amazonaws.com/my-bucket/exports/
  Method: PUT
  Headers:
    Authorization: AWS4-HMAC-SHA256 ...
  Body: {{export_data.csv}}

Step 4: Notify (Notification)
  Email to: data-team@company.com
  Subject: Daily Sales Export - {{export_data.row_count}} orders

Circuit Breaker Pattern

Workflow: "Circuit Breaker for External API"

Step 1: Check Circuit State (Database)
  Query:
  SELECT * FROM circuit_breaker
  WHERE service = 'payment_api'
  ORDER BY updated_at DESC LIMIT 1
  Output: circuit_state

Step 2: Is Circuit Open? (Condition)
  Expression: circuit_state.status == 'open' AND circuit_state.since < datetime.now() - timedelta(minutes=5)
  
  If Open & Timeout Passed:
    → Reset circuit to 'half_open'
  
  If Closed:
    → Attempt API call
    → If failure: Open circuit
    → If success: Keep closed
  
  If Open & Recently Opened:
    → Use cached/fallback data
    → Skip API call

Cache-First with Fallback

Workflow: "Product Data with Cache"

Step 1: Check Cache (Database)
  Query:
  SELECT * FROM cache
  WHERE key = 'products_list'
  AND expires_at > NOW()
  Output: cached_data

Step 2: Use Cache or Fetch (Condition)
  Expression: cached_data is not None
  
  True Branch (Cache Hit):
    → Use {{cached_data.value}}
    → Log cache hit
  
  False Branch (Cache Miss):
    → Fetch from API
    → Parse response
    → Store in cache with 1 hour TTL
    → Use fresh data

Step 3: Return Data
  Output: final_products