merge¶

Configuration Merge Pattern for Secure Configuration Management

This module solves a critical security problem in configuration management: separating sensitive data from non-sensitive data while maintaining a unified configuration structure for application consumption.

The Security Problem

In production applications, you need to store: - Non-sensitive config: Database hosts, timeouts, feature flags (safe to version control) - Sensitive config: Passwords, API keys, certificates (must NOT be in version control)

Traditional Problems

Without proper separation, developers often:

🚨 Accidentally commit secrets to version control
🔄 Duplicate config structure between files (maintenance nightmare)
🐛 Manual config assembly (error-prone, inconsistent)
🚀 Complex deployment (hard to automate, environment-specific)

The Solution: Structural Merging

This module provides recursive, structure-aware merging that:

✅ Preserves data types: Handles dicts, lists, and nested structures
✅ Maintains relationships: Merges corresponding list items by position
✅ Validates structure: Ensures configs have compatible schemas
✅ Prevents corruption: Immutable operations (returns new objects)

Example Use Case

# config.json (safe to commit)
{
    "database": {
        "host": "prod-db.com",
        "port": 5432
    },
    "users": [
        {"username": "admin"},
        {"username": "app"}
    ]
}

# secrets.json (encrypted, never committed)
{
    "database": {
        "password": "secret123"
    },
    "users": [
        {"password": "admin-pwd"},
        {"password": "app-pwd"}
    ]
}

# Result after merging
{
    "database": {
        "host": "prod-db.com",
        "port": 5432,
        "password": "secret123"
    },
    "users": [
        {"username": "admin", "password": "admin-pwd"},
        {"username": "app", "password": "app-pwd"}
    ]
}

When to Use This Pattern

✅ Multi-environment deployments with sensitive config
✅ Microservices with shared config structure
✅ CI/CD pipelines that inject secrets at deployment time
✅ Configuration templates that need runtime data injection
✅ Compliance scenarios requiring secret/non-secret separation

configcraft.merge.deep_merge(data1: dict, data2: dict, _fullpath: str | None = None) → dict[source]¶

Intelligently merge two configuration dictionaries while preserving structure and relationships.

This function solves the critical problem of safely combining configuration files without losing data or breaking relationships between configuration elements.

Why Use This Instead of Simple Dict Updates?

Standard dict.update() operations are shallow and destructive: - ❌ Overwrites entire nested dictionaries - ❌ Replaces entire lists (losing relationships) - ❌ No validation of structural compatibility

This function provides deep, intelligent merging: - ✅ Recursive: Merges nested dictionaries at any depth - ✅ Positional: Merges list items by position (maintains relationships) - ✅ Immutable: Returns new objects (original data unchanged) - ✅ Validated: Ensures structural compatibility between inputs

Common Use Cases:

Secret Injection: Merge non-sensitive config with secrets
Environment Overrides: Combine base config with environment-specific values
Modular Config: Assemble configuration from multiple sources
Template Expansion: Fill configuration templates with runtime data

Examples:

Basic merging:

>>> base_config = {
...     "database": {"host": "localhost", "port": 5432},
...     "features": {"logging": True}
... }
>>> secrets = {
...     "database": {"password": "secret123"},
...     "features": {"analytics": False}
... }
>>> result = deep_merge(base_config, secrets)
>>> # Result: {
>>> #     "database": {"host": "localhost", "port": 5432, "password": "secret123"},
>>> #     "features": {"logging": True, "analytics": False}
>>> # }

List merging (by position):

>>> user_config = {
...     "users": [
...         {"username": "alice", "role": "admin"},
...         {"username": "bob", "role": "user"}
...     ]
... }
>>> password_config = {
...     "users": [
...         {"password": "alice-secret"},
...         {"password": "bob-secret"}
...     ]
... }
>>> result = deep_merge(user_config, password_config)
>>> # Result: {
>>> #     "users": [
>>> #         {"username": "alice", "role": "admin", "password": "alice-secret"},
>>> #         {"username": "bob", "role": "user", "password": "bob-secret"}
>>> #     ]
>>> # }

Parameters:

data1 – Base configuration dictionary (typically non-sensitive config)
data2 – Override configuration dictionary (typically secrets or environment-specific)
_fullpath – Internal parameter for error reporting (do not use)

Raises:

ValueError – When lists have different lengths (structural mismatch)
TypeError – When attempting to merge incompatible data types

Returns:

New dictionary containing the intelligently merged configuration

Note

This operation is immutable - original dictionaries are not modified. The function creates deep copies before merging to prevent side effects.