Nested Configuration

In this tutorial, you’ll learn how to work with nested configuration structures, mapping flat keys to nested dataclass hierarchies.

Learning Objectives

By the end of this tutorial, you’ll be able to:

  • Define nested configuration models

  • Use dot notation for nested keys

  • Load nested configuration from multiple sources

Step 1: Defining Nested Models

Let’s create a configuration with nested structures:

 1from dataclasses import dataclass, field
 2from varlord import Config
 3
 4@dataclass(frozen=True)
 5class DBConfig:
 6    host: str = field(default="localhost")
 7    port: int = field(default=5432)
 8    database: str = field(default="mydb")
 9
10@dataclass(frozen=True)
11class AppConfig:
12    host: str = field(default="0.0.0.0")
13    port: int = field(default=8000)
14    db: DBConfig = field(default_factory=lambda: DBConfig())  # Nested configuration
15
16# Initialize with defaults (automatic)
17cfg = Config(
18    model=AppConfig,
19    sources=[],  # Defaults are automatically applied
20)
21
22app = cfg.load()
23print(f"App: {app.host}:{app.port}")
24print(f"DB: {app.db.host}:{app.db.port}/{app.db.database}")

Expected Output:

App: 0.0.0.0:8000
DB: localhost:5432/mydb

Step 2: Using Default Factory for Nested Objects

To provide default nested objects, use field(default_factory=...):

 1from dataclasses import dataclass, field
 2from varlord import Config
 3
 4@dataclass(frozen=True)
 5class DBConfig:
 6    host: str = field(default="localhost")
 7    port: int = field(default=5432)
 8    database: str = field(default="mydb")
 9
10@dataclass(frozen=True)
11class AppConfig:
12    host: str = field(default="0.0.0.0")
13    port: int = field(default=8000)
14    db: DBConfig = field(default_factory=lambda: DBConfig())
15
16cfg = Config(
17    model=AppConfig,
18    sources=[],  # Defaults are automatically applied
19)
20
21app = cfg.load()
22print(f"App: {app.host}:{app.port}")
23print(f"DB: {app.db.host}:{app.db.port}/{app.db.database}")

Expected Output:

App: 0.0.0.0:8000
DB: localhost:5432/mydb

Key Points:

  • Use field(default_factory=...) to create default nested objects

  • The factory function is called to create a new instance each time

Step 3: Loading Nested Configuration from Environment

Environment variables use double underscore (__) as a separator for nested keys:

 1import os
 2from dataclasses import dataclass, field
 3from varlord import Config, sources
 4
 5@dataclass(frozen=True)
 6class DBConfig:
 7    host: str = field(default="localhost")
 8    port: int = field(default=5432)
 9    database: str = field(default="mydb")
10
11@dataclass(frozen=True)
12class AppConfig:
13    host: str = field(default="0.0.0.0")
14    port: int = field(default=8000)
15    db: DBConfig = field(default_factory=lambda: DBConfig())
16
17# Set nested environment variables (no prefix needed - filtered by model)
18os.environ["DB__HOST"] = "db.example.com"
19os.environ["DB__PORT"] = "3306"
20os.environ["DB__DATABASE"] = "production"
21
22cfg = Config(
23    model=AppConfig,
24    sources=[
25        sources.Env(),  # Only loads DB__HOST, DB__PORT, DB__DATABASE (filtered by model)
26    ],
27)
28
29app = cfg.load()
30print(f"App: {app.host}:{app.port}")
31print(f"DB: {app.db.host}:{app.db.port}/{app.db.database}")

Expected Output:

App: 0.0.0.0:8000
DB: db.example.com:3306/production

Key Mapping:

  • DB__HOST → db.host

  • DB__PORT → db.port

  • DB__DATABASE → db.database

The double underscore (__) is converted to a dot (.) for nested structure mapping. Only environment variables that match model fields are loaded.

Step 4: Loading Nested Configuration from CLI

Command-line arguments use hyphens for nested keys:

 1import sys
 2from dataclasses import dataclass, field
 3from varlord import Config, sources
 4
 5@dataclass(frozen=True)
 6class DBConfig:
 7    host: str = field(default="localhost")
 8    port: int = field(default=5432)
 9
10@dataclass(frozen=True)
11class AppConfig:
12    host: str = field(default="0.0.0.0")
13    port: int = field(default=8000)
14    db: DBConfig = field(default_factory=lambda: DBConfig())
15
16# Command-line arguments for nested fields
17sys.argv = [
18    "app.py",
19    "--db-host", "db.example.com",
20    "--db-port", "3306",
21]
22
23cfg = Config(
24    model=AppConfig,
25    sources=[
26        sources.CLI(),  # Model auto-injected, only parses model fields
27    ],
28)
29
30app = cfg.load()
31print(f"DB: {app.db.host}:{app.db.port}")

Expected Output:

DB: db.example.com:3306

CLI Nested Key Format:

  • --db-host → db.host

  • --db-port → db.port

Hyphens are converted to dots for nested structure mapping.

Step 5: Deeply Nested Configuration

You can nest multiple levels:

 1import os
 2from dataclasses import dataclass, field
 3from varlord import Config, sources
 4
 5@dataclass(frozen=True)
 6class CacheConfig:
 7    enabled: bool = field(default=False)
 8    ttl: int = field(default=3600)
 9
10@dataclass(frozen=True)
11class DBConfig:
12    host: str = field(default="localhost")
13    port: int = field(default=5432)
14    cache: CacheConfig = field(default_factory=lambda: CacheConfig())
15
16@dataclass(frozen=True)
17class AppConfig:
18    host: str = field(default="0.0.0.0")
19    db: DBConfig = field(default_factory=lambda: DBConfig())
20
21# Set deeply nested environment variables (no prefix needed)
22os.environ["DB__CACHE__ENABLED"] = "true"
23os.environ["DB__CACHE__TTL"] = "7200"
24
25cfg = Config(
26    model=AppConfig,
27    sources=[
28        sources.Env(),  # Only loads variables matching model fields
29    ],
30)
31
32app = cfg.load()
33print(f"DB Cache Enabled: {app.db.cache.enabled}")
34print(f"DB Cache TTL: {app.db.cache.ttl}")

Expected Output:

DB Cache Enabled: True
DB Cache TTL: 7200

Deep Nesting Mapping:

  • DB__CACHE__ENABLED → db.cache.enabled

  • DB__CACHE__TTL → db.cache.ttl

Step 6: Complete Example

Here’s a complete example with multiple nested structures:

 1import os
 2import sys
 3from dataclasses import dataclass, field
 4from varlord import Config, sources
 5
 6@dataclass(frozen=True)
 7class DBConfig:
 8    host: str = field(default="localhost")
 9    port: int = field(default=5432)
10    database: str = field(default="mydb")
11
12@dataclass(frozen=True)
13class RedisConfig:
14    host: str = field(default="localhost")
15    port: int = field(default=6379)
16
17@dataclass(frozen=True)
18class AppConfig:
19    host: str = field(default="0.0.0.0")
20    port: int = field(default=8000)
21    db: DBConfig = field(default_factory=lambda: DBConfig())
22    redis: RedisConfig = field(default_factory=lambda: RedisConfig())
23
24def main():
25    # Set environment variables (no prefix needed)
26    os.environ["DB__HOST"] = "db.example.com"
27    os.environ["DB__PORT"] = "3306"
28    os.environ["REDIS__HOST"] = "redis.example.com"
29
30    # Set CLI arguments
31    sys.argv = ["app.py", "--port", "9000"]
32
33    cfg = Config(
34        model=AppConfig,
35        sources=[
36            sources.Env(),  # Priority 1 (overrides defaults)
37            sources.CLI(),  # Priority 2 (highest, overrides env)
38        ],
39    )
40
41    app = cfg.load()
42
43    print("Configuration loaded:")
44    print(f"  App: {app.host}:{app.port}")
45    print(f"  DB: {app.db.host}:{app.db.port}/{app.db.database}")
46    print(f"  Redis: {app.redis.host}:{app.redis.port}")
47
48if __name__ == "__main__":
49    main()

Expected Output:

Configuration loaded:
  App: 0.0.0.0:9000
  DB: db.example.com:3306/mydb
  Redis: redis.example.com:6379

Common Pitfalls

Pitfall 1: Forgetting default_factory for nested objects

@dataclass(frozen=True)
class AppConfig:
    host: str = "0.0.0.0"
    db: DBConfig = None  # This will be None!

# If no source provides db values, app.db will be None
app = cfg.load()
print(app.db.host)  # AttributeError: 'NoneType' object has no attribute 'host'

Solution: Use field(default_factory=lambda: DBConfig()) to provide default nested objects.

Pitfall 2: Wrong separator in environment variables

os.environ["DB_HOST"] = "db.example.com"  # Single underscore

cfg = Config(
    model=AppConfig,
    sources=[
        sources.Env(),  # Looking for DB__HOST (double underscore)
    ],
)
# DB_HOST won't be recognized as nested key (becomes flat key "db_host")

Solution: Use double underscore (__) as separator for nested keys in environment variables. For db.host, use DB__HOST.

Pitfall 3: Mixing flat and nested keys incorrectly

os.environ["DB_HOST"] = "db.example.com"  # This becomes flat key "db_host"

# If your model has db.host (nested), this won't map correctly
# Use DB__HOST instead

Solution: Always use the correct separator (__ for env, - for CLI) for nested keys.

Best Practices

  1. Use default_factory for nested objects: Ensures defaults are always available

  2. Use consistent naming: Keep nested structure consistent across sources

  3. Document your structure: Comment complex nested configurations

  4. Test with different sources: Verify nested keys work from all sources

Next Steps

Now that you understand nested configuration, let’s add Validation to ensure configuration values are correct.