Multiple Sources

In this tutorial, you’ll learn how to combine configuration from multiple sources, understanding how priority works and how later sources override earlier ones.

Learning Objectives

By the end of this tutorial, you’ll be able to:

  • Load configuration from multiple sources (defaults, environment, CLI)

  • Understand source priority and override behavior

  • Use the convenient Config.from_model() method

Step 1: Understanding Source Priority

Varlord merges configurations from multiple sources. Later sources override earlier ones. Let’s see this in action:

 1import os
 2from dataclasses import dataclass, field
 3from varlord import Config, sources
 4
 5@dataclass(frozen=True)
 6class AppConfig:
 7    host: str = field(default="127.0.0.1")  # Default
 8    port: int = field(default=8000)          # Default
 9    debug: bool = field(default=False)       # Default
10
11# Set environment variable (no prefix needed - filtered by model)
12os.environ["HOST"] = "0.0.0.0"
13os.environ["PORT"] = "9000"
14
15cfg = Config(
16    model=AppConfig,
17    sources=[
18        sources.Env(),  # Model auto-injected, defaults applied first, then env overrides
19    ],
20)
21
22app = cfg.load()
23print(f"Host: {app.host}")  # From env: 0.0.0.0
24print(f"Port: {app.port}")  # From env: 9000
25print(f"Debug: {app.debug}")  # From defaults: False

Expected Output:

Host: 0.0.0.0
Port: 9000
Debug: False

Key Points:

  • Model defaults are automatically applied first (lowest priority)

  • Environment variables override defaults

  • Only environment variables that match model fields are loaded

  • Fields not in environment keep their default values

Step 2: Adding Command-Line Arguments

Command-line arguments have the highest priority (when listed last):

 1import sys
 2import os
 3from dataclasses import dataclass, field
 4from varlord import Config, sources
 5
 6@dataclass(frozen=True)
 7class AppConfig:
 8    host: str = field(default="127.0.0.1")
 9    port: int = field(default=8000)
10    debug: bool = field(default=False)
11
12# Set environment variable
13os.environ["PORT"] = "9000"
14
15# Simulate command-line arguments
16sys.argv = ["app.py", "--host", "192.168.1.1", "--port", "8080", "--debug"]
17
18cfg = Config(
19    model=AppConfig,
20    sources=[
21        sources.Env(),         # 1. Environment (overrides defaults)
22        sources.CLI(),         # 2. CLI (highest priority, overrides env)
23    ],
24)
25
26app = cfg.load()
27print(f"Host: {app.host}")   # From CLI: 192.168.1.1
28print(f"Port: {app.port}")   # From CLI: 8080 (overrides env)
29print(f"Debug: {app.debug}")  # From CLI: True

Expected Output:

Host: 192.168.1.1
Port: 8080
Debug: True

Priority Order (lowest to highest):

  1. Model defaults (automatically applied)

  2. Environment variables

  3. Command-line arguments

Step 3: Using Config.from_model() Convenience Method

Varlord provides a convenient method to set up common sources:

 1import os
 2from dataclasses import dataclass, field
 3from varlord import Config
 4
 5@dataclass(frozen=True)
 6class AppConfig:
 7    host: str = field(default="127.0.0.1")
 8    port: int = field(default=8000)
 9    debug: bool = field(default=False)
10
11# Set environment variables (no prefix needed)
12os.environ["HOST"] = "0.0.0.0"
13os.environ["PORT"] = "9000"
14
15# Convenient setup
16cfg = Config.from_model(
17    model=AppConfig,
18    cli=True,  # Enable CLI arguments
19)
20
21app = cfg.load()
22print(f"Host: {app.host}")
23print(f"Port: {app.port}")

Expected Output:

Host: 0.0.0.0
Port: 9000

Benefits of ``Config.from_model()``:

  • Less boilerplate code

  • Automatically sets up common sources

  • Model is automatically injected to all sources

  • Model defaults are automatically applied

Step 4: Environment Variable Naming

Environment variables are normalized to lowercase and use dot notation for nested keys:

 1import os
 2from dataclasses import dataclass, field
 3from varlord import Config, sources
 4
 5@dataclass(frozen=True)
 6class AppConfig:
 7    host: str = field(default="127.0.0.1")
 8    port: int = field(default=8000)
 9
10# Environment variables (filtered by model - no prefix needed)
11os.environ["HOST"] = "0.0.0.0"
12os.environ["PORT"] = "9000"
13os.environ["UNRELATED_VAR"] = "ignored"  # Will be filtered out
14
15cfg = Config(
16    model=AppConfig,
17    sources=[
18        sources.Env(),  # Model auto-injected, only loads HOST and PORT (filtered by model)
19    ],
20)
21
22app = cfg.load()
23print(f"Host: {app.host}")
24print(f"Port: {app.port}")

Key Mapping Rules:

  • HOST → host (lowercase, matches model field)

  • PORT → port (lowercase, matches model field)

  • UNRELATED_VAR → ignored (not in model)

  • For nested keys: DB__HOST → db.host (__ becomes .)

Step 5: Command-Line Argument Format

Command-line arguments use kebab-case and are converted to dot notation:

 1import sys
 2from dataclasses import dataclass, field
 3from varlord import Config, sources
 4
 5@dataclass(frozen=True)
 6class AppConfig:
 7    host: str = field(default="127.0.0.1")
 8    port: int = field(default=8000)
 9    debug: bool = field(default=False)
10
11# Command-line arguments
12sys.argv = [
13    "app.py",
14    "--host", "0.0.0.0",
15    "--port", "8080",
16    "--debug",  # Boolean flag (no value needed)
17]
18
19cfg = Config(
20    model=AppConfig,
21    sources=[
22        sources.CLI(),  # Model auto-injected, only parses model fields
23    ],
24)
25
26app = cfg.load()
27print(f"Host: {app.host}")
28print(f"Port: {app.port}")
29print(f"Debug: {app.debug}")

Expected Output:

Host: 0.0.0.0
Port: 8080
Debug: True

CLI Argument Rules:

  • Use --field-name for regular fields

  • Use --flag for boolean True, --no-flag for boolean False

  • Arguments are automatically converted to the correct type

Step 6: Complete Example

Here’s a complete example combining all sources:

 1import os
 2import sys
 3from dataclasses import dataclass, field
 4from varlord import Config, sources
 5
 6@dataclass(frozen=True)
 7class AppConfig:
 8    host: str = field(default="127.0.0.1")
 9    port: int = field(default=8000)
10    debug: bool = field(default=False)
11    app_name: str = field(default="MyApp")
12
13def main():
14    # Set environment variables (no prefix needed)
15    os.environ["PORT"] = "9000"
16    os.environ["DEBUG"] = "true"
17
18    # Simulate CLI arguments
19    sys.argv = ["app.py", "--host", "0.0.0.0"]
20
21    # Create configuration with multiple sources
22    cfg = Config(
23        model=AppConfig,
24        sources=[
25            sources.Env(),         # Priority 1 (overrides defaults)
26            sources.CLI(),         # Priority 2 (highest, overrides env)
27        ],
28    )
29
30    app = cfg.load()
31
32    print("Configuration (merged from all sources):")
33    print(f"  Host: {app.host}")      # From CLI: 0.0.0.0
34    print(f"  Port: {app.port}")      # From Env: 9000
35    print(f"  Debug: {app.debug}")   # From Env: True
36    print(f"  App Name: {app.app_name}")  # From Defaults: MyApp
37
38if __name__ == "__main__":
39    main()

Expected Output:

Configuration (merged from all sources):
  Host: 0.0.0.0
  Port: 9000
  Debug: True
  App Name: MyApp

Common Pitfalls

Pitfall 1: Wrong source order

cfg = Config(
    model=AppConfig,
    sources=[
        sources.CLI(),  # CLI first!
        sources.Env(),   # Env will override CLI
    ],
)
# CLI arguments will be overridden by env vars - probably not what you want!

Solution: Always put sources in priority order (lowest to highest): Env → CLI (defaults are automatically applied first).

Pitfall 2: Environment variables not matching model fields

os.environ["MY_HOST"] = "0.0.0.0"  # Doesn't match model field

cfg = Config(
    model=AppConfig,
    sources=[
        sources.Env(),  # Only loads fields defined in model
    ],
)
# MY_HOST won't be loaded because it doesn't match any model field

Solution: Use environment variable names that match model field names (normalized to uppercase). For host field, use HOST environment variable.

Pitfall 3: Type conversion issues

os.environ["APP_PORT"] = "not-a-number"

# This will raise ValueError during type conversion
app = cfg.load()

Solution: Ensure environment variables can be converted to the target type. Varlord automatically converts strings to the appropriate types.

Best Practices

  1. Model defaults are automatic: No need to include sources.Defaults

  2. All sources filter by model: Only model-defined fields are loaded

  3. Order sources by priority: Env → CLI (defaults applied first automatically)

  4. Use Config.from_model() for common setups: Reduces boilerplate

  5. Fields are automatically determined: Use Optional[T] type annotation or default values for optional fields

Next Steps

Now that you understand multiple sources, let’s learn about Nested Configuration to handle complex configuration structures.