injinja 🥷

Injinja: Injectable Jinja Configuration tool.

Insanely configurable... config system.

Features

• Recursive Deep Merge: Sick of scrolling giant mega config files? Would you rather have organised folders of smaller configs that act like they are part of the same big config? We recursively deep merge your config.
• Powerful: Any of your config files are now empowered with the full programming capabilities of Jinja templating engine for better parametrising the one set of config files.
• Flexible: You design your configuration schema yourself in any of JSON, YAML or TOML.
• Enabling: Separate your project config out to allow "Drive By Contributors". Even non-technical colleagues can edit a text file in a browser to raise a PR.
• Nothing Implicit: By design nothing is implicit and every input is explicit. We prefer powerful tools rather than "magic" tools.

Simplified Architecture

1. Dynamic Configuration: Environment variables and CLI flags provide runtime values
2. Static Configuration: YAML/JSON/TOML files that can themselves be Jinja templates
3. Schema Validation: Validate merged configuration against schemas before templating
4. Template Rendering: Apply the validated configuration to your target template

Quick Example

Step 1: Config files with the power of Jinja

config/databases.yml

databases:
  {{ prefix | default('') | upper }}{{env_name | upper}}_BRONZE:
    description: Raw ingestion layer of our medallion architecture. Read only access for dbt. Write only for ingestion tools.

  {{ prefix | default('') | upper }}{{env_name | upper}}_SILVER:
    description: Primary data modelling area manmaged by dbt.

  {{ prefix | default('') | upper }}{{env_name | upper}}_GOLD:
    description: Curated and well modelled data. Read only access to BI tools.

Step 2: Templating foundational files with your convention

sql/databases.sql.j2

-- Create Databases
{% for database_name, database_properties in databases.items() -%}
CREATE OR ALTER DATABASE {{ database_name }}
    {%- if database_properties.description is defined %}
    WITH COMMENT = '{{ database_properties.description }}'
    {%- endif -%}
    ;
{% endfor %}

Step 3: One config for every environment (and branch)

3a Generate PROD SQL statements

# Template SQL DDL with dynamic environment variables and static config
uvx injinja \
-e env_name=prod \
-c 'config/*' \
-t sql/databases.sql.j2

-- Create Databases
CREATE OR ALTER DATABASE PROD_BRONZE
    WITH COMMENT = 'Raw ingestion layer of our medallion architecture. Read only access for dbt. Write only for ingestion tools.';
CREATE OR ALTER DATABASE PROD_SILVER
    WITH COMMENT = 'Primary data modelling area manmaged by dbt.';
CREATE OR ALTER DATABASE PROD_GOLD
    WITH COMMENT = 'Curated and well modelled data. Read only access to BI tools.';

3b Generate DEV Databases per branch

# Inject normalised git branch prefix
uvx injinja \
-e env_name=dev \
-e prefix="$(git symbolic-ref --short HEAD | sed 's/\//_/g')__" \ 
-c 'config/*' \
-t sql/databases.sql.j2

Assuming our git branch was JIRA-123/incremental-model:

-- Create Databases
CREATE OR ALTER DATABASE JIRA-123_INCREMENTAL-MODEL__DEV_BRONZE
    WITH COMMENT = 'Raw ingestion layer of our medallion architecture. Read only access for dbt. Write only for ingestion tools.';
CREATE OR ALTER DATABASE JIRA-123_INCREMENTAL-MODEL__DEV_SILVER
    WITH COMMENT = 'Primary data modelling area manmaged by dbt.';
CREATE OR ALTER DATABASE JIRA-123_INCREMENTAL-MODEL__DEV_GOLD
    WITH COMMENT = 'Curated and well modelled data. Read only access to BI tools.';

(Optional) Step 4: Run the templated output

This command pipes the output directly from stdout to stdin of the snow sql cli tool.

uvx injinja ... | snow sql -i

Summary

Now you can see how you can have enhanced configuration which stays as the central source of truth for the structure of your environments and then parametrise as need be.

Next Steps

Ready to dive in? Check out our Installation Guide and Quick Start to get up and running in minutes.

For a deeper understanding of the architecture and advanced usage patterns, explore our User Guide and Architecture documentation.