Starting out in my NetDevOps journey I didn’t really have much love for IaC frameworks *cough ansible cough* – the whole concept of trying to implement code-logic in a domain-specific language didn’t sit right with me. Ansible just felt like unnecessary abstraction, any attempt at trying to implement any advanced logic and the wheels quickly fell off, not to mention the debugging hell. For IaC I much prefer something like Nornir where plays can be written in an actual Turing complete programming language.
However for all its faults, I’ve learned to like Ansible – and by the same token, I’ve learned to really like Terraform. I suppose where Terraform provides advantages is its ability to provide a full configuration lifecycle system and really allow us to implement the concept of immutable infrastructure practices (yeah… so can netconf with a full config-replace I suppose) however If you are using Terraform to manage your public/private cloud presence then I suppose it can be helpful to use the same toolset for your traditional network infra as well as cloud resources.
A few vendors have produced officially maintained terraform providers for various products. However one of the issues is network vendors often have a large portfolio of different platforms and associated software versions – creating/maintaining terraform providers for each of these would be an absolutely massive task. The clever bods over at Juniper Networks realized this and have created a tool that allows customers to create their own ‘lean/targeted’ Terraform providers based on the features/platforms/software versions they are running.
JTAF uses the publicly available Juniper YANG models to create terraform providers. As the project is pretty new there is are a limited number of examples around. I thought I’d go through my experience of creating a terraform provider for Juniper vSRX 19.2R1 to manage security policy.
Pre-Reqs
There are 2 application pre-reqs that are required before we can use JTAF:
- pyang (https://pypi.org/project/pyang/)
- go (https://go.dev)
Please ensure Golang and Pyang are installed and operational on your machine before proceeding
Downlaod JTAF/YANG
The first place to start is by actually downloading JTAF tool itself.
Clone the repo (https://github.com/Juniper/junos-terraform) to your local machine as below:
Also download the Junper YANG model repository (https://github.com/Juniper/yang.git) – These Yang models are the baselines from what our custom provider will be based upon.
Environment Setup
create a JTAF config file in the TOML markup format, this will instruct JTAF to where the YANG models, xpath file, and output provider directory lives. Give your provider a name, ours will be called “vsrxsec”
Please ensure that the output directories exist as JTAF will look for these as part of the provider creation process
craig@ubuntu:~/Desktop/jtaf-exampes$ cat config.toml
yangDir = "/home/craig/Desktop/jtaf-exampes/yangfiles"
providerDir = "/home/craig/Desktop/jtaf-exampes/terraform_providers"
xpathPath = "/home/craig/Desktop/jtaf-exampes/xpath_sample.xml"
fileType = "text"
providerName = "vsrxsec"Identify Required YANG Models
In order to build our provider, we need to identify the required YANG models that relate to the resources we want to create. How I seem to do this is by looking at the Juniper CLI in XML and deducing the hierarchy from there – I’m sure there are much more elegant solutions but that was the way I did it 🙂
Looking at the below – in order to create the address-book we are under the security highrachy in the structure.
Identify the YANG models from the downloaded YANG files repository and copy these over to the yangDir path that is denoted in your config.TOML file
One thing I did notice is that there is a dependency on the ‘common’ yang models for the software version of your choosing, ensure these are copied to the yangDir path with the targeted YANG models, In the interest of eliminating any dependencies I am copying all YANG models for junos-es (srx) for 19.2 into our yangDir folder.
Process YANG
Now the YANG files have been identified and copied to the required working directory, we can use the first half of the JTAF tool to process these yang modules into their counterpart YIN (an XML representation of YANG) and an XPath reference file.
Navigate to the cmd/processYANG directory of the cloned JTAF repo and build the go file with go build. This will present an executable binary that we can use to process the identified YANG files.
Run the process yang binary with the -config flag set to the location of the config.toml file, this will allow JTAF to find the location of the identified YANG models. Now play the waiting game! … This will take a bit of time go and have a shower, hot bath, cook dinner, grab a 20 min power nap … whatever takes your fancy 🙂
Identify Xpaths
After the process YANG section of JTAF has run then there should be equivalent .yin and xpath files for each of the YANG modules that are in the yangDir directory indicated in the config.toml file
.yin files are simply xml representations of the YANG data
The xpath files will help us in identifying the resource capabilities that we want to build out in our provider.
As we are looking to manage security policy lets identify the required xpaths that we need to feed into JTAF to ensure it builds out the provider correctly. As shown above the best way I have found to do this is to actually build out some configuration in Junos first and then look at the XML representation of this to deduce the hierarchy (don’t worry you don’t have to commit it, just rollback 0!)
one of the ‘guidelines’ for JTAF is to build using the smallest unit of possible concern – so instead of identifying the top-level ‘/security’ xpath, lets try to be as granular as possible in identifying the xpaths we need.
Taking the below, to find the xpath for the ip-prefix address book we need to identify the security/address-book/address/ip-prefix xpath string
Lets open the junos-es-conf-security@2019-01-01_xpath.txt and validate the xpaths we require, we can see that security/address-book/address/ip-prefix is a valid xpath expression
We can continue the same logic to identify all the required xpaths we need as shown below and use these to build out our terraform provider.
- create address-book prefix (/security/address-book/address/ip-prefix)
- match policy source (/security/policies/policy/policy/match/source-address)
- match policy dest (/security/policies/policy/policy/match/destination-address)
- match policy application (/security/policies/policy/policy/match/application)
- apply then permit (/security/policies/policy/policy/then/permit)
Modify the xpath directory that is referenced in the config.toml file with the identified xpaths and save.
Process Providers
Now we have all the pieces in place to generate our terraform provider!
Navigate to the junos-terraform/cmd/processProviders dir and build the go file using go build and then run the binary with the config flag set to the location of the config.toml file.
This part doesnt take long at all in comparison to processing the YANG data that we did earlier
Now all the component pieces have been generated to actually build the terraform provider from the source. Navigate to the terraform_providers output directory that was specified in the config.toml file and build the provider as below:
Great, we have compiled the provider binary – we just now need to add it into the terraform plugins directory so that we can reference it in our HCL!
Using the provider
I suppose one of the things that JTAF doesn’t generate is the documentation required to actually use the provider. However, this is relatively trivial to figure out, by looking at the Golang source files for the provider you can deduce the required inputs if you know how to read a Golang struct.
I have written some HCL that leverages our new provider (on my github for reference) lets run through the terraform lifecycle with our new provider.
Running a terraform init pulls in our new provider wth no issues.
Running terraform plan we can see the execution plan of what resources will be created.
Running terraform apply creates our resources
Lets check the configuration in our vSRX – we can see the security policy has been created successfully!
Note on Mutability…
If we require to change any of the configurations that is managed via terraform on our devices, we need to taint the commit resource for that device using ‘terraform taint …..’ to ensure the commit resource is recreated.
Conclusion
Well that’s it, creating a custom terraform provider using JTAF! I hope you found this somewhat useful, if you want to check out the HCL I used its all on my github (https://github.com/thecraigus/jtaf-terraform-labbing)
I also encourage you to check out Chris Russells JTAF blog also! https://nifry.com/2022/02/18/junipers-terraform-automation-framework-jtaf/
RouteDefault: Musings of a Network Engineer 