ASA TCP state bypass

What it does By default an ASA does stateful inspection of all traffic. It must see the entire conversation to be able to set up the connection and pass the traffic. If traffic is asymmetric, such that the ASA only sees traffic in one direction, the packets will not be passed. Additionally, even if the traffic is symmetric and a new connection is established, subsequent fast path packets will be inspected for things such as TCP sequence number randomization, TCP normalization, and other checks....

April 17, 2021 · 3 min · Jason Lavoie
Multi-region dual-stack TGW/DXGW design

Where AWS IPv6 networking fails

Introduction AWS has made much progress over the years with IPv6 support. From S3, EC2, Cloudfront, Route53, and EC2 support back in 2016, to more recent updates to NLB and the EC2 API, I’ve appreciated every advancement and patiently waited for the next. Unfortunately, there are still pieces missing that prevent me from making full use of IPv6 in my employer’s current environment. Existing architecture The architecture is modeled after one of AWS’s recommended connectivity designs....

April 15, 2021 · 4 min · Jason Lavoie

iproute2 blackhole route

Today I was doing some empirical testing of an application’s behavior when one of its authentication servers becomes unreachable. I typically do this with a null route on an upstream device, but noticed that iproute2 has this built in with a nice, memorable syntax. According to ip-route(8), one of the route types is blackhole: blackhole - these destinations are unreachable. Packets are discarded silently. The local senders get an EINVAL error....

April 14, 2021 · 1 min · Jason Lavoie

Iteration in Puppet

The problem In the process of modernizing our puppet infrastructure, I’ve been able to remove/delete many of the custom modules I had written many years ago and use community developed and supported modules from the puppet forge. Many of these modules accommodate a pattern of a single include in the manifest, and putting all the site-specific configuration data (if any) in hiera. That said, some don’t allow this, and resources must be explicitly configured....

April 2, 2021 · 3 min · Jason Lavoie

Terraform state replace provider

I recently had a revisit an old terraform project and update it. I had built a dev environment for our applications team, and they wanted to move it to production. Typically, whenever I go through a process like this, I take the opportunity to update things like pre-commit hooks and bump the terraform version to the most recent stable release. This happened to be a migration from a 0.12.x to a 0....

March 31, 2021 · 2 min · Jason Lavoie
Cisco 3850 model number sticker

Cisco switch model changes after licensing

When the licensing is updated on certain Cisco switches, the reported model number also changes. One of my coworkers ran into this issue recently while trying to coordinate an RMA with TAC for a 3850 switch. He replicated this in the lab and sent me some screenshots of his terminal session to document what he saw. I thought I’d share it here to help others. Out of the box, with the ipbase license, the switch shows up as an “-S” model....

March 22, 2021 · 1 min · Jason Lavoie

Cleaning up old git branches

We make heavy use of puppet environments in our workflow. Using r10k, git branches are magically mapped to environments. This allows a process where anyone one the team can individually work on a new feature or change, and then we can collaborate and review/revise/test in a controlled manner. We can rebase to the production branch, and use the diff output as part of our change-management documentation. Once the change is merged, however, sometimes the original branch is not deleted....

March 11, 2021 · 2 min · Jason Lavoie

Terraform validate list object

Since version 0.13, terraform has support for custom validation rules for input variables. The example in the documentation shows how to test a single value: variable "image_id" { type = string description = "The id of the machine image (AMI) to use for the server." validation { # regex(...) fails if it cannot find a match condition = can(regex("^ami-", var.image_id)) error_message = "The image_id value must be a valid AMI id, starting with \"ami-\"....

March 8, 2021 · 1 min · Jason Lavoie
Terraform lock file error

Terraform providers lock

As of version 0.14, terraform now produces a .terraform.lock.hcl file to record which versions of dependencies – currently, just providers – were chosen when terraform init was run. They recommend adding this file to your version control system so that all future runs will use and verify those same dependencies. These can be manually upgraded by running terraform init -upgrade. I commonly will develop locally and generate the lock file on my Mac....

March 3, 2021 · 2 min · Jason Lavoie

Ensuring PXE at every boot

By default VMware virtual machines only PXE boot on first install. Once an operating system has been installed on the hard drive, it will boot that and never try to network boot again. This is due to the default BIOS boot order. By changing the boot order, they can be configured to try a network boot first and after a short timeout boot from disk. In the vSphere client, find the VM in question, and chose Edit Settings....

February 24, 2021 · 2 min · Jason Lavoie
example TextFSM template with syntax highlighting

Writing a vim syntax plugin

Motivation I was creating a TextFSM template, and was disappointed with the lack of syntax highlighting support for this filetype in my favorite text editor. Typically, this is just a plugin away, but an exhaustive search turned up nothing. So, like all good geeks, I rolled up my sleeves and made one myself. Process Over the years, I have written little snippets in my .vim/syntax/ directory, or added some missing syntax to someone else’s syntax plugin, but had never written one from scratch....

February 23, 2021 · 6 min · Jason Lavoie
PXEboot bootscreen

Start Puppet in Debian Preseed

I have a nice netboot setup where we can PXEboot hosts to an automated installer. The last step ensures the puppet agent is running and pointed at the correct puppetmaster. The .preseed files are generated from an erb template that ends in the following: [...] <% if @distcodename == "jessie" -%> d-i preseed/late_command string \ echo -e 'DAEMON_OPTS="--server <%= @puppetmaster %>"' > /target/etc/default/puppet ; \ rm -f /target/var/lib/puppet/state/agent_disabled.lock <% else -%> d-i preseed/late_command string \ in-target sed -i '/\[main\]/a server = <%= @puppetmaster %>' /etc/puppet/puppet....

February 19, 2021 · 1 min · Jason Lavoie
GitHub Drift Success

Terraform Drift Detection with GitHub Actions

The Problem A common issue with infrastructure as code, is that it is often possible for someone to go in after deployment and manually change things. I still want to preserve the ability for the infrastructure folks to go in and make emergency changes, but I also want to discourage this practice as much as possible. To this end, I’ve been using a pattern where any “out of band” changes are alerted to the rest of the team....

September 15, 2020 · 3 min · Jason Lavoie