Chris Harrington

AWS Certified

2023-04-05T00:00:00+00:00

Well it has been a bit. Like most professional blogs, this fell disused while I was focused on my “real job”.

In the mean time, though, I am happy to share that I’ve been certified as both an AWS Certified Cloud Practitioner and an AWS Certified DevOps Engineer - Professional. I’m continuing to work on building skills, and I’m taking the exam to become an AWS Certified Solutions Architect - Associate today. We’ll see how that goes. (Update: It went well! I’m now also an AWS Certified Solutions Architect - Associate.)

One thing I took away from the DevOps Engineer certification exam is that I do not know nearly enough about the AWS CodeCommit, CodeBuild, CodePipeline, and CodeDeploy products. So as a personal project, I’m thinking I’ll propose a CodePipeline process at work, to wean us away from our current CI/CD vendor. Could be fun, and I hope to learn from the process.

Directly accessing USB storage in WSL2 Debian

2022-07-30T00:00:00+00:00

I needed to access a USB storage device in WSL2 so I could install LVM and LUKS on it. The only full Linux boxes I have at my disposal don’t have USB-C, and I didn’t have an adapter, so naturally I turned to the most absurd and indirect way to solve this problem. I followed some guides partially, including this Microsoft blog and this Stack Exchange answer, but none matched my situation exactly.

Overview

The core USB-passthrough function in WSL2 is powered by USB-IP. This is a protocol that translates USB traffic into IP traffic. WSL2 uses this to host a USB device on the Windows side as a server, and then connects the device on the Linux side as a client. Once the USB device is present on the Linux side, though, the current kernel implementation does not provide USB storage out of the box.

That means we’ll need to complete a handful of tasks:

Get the WSL2 Kernel and build the usb-storage module
Install the Linux-side USB-IP tool
Install the Windows-side USB-IP tool/driver
Connect the device to Linux

My situation

You should always use caution following any guide on the internet. In particular, this guide may be out of date by the time you read it. (It may also be malicious, so be careful to verify anything a guide asks you to download or run.) So you can follow along, I have WSL2 running Debian. Inside of Linux, cat /etc/issue reports Debian GNU/Linux 11, and uname -r reports 5.10.102.1-microsoft-standard-WSL2. I’m currently running Windows 10 21H2 (as reported by running winver). If yours are similar, you’ll probably have success following this guide.

WSL prerequisites

Because we’re going to be doing stuff that is sensitive to the kernel version you’re using, it’s best if we get the most up-to-date WSL2 kernel available before we go too far. This update actually happens in Windows: you will need to run this in an elevated (Administrator) Command Prompt or PowerShell:

wsl --update

Debian prerequisites

Next, before we try to build a kernel, you will want to get some things installed by running inside your Debian shell:

sudo apt-get update
sudo apt-get install build-essential flex bison libssl-dev bc libelf-dev usb-ip

This sets up Debian to be able to compile the kernel as well as grabbing the usb-ip Linux software while we’re at it.

Linux Kernel usb-storage module

If you’re relatively seasoned with Linux, you might be searching for the comment section already. “Wait, why not use uas? It’s way faster.” This is correct, and that was my initial goal. However, it turns out that the Linux USB-IP host controller is incompatible with uas, and so we will need to fall back to the more basic variation, usb-storage.

To start, we need to check out the WSL2 kernel source code inside the Debian shell. This will take a long time. The filesystem on WSL2 is not very fast, and the git operation does millions of filesystem operations and downloads gigs of data.

git clone https://github.com/microsoft/WSL2-Linux-Kernel.git ~/wsl2k && cd ~/wsl2k

(Have you done this step before? Jump down to the cleaning up section for how to reuse your git repo to save yourself some time.)

Now, we need to check out the specific kernel revision that matches your running kernel. This one-liner tries to construct the right tag and branch to match your kernel. Don’t worry if this doesn’t work, since it will just keep using the default branch’s code, which should still match the kernel you updated to earlier (by running wsl --update).

export wslBranch=$(git ls-remote --symref https://github.com/microsoft/WSL2-Linux-Kernel.git HEAD | grep -oE 'refs/heads/[^[:space:]]+' | cut -f 3 -d /)
export wslTag=tags/linux-msft-wsl-$(grep -oE '([0-9]+\.?)+' /proc/version | head -n1)
git checkout -f -B $wslBranch $wslTag

Now we’ll unpack the existing kernel configuration, and make a couple of changes. The first is to disable the BTF debugging data, as it requires software that apt-get refused to install for me. The second is to explicitly enable the usb-storage module. The last line runs yes (but paradoxically with n) to update the kernel configuration to be satisfied with the previous change.

gunzip < /proc/config.gz > .config
sed -ix 's/CONFIG_DEBUG_INFO_BTF=y/CONFIG_DEBUG_INFO_BTF=n/' .config
sed -ix 's/^.*CONFIG_USB_STORAGE.*$/CONFIG_USB_STORAGE=m/' .config
yes n | make oldconfig

Now, we compile. If you have lots of CPU power, you can replace the first make with make -jX where X is the number of cores you want to use, and the compile may run faster.

make modules && sudo make modules_install

Another surprising problem I encountered was to find that the modules_install does not use the same location that the kernel expects. I ended up with modules in /lib/modules/5.10.102.1-microsoft-standard-WSL2+ but modprobe wanted no plus symbol at the end. So you may also need to copy your module directory into the right place.

sudo cp -av /lib/modules/$(uname -r)+ /lib/modules/$(uname -r)

Finally, to make sure the module compiled correctly and is matched with your kernel, probe it. If this command outputs nothing, you should be in business.

sudo modprobe usb-storage

Keep this terminal open in the background so your Debian VM doesn’t suspend.

Windows USB-IP

You’ll want to download and install the latest release of usbipd-win. I used usbipd-win 2.3.0 because it was the most current at the time. This will require administrative access on Windows. I didn’t need to reboot, so you probably don’t need to either.

Before you continue, note that Windows has probably mounted your USB storage or is otherwise hanging on to it. You will want to safely eject the device first, to minimize the risk that something will be upset (either Windows or Linux).

You’ll need to identify which device ID you want to connect to Linux. The easiest way is to open an elevated (Administrator) Command Prompt or PowerShell, list the devices, identify the “bus ID” of the device, and then attach it. In my case, the ID of my storage device was 1-13. You will need to substitute your own.

usbipd wsl list
usbipd wsl attach --busid 1-13

If all goes according to plan, you should be able to run usbipd wsl list again and see the state of your device change to Attached - Debian. If so, you should be able to swing back over to your Debian terminal and do the normal things, like dmesg or lsblk to see your device.

Cleaning things up

The kernel sources we checked out in ~/wsl2k, alongside the built object files from make, clock in at a staggering 6.2 gigabytes in my case. If everything is working for you, you can clear this out in your Debian shell. Note that the -f flag here is because git will create read-only files which rm will otherwise prompt you about.

cd ; rm -rf ~/wsl2k

Note that WSL2 automatically updates kernels behind your back. As a result, you may find that you are unable to sudo modprobe usb-storage anymore in the future. If that’s the case, you will need the kernel sources again. If you anticipate needing the direct USB Storage ability in the future, you should hang on to the ~/wsl2k directory. In the directions above, instead of git clone ... ; cd ~/wsl2k do cd ~/wsl2k ; git fetch ; git reset --hard ; make distclean

If you don’t need to build kernels or anything anymore, you can also remove the packages we installed. Just replace apt-get install with apt-get remove. However, be warned: you may have actually been using some of these packages already for other purposes. If in any doubt, just leave them in place. You also likely want to leave usb-ip installed unless you’re certain you won’t be using the USB passthrough function again.

To disconnect your USB device from the passthrough mechanism, you can run this in an elevated (Administrator) Command Prompt or PowerShell. Make sure you unmount any USB storage filesystems you have mounted in Debian first. The --all does just what it says, although if you only want to disconnect one passed-through device, specify the ID as before with --busid 1-13 instead. (Naturally, substitute your ID from usbipd wsl list.)

usbipd wsl detach --all

Creating a DNS-based URL Shortener [Part 1]

2022-05-09T00:00:00+00:00

It’s 2022, and nobody wants to use URL shorteners anymore. They are user-hostile, harmful to the health of the open internet, and virtually unnecessary. So let’s make one.

To avoid an aphorism, there are a lot of ways to accomplish this task. I wanted to play around with AWS Route 53 a bit, so that’s where I started when I wrote the code we’ll discuss.

What we’ll use

Go 1.18+
AWS
- ACM
- API Gateway
- Graviton2
- Lambda
- Route 53
DNS
- You should have tools like dig available
- You can alternately use mxtoolbox
Your domain, if you have one
- In this post I’ll refer to it as example.com
- If you don’t have a domain connected to Route 53, you can buy one

Set up the AWS Services

Part 1 of this series will be configuring your AWS account. If you already know your way around AWS, you can probably skip it, but this part will cover every AWS step needed to get working.

Making sure Route 53 and your domain agree

First, you’ll want to have your example.com domain pointed at your Route 53 hosted zone name servers. When you run dig -t ns example.com you should get a set of records that look the same (though not necessarily in the same order) as the NS record in your hosted zone. If it doesn’t look right, then you likely need to configure the authoritative nameservers with your domain registrar.

Now if you don’t have a domain, it turns out you can just buy one through Amazon’s Route 53 Domains. As of this writing, .link domains are $5 USD, and .click domains are $3 USD. Be the owner of risky.click for three bucks! Or don’t, it’s apparently already taken. Anyway, when you buy a domain through AWS, the console will automatically create a Route 53 hosted zone for it, skipping some faffing around.

Requesting an ACM certificate

AWS offers free SSL certificates for services running on their platform. We’ll need one, since it isn’t 1997 anymore. Request one from ACM in us-east-1:

Click “Request”.
Select (or leave selected) “Request a public certificate” and click “Next”.
Under “Fully qualified domain name”, you’ll put *.example.com – again, substituting your real domain here.
Choose DNS validation – Route 53 will make this seamless for you – then click “Request”.
You may not be able to choose your pending certificate yet, if not, refresh or visit the certificate list again.
Assuming the certificate is indeed “Pending validation”, you should be able to click “Create records in Route 53”.
You should see a summary of the records that will be created on your behalf, click “Create records”.
In a minute or two, you should be able to refresh your pending certificate and see it is now Issued.

Creating your API Gateway API

Once your certificate is issued, go visit API Gateway in us-east-1.

Click “Create API”, then click “Build” under HTTP API.
Under Integrations, skip the “Add integration” button. We’ll do that in a minute.
Invent a name that’s meaningful to you. We’ll pretend you wrote shorturl.
Click “Review and Create” and then “Create”.
You should be looking at your API. (If not, go back to the API list and click on it.) On the left edge, click “Custom domain names”, then click “Create”.
You need a domain name for your shortener. I suggest creating a subdomain for this app, like short.example.com.
Under ACM certificate, you’re going to select the certificate we requested earlier. It should look like *.example.com.
Click “Create domain name”. You should be looking at your custom domain name now. Click “API mappings” then “Configure API mappings”.
Click “Add new mapping”. Under API, select your API, the one we’re pretending you called shorturl. Under stage, select $default. Click “Save”.

Creating the Route 53 Alias

Now that you have created an API Gateway API with a custom domain, we can create a record in Route 53 to connect that together.

In your hosted zone, click “Create record”. If you see a link that says “Switch to quick create”, click that.
Under Record name, type just the subdomain part of your domain. So if your domain is short.example.com, type short.
Under Record type, A should already be selected. If it isn’t, select it.
Next to Value, turn on the toggle switch for Alias. This section will then read “Route traffic to”.
Under Route traffic to, choose Alias to API Gateway API.
In the next dropdown that appears, you’ll choose the us-east-1 region, labelled “US East (N. Virginia)”.
In the next dropdown that appears, clicking inside should reveal the API you created above. (It may just be a hostname, not the name you gave it.) If nothing appears, you either have the wrong name under “Record name” or you have the wrong name in your API Gateway Custom domain. These must match. Once you do see your API, select it.
You can leave everything else as is and click “Create records”.

At this point, you should be able to open your web browser and go to your domain short.example.com. We’re expecting to see {"message":"Not Found"}, which is perfect, because API Gateway is responding to our request, but there’s no route yet defined that matches. If that’s the result you get, you’ve done everything correctly up to this step.

Creating the Lambda function

At the heart of this very silly app is a Lambda function. It is invoked by API Gateway and updates Route 53 in order to service requests. You’ll want to head to the us-east-1 Lambda page.

Click “Create function”. Leave “Author from scratch” selected.
Under Function name, provide a name for your function. It isn’t important, we’ll pretend you called it shorturlfunc.
Under Runtime, scroll down to and select Provide your own bootstrap on Amazon Linux 2
Under Architecture, select arm64. This enables the AWS Graviton2 architecture, which is purely optional, but fun.
The remaining options are fine if left alone, go ahead and click “Create function”.

Connecting it to API Gateway

Head back to API Gateway in us-east-1, and select the API you created. It’s time to connect your Lambda function to the internet.

Expand the “Develop” section on the left side if necessary, and click “Routes”. Then click “Create”.
Under “Route and Method”, leave the drop-down set to ANY. In the next field, change / to /{x}. Click “Create”.
You should be back to the Routes view, so click “Create” again.
This time, change ANY to POST. Leave / as is. Click “Create”.
On the left side, click “Integrations”. You should be looking at the Integrations view, and one of your routes should be selected.
Click “Create and attach an integration”.
Under “Attach this integration to a route”, leave the value that’s there.
Under “Integration target”, “Integration type”, choose Lambda function.
The “AWS Region” should already be us-east-1. Under “Lambda function”, click inside the field to load functions in that region. You should see the function we created in the last section, and maybe you called it shorturlfunc. Select it.
Skip everything else and click “Create” at the bottom of the page.
You should be back to the Integrations view, and one of your routes should show “AWS Lambda” on it. Select the other route.
You should now see a view that offers a choice between creating a new integration or using an existing one. We’ll do the latter.
In the dropdown, you should see an option named like your Lambda function (eg shorturlfunc). Select it. Click “Attach integration”.

We’re pretty close now. Now visiting your domain in a browser should produce {"message":"Internal Server Error"}. This doesn’t seem like an improvement, but it is; API Gateway is trying to invoke your function, but something goes wrong. In fact, your Lambda function is missing its bootstrap program. That will come in Part 2; for now, we have a couple of small things to do.

Short detour on Route 53 for the Zone ID

Route 53 creates a unique identifier for your hosted zone, and we’re going to need it. It looks like Z01010101S99Z9ZXZXZX2, and is available in the “Hosted zone details” section above your zone records. You need this value, so go get it before we continue.

Back to Lambda to finish setup

In your Lambda function, we’ll need to tell your function which zone we modify to store URLs.

While looking at the Lambda function, find the tab strip and click “Configuration”.
Under the configuration view, on the left side, find and click “Environment variables”.
In the Environment variables view, click “Edit”. Then click “Add environment variable”.
Under “Key”, put ROUTE53_ZONE_ID. Under “Value”, paste your Zone ID we got above. (Don’t lose it, we need it for one more thing.)
Click “Add environment variable” again. Under “Key”, put ROUTE53_RECORD_TTL. Under “Value”, put 1h.
Click “Save”.
Back on the page for your function, under the configuration view, on the left side, find and click “Permissions”.
You should see a section titled “Execution role” with a link under “Role name”. Click that link, we’re headed to IAM.

Modifying the Lambda role

IAM Roles are short-term assumed identities that are scoped to work with certain services. Lambda uses them in order to grant permissions to functions as they’re executing. We’ll do that here, by granting your function permission to modify Route 53 records.

You should be looking at the “Permissions” view. Find and click the button labeled “Add permissions”. This will offer two options; click “Create inline policy”.
Welcome to the IAM Policy Visual Editor! Click on “Choose a service” and in the search field that appears, type 53.
There are several “Route 53” services, but we specifically want the original. Click the option that reads “Route 53”.
In the search field, type change, and check the box next to “ChangeResourceRecordSets”.
Expand the “Resources” section, and click “Add ARN”. In the field that appears, paste your Hosted Zone ID. Click “Add”.
Click on “Review policy”. For “Name”, type short-url-lambda-modify-route53. It doesn’t have to be exact, but policy names are restricted to letters, numbers, and a handful of symbols – no spaces.
Click “Create policy”.

And that’s it!

Wrapping up

You’ve configured all the AWS pieces more-or-less how they need to be configured. The only thing that is missing is the code to run in the Lambda function. In Part 2, we’ll check out the code, compile it, and deploy it. Then we’ll discuss what it’s doing, identify some flaws, and add a feature.

The Rigol DS1202Z-E does not support GPT on USB storage

2022-04-16T00:00:00+00:00

If you happen to own or use a Rigol osilloscope, you may have been a victim of the error message:

Flash Drive Not a DOS Disk,Please Format Flash Drive!

I looked up a few threads, including this top Google result, that were all over the place: You didn’t eject the drive before you took it out of your computer. You can’t use USB3 drives with your scope. Some drives just are too big, or too old, or too new, or they simply aren’t compatible for some other mystery reason. The answer these (and other) threads seem to arrive at is always: just keep trying new USB sticks until you find one that works.

As an IT person, I don’t like this explanation. It doesn’t actually solve any problem, and it definitely doesn’t prevent it from happening in the future. After encountering this issue myself and actually finding the real reason, I’d like to share it with you.

My Rigol DS1202Z-E (and possibly other Rigol scopes) does not support USB disks that use a GUID Partition Table, aka GPT. It’s not worth your time for me to explain what this is, why it’s on your drive, or why you would (or wouldn’t) want it, and so on. The most important part to understand is that you don’t need to buy a new USB stick, and “formatting” the drive, as the on-screen diagnostic requests, will not help. At least, not exactly.

Fixing your USB drive

The single most important part of this article is to be adamantly clear: you cannot fix your drive without erasing it. All of the steps below carry the risk of data loss, so you need to proceed carefully.

Fixing it on Windows 10

You’re going to need to open an elevated command prompt or PowerShell window. You can do this by holding the Windows Key and tapping X, you should get a menu offering “Windows PowerShell (Admin)” or “Command Prompt (Admin)”. Both of these are fine, so whichever one you have, select that. If your computer is configured to ask permission before elevating permission (as it should) you will be asked to confirm, either by clicking “Yes” or entering your admin-level password.

At this point, you want to start with the USB disk disconnected. This is critical; we are dealing with tools that could potentially erase your hard drive, so we need to be certain which drive is which. Once you’ve made sure the USB disk is definitely not inserted into your computer, you’re going to run diskpart (and press enter) in your admin command window. This will take a moment, and when it’s ready, you will see a DISKPART> prompt. Here, you will type list disk and press enter; this will display all of the disks in your computer that you absolutely do not want to ruin. Pay attention to the numbers.

Go ahead and insert your USB stick. Give your computer a moment to collect itself, and go back to that window and type list disk again. (Don’t forget to hit enter.) You should see another entry, and the new entry size should be about the size of your USB device. (As a caveat: the companies that make storage devices have changed how they define “gigabyte” over the years, and some of them also slightly fudge their numbers. In this screenshot, my device is labeled “16GB” but diskpart says it has “14 GB”.)

Note in the screenshot the asterisk under the Gpt column; this corresponds to a GPT-layout device. This is what is causing my problem with the Rigol. If your USB device does not have this asterisk, stop here. My instructions may do more harm than good for you. You can just close the command window to exit the disk erasing danger zone.

The next step is to select the disk. In my screenshots, I am using disk 2 because that’s where it is showing up for me, but it will likely show up as a different number for you. If your USB device isn’t disk 2 then you need to use your disk number in the following command.

select disk 2

then

list disk

This verifies we really, really have the right disk selected. You’ll see an asterisk on the left edge of the disk you selected. Now, be aware: the following step will destroy data. If you have files you want to keep on this USB device, pause here and copy them off the device.

clean

This step erases all the partitions on the device, and prepares the device to convert it to the more basic MBR-layout.

convert mbr

This changes the device to use MBR.

list disk

Now you should see your disk is missing the asterisk in the Gpt column. This means it worked! A couple more steps while we’re here to make the Rigol happy.

create partition primary

This creates a new partition, which is the thing you actually are working with when you see a Drive in Windows. C: is a partition on a disk that may have several partitions. This will create an empty partition, we need to format it though. (Note that partition 1 here is always going to be the same; we had zero partitions before, and 1 is the first one which we just created.)

select paritition 1

format fs=fat32 quick

This selects the new partition we made, and then puts a FAT32 filesystem on it. Windows may pop up your new empty USB Drive now, and you should be all set. You can close the Command window, eject the drive, and jam it into your Rigol meter.

What is a side-channel attack?

2020-07-05T00:00:00+00:00

A side-channel attack is a name for a broad class of different security challenges. The obligatory Wikipedia article does a good job of discussing it, especially in a technical sense. Here I’ll discuss a contrived example that lays out the concept, but doesn’t actually get into computing at all.

There’s Got to Be a Way

Let’s invent a scenario: It’s the 1990’s, and you are a private investigator working for a big bank company. This bank has noticed small sums of cash vanishing from several locations on a predictable schedule, and they suspect some employees are cooperating to smuggle it out of the location. Now, all of the cash disappearing from the vault is treated with a chemical… but there’s a problem. All of the employees know that this chemical can be washed off in a typical home clothes washing machine. It just takes a particularly heavy duty cycle.

The bank is emphatic that all the employees involved in the heist need to be caught at the same time. That means: no breaking down doors or tense Law & Order-style interviews with suspects. We need to detect as many of them as possible without notice. “No problem,” you say, “I can detect if they’re removing that chemical without ever entering their home, or even looking in a window.”

The trick is to measure the consumption of power. There’s two means to accomplish this. Assuming the residential power isn’t buried, the first method would involve an individual with a cheap ladder, a plausible high-visibility vest, and a clamp-on ammeter. If that’s not possible, building-external electricity meters can be observed from a distance with a reasonably capable pair of binoculars. Ever noticed that little wheel inside that turns kind of slowly and has the large dark mark on one part? That wheel turning once represents some fixed amount of power consumption. (Well, in the 90’s, anyway. Today the purely digital ones are more common, but they have similar indicators.)

All the Small Things

Homes are full of devices that consume power. Let’s take a look at a few classes of power consumers.

Continuous, unvarying

A lot of power consumption in a typical house is pretty boring. Lights, fans, televisions, these all draw a reasonably consistent amount of power during operation, and many of these are left on for long periods of time. A 60 watt lightbulb (remember those?) will consume that amount of power, ramping up nearly instantly to its peak consumption, and immediately dropping off to zero consumption when it’s switched off again.

Periodic

Now consider a space heater: it will draw a lot of power, with a relatively predictable on/off cycle. If it’s rated for 1800 watts, you’d expect an increase of 1800 watts on a power meter for (say) 10 minutes, then a drop of 1800 watts for 2 minutes, then an increase again for 10 minutes, and so on. This ends up being pretty predictable, too.

Non-periodic

Here’s where things get good. A device like a washing machine has varied cycle programs designed to be good for washing clothes. The machine will run several motors on and off with varying speeds, in bursts of different lengths, all with widely different loads. The consumption of the pump motor is different than the consumption of the basket spinning motor at low speed, and that consumption is different still from the basket spinning motor at high speed.

I Learned from the Best

To get a good idea of what to look for, you need to take some measurements of your own washing machine. Naturally you will want to load the unit with actual cash to make sure the behavior of your machine is similar to the behavior of the employees you’re trying to monitor. Taking detailed recordings of the power consumption of your machine, the important values to record are not the absolute amounts, but their relative scale. Making sure to measure only the consumption of your machine, you can create a sort of power fingerprint for this activity. Ideally, you will want to measure this several times, gathering several samples and seeing if there’s variation in your machine. In a perfect world, you might even buy several machines and measure them all, but you will likely find they are all very similar on their most “heavy duty” settings.

When I Come Around

Now your task is simple. We know what the consumption pattern of a machine running its heaviest duty cycle looks like. You now task your fleet of interns (oh, uh, let’s say you have interns.) to remotely record the power consumption of each suspect house every evening after the bank closes. They gather enough data to plot, minute-by-minute, the consumption pattern of their assigned home.

It’s important to capture data even on nights where no cash goes missing. We will want to know if these employees are just always doing lots of laundry. Or if they have some other strange influence on their power usage, which may be due to other, unrelated activities.

Because people use their homes for things other than laundry, the data will be noisy, but that’s okay. Remember that the bulk of the things inside a home will either be continuous or periodic; both modify our target’s consumption by a relatively fixed, predictable amount. We know if we see consumption jump up by 1000ish watts, we can probably filter that out by looking at the following data just with that as the new “bottom” of our scale. What we’re looking for here is jumps up and down in consumption that match our own pattern of measurements, not absolute values.

We’ve Got It Goin’ On

The name of the game at this point is “correlation”. We have a pattern of power usage we can look for, we have a set of measurements from homes that may or may not be participating in this heist, we have the schedule where the cash vanishes, and that’s all we need for our side-channel attack. Using this information, we can detect with reasonably high probability when a home is doing this particular task of washing the cash, and from there, we can deduce which employees are in on the heist.

Mostly.

Of course part of the problem with a contrived hypothetical scenario is that it deliberately elides inconvenient details.

I Don’t Want to Miss a Thing

So how inaccurate is our goofy example? Well, if you’re talking about attacking a computer CPU, it’s not too bad! You can think of the whole home as the CPU itself, and the home appliances & washing machine as representing internal processor structures. Our initial measurement that we did on our own washing machine would be analogous to measuring power usage with our CPU as otherwise idle as possible. Note that while in our example we measure the power consumption of the CPU, there are other things we can measure, such as execution times. There’s a surprising amount of information available to gather about the state of a CPU: even the temperature of a certain core can be a kind of data point, as it relates to power consumption and workload.

Of course, washing machines can have a lot of variations, but so do many of the targets of a side-channel attack. Some attacks are highly targeted and limited to a specific environment, but some can be generalized. Some attacks are general enough that they can run inside of a script on a web page, detecting with a reasonable confidence what kind of processor they’re running on, and then tuning their attack from there.

However, one thing our scenario ignores is repetition. Most side-channel attacks are, at their core, statistical. This means taking hundreds or thousands of samples, repeating the same task over and over again. Indeed, attacks leveraging Spectre generally need to trigger the execution of some code millions of times just to exfiltrate a few hundreds of bytes. And while some side-channel attacks can be totally passive, the majority have some component that can trigger the behavior we want to spy on. For example, if some malware wants to find the encryption key a browser session is using, it may repeatedly trigger a reload of something in that session.

The great thing about statistical analysis is that, with enough of it, you can make other sources of noise in your data invisible. And the great thing about computers is that they can do things over and over really quickly.

TechRepublic has a pretty detailed article on Spectre and Meltdown that makes for pretty good reading.

Detecting duplicate images - Part 2: The naive way

2020-07-04T00:00:00+00:00

Let’s start by defining some data types we can use for output. Go has some great standard library support for JSON encoding which makes this pretty painless, and if we optimize for this encoding, most other encoding packages will be easy to implement.

In order to achieve our goal, the minimum we need to know is:

What the two images we’re comparing are, and
How different they are.

Let’s define a Subject type and a Comparison type.

type Subject struct {
	Filename string
}

type Comparison struct {
	Distance uint
	SubjectA *Subject
	SubjectB *Subject
}

To start out, we’ll assume all Subjects are local files. It’s useful to abstract this away now at this step, though, since we can keep the notion of a Subject later as we add other sources or interfaces.

What does the Distance member of Comparison mean? How different is a value of 1 versus a value of 65536? For now, we won’t make any concrete assertions, except to say that a larger value indicates a larger difference, and relatively small values indicates “similar”. The only value we can define concretely is to say that 0 means “identical”.

Of course we will need to track all of our Subjects, so let’s make a type for that.

type SubjectList struct {
	list []*Subject
}

func NewSubjectList() *SubjectList {
	// Assume for now we'll have the same number of subjects as arguments
	return &SubjectList{list: make([]Subject, 0, flag.NArgs())}
}

func (s *SubjectList) AddFile(path string) error {
	s.list = append(s.list, &Subject{Filename: path})
	return nil
}

How do we specify Subjects? Let’s start by passing filenames to an invocation of our program. Once again the standard library makes our lives easier by providing the flag package. Using flag now lets us add command-line options later. Also, for the sake of convenience, let’s allow the user to pass in a list of files, or directories, or a combination of both. I like to handle this with the path/filepath package.

func main() {
	flag.Parse()
	sl := NewSubjectList()
	for _, fn := range flag.Args() {
		filepath.Walk(fn, sl.filewalkfunc())
	}
}

func (s *SubjectList) filewalkfunc() func(p string, info os.FileInfo, err error) error {
	return func(p string, info os.FileInfo, err error) error {
		if err != nil {
			return err
		}
		if info.IsDir() {
			return nil
		}
		return s.AddFile(p)
	}
}

If filewalkfunc looks weird to you, don’t worry, it is a little weird. Because filepath.Walk expects a function, we need some way to pass our list object along with that function. We could create a global variable and reference it from our function, but some smart Go folks believe this is harmful. Besides, writing tests with globals (Go calls them Package Scoped Variables, because that’s precisely what they are) is pretty painful. Anyway, filewalkfunc returns a function that closes around a pointer to our SubjectList. That’s the call to s.AddFile. The outer function definition does look pretty weird, but it makes sense when you remember Go funcs are a first-class type.

Now that we have gathered up all of our Subjects, it’s time to actually do something with them. Our strategy for this first (naive!) version will be to iterate through our list and compare each image with every other image. We’ll first define the nice abstract way we want this to happen:

func (s *SubjectList) CompareAll() []Comparison {
	// we know that a list of size N will have ((N-1)*N)/2 comparisons. this
	// relationship is the "Triangular numbers"! https://oeis.org/A000217
	r := make([]Comparison, 0, ((len(s.list)-1)*len(s.list))/2)
	for len(s.list) > 0 {
		// take the last subject
		subjecta := s.list[len(s.list)-1]
		// modify the slice to exclude it
		s.list = s.list[:len(s.list)-1]
		// now, compare all the remaining subjects
		for _, subjectb := range s.list {
			r = append(r, subjecta.CompareTo(subjectb))
		}
	}
	return r
}

In the next article, we will implement Subject.CompareTo(). We will need to address two core problems: how we compare two pictures that might not be the same size, and how we quantify the difference between pixels in two images.

See the complete code for this article on GitHub.

Detecting duplicate images - Part 1: Concept

2020-06-25T00:00:00+00:00

Say you have thousands of image files, and you know at least some of them are highly simliar. It would be handy to be able to find the images that are most similar, and consolidate. How could we approach this problem?

What we have

For the sake of our discussion, let’s say we have 10,000+ JPEG files, all around 16:9 aspect ratio, but of a variety of different sizes. Let’s say that these are large enough that we don’t want to copy them to our local computer running some desktop photo organizing software, or that we have so many files that such software might be impractical to use. Maybe our files are out in Amazon S3 or need to be read in over HTTP.

What we need

Let’s say we want to deal with these duplicates or similar images with some business logic, rather than simply deleting them. Ideally, our program should:

Provide machine-readable output,
Be able to detect images that are similar but not identical, such as images that were scaled, slightly cropped, watermarked, or edited,
Provide some kind of “score” through which we can rank similarity, and
Be as fast, efficient, and accurate as possible.

Approach

In this series, we will use Go. It has exhaustive support for S3, natural support for multi-core processing, and is a joy to use. In part 2, we start with a naive comparison approach that fails many of our requirements above. In the following parts, we will refine until we have a reusable tool.

Welcome to my portfolio

2020-06-24T00:00:00+00:00

As we turn the corner into the latter half of 2020, everything seems to be in upheaval. It’s hard to imagine anyone not aware of the ongoing COVID-19 pandemic, or of the reignited racial conflict. Businesses are closing; many permanently. Things are so uncertain, that the phrase “uncertain times” has now been criticised as insincere.

So, what better time, uncertain as it is, to begin anew? After 15 years in an industry that centered entirely around how technology can optimize the number of people packed into a conference room, it seems as though I wouldn’t be alone in taking an introspective pause. To paraphrase Ian Malcom, perhaps we should have stopped to consider whether this was the right strategy.

In my industry, we didn’t weigh the consequences of commonly shared touch interfaces and cables. Of tightly packed row seating or intimate “collaboration pods”. In a future where conference room technology may become obsolete, or indeed maybe even dangerous, perhaps now is the moment to do something new. And whether or not you believe “this will all blow over”, it is clear that corporations have now been made immediately aware that offices, conference rooms, and the technology and furniture that went with them, may not be the value proposition they once appeared to be.

Where I find myself at this moment is looking back at my coworkers, friends, and teammates, all of whom were critical to my long successful time in corporate AV. Every one of them will be missed dearly. It was a privilege to work alongside these dedicated, intelligent, thoughtful folks. I’m hopeful this new chapter gives me the opportunity to join a new team where I can learn and contribute.

That leaves me here. I’m beginning to collect some examples of my work and I will be archiving them in this portfolio. And I’m looking forward to showcasing some of my projects and talents.

If you’re looking for a Minneapolis-based developer – or one who is comfortable working remote – I am available now for full time and contract work.