First thing to know: AI chip smuggling looks like a growing problem. In early 2023 news reports indicated some small-scale cases (tens of chips at a time) More recent reports show much larger-scale operations, able to smuggle thousands of chips at a time.

And the smugglers aren’t exactly hiding it. From a quick search on Baidu marketplaces for local listings, I found over dozens of listings for export-controlled chips. Many of the listings show big stacks of H100 servers and shipping boxes.

Why is smuggling so easy? One key reason: we have no good way of telling what happens to a chip once it’s shipped overseas. So enforcement relies mostly on checking that the company you’re exporting to isn’t on a blacklist (the “Entity List”)

But the Entity List is easily circumvented — shell companies can be set up in hours, while it takes months to years to investigate them and add them to the list.

One solution is manual inspections, but this is expensive and easily fooled. In one reported case, a smuggler rented space in a Malaysian data center and installed the servers, then packed the order up and shipped it over to China once the inspectors left.

A better idea is to use technology installed on GPUs to automatically check whether they’ve been smuggled into a restricted country. 1. Collect location data, check for red flags (e.g. a bunch of chips have gone dark) 2. Use this to target investigations into upstream firms

But how do you actually implement tracking on today’s data center AI chips? First option is GPS. But this would require adding a GPS receiver to the GPU, and commercial signals could be spoofed for as little as $200.

Second option is what your cell phone does when it doesn’t have a GPS signal. Listen to radio signals from cell towers, and then map your location onto the known location of the towers.

But this requires adding an antenna to the GPU, and can easily be spoofed using cheap hardware (Raspberry Pi + wifi card) https://www.journaldulapin.com...

A better approach is “constraint-based geolocation.” Trusted servers (“landmarks”) send pings over the internet to the GPU, and use the round-trip time to calculate itslocation. The more landmarks you have / the closer the landmarks are to the GPU, the better your accuracy.

This technique is: - simple - widely used - possible to implement with a software update on any GPU that has a cryptographic module on board that enables key signing (so it can prove it's the GPU you’re trying to ping) – this is basically every NVIDIA data center GPU.

And NVIDIA has already suggested doing what sounds like exactly this. So feels like a no-brainer. https://x.com/taoburr/status/1...

The downside: internet geolocation only gets you to a coarse level of accuracy – to within tens of miles, rather than the exact location. See here for an overview of the accuracy of existing commercial services that use this approach: https://ar5iv.labs.arxiv.org/h...

For export controls, coarse accuracy may be a feature rather than a bug. Highly accurate location tracking might be concerning to some companies that might legitimately want to hide the location of their data centers.

Of course, a smuggler can prevent internet-based geolocation by simply disconnecting the chip from the internet. But if a bunch of chips “go dark”, this is evidence that warrants further investigation. The goal is prioritizing limited export control enforcement resources.

For big shipments with a known end-user where you want to know that the chips haven’t been moved from a specific location, or for data centers that need the GPUs to be always airgapped, you could also ship a hardened landmark server with the chips, and have it installed on-site.

This would require regular inspections to ensure the landmark hasn’t been tampered with.

The Cotton bill takes the right approach on these technical implementation questions: - Don’t tell how industry how to implement it - Instead prescribe the goal (catching smuggling) - Then let industry figure out the best implementation

Key additional questions I want to see more details on: - How is the information handled? Who is actually responsible for collecting and reporting the geolocation data? - How does this work with multiple exports/re-exports/in-country transfers of chips?

Last point: this bill could also create a path to stronger, simpler, and more industry-friendly export controls. The whole reason we have blanket bans in the first place is we have no good way of knowing what happens to a chip once it's sent overseas.

With features like geolocation, hopefully we can move towards a regime with more surgical controls and less blanket bans, opening up more market access for chip companies.

In summary: - the current approach to tackling smuggling is failing, and the govt has limited enforcement capacity - automated chip tracking is a potentially elegant solution: it’s implementable today, highly scalable, and doesn’t require the government to spend any money

@fiiiiiist How confident are you on the timeline for implementation? The bill only gives a 6 month window.

@TheZvi On the software development side 6 months feels quite reasonable - you can build a simple version today using the existing NVIDIA stack very quickly. I'd expect the hardest thing to get right within this time frame will be processes for sharing country-level location data with

@fiiiiiist Great thread!

@fiiiiiist It sounds challenging, but maybe one day someone will make an implementation similar to Denuvo to Nvidia chips. If software piracy can be controlled, so can chip smuggling.

@fiiiiiist @threadreaderapp unroll