Google open-sourced a compiler that runs AI on data it cannot read

Google published HEIR, an open-source compiler toolchain that converts pre-trained AI models to run on fully homomorphically encrypted inputs. The server does the inference and returns a result without ever decrypting the data. The announcement drew a 498-point Hacker News thread in mid-August, mostly asking the right question: how slow?

What HEIR actually automates

Fully homomorphic encryption has let you compute on ciphertext in theory since 2009. In practice, converting a model into an FHE circuit was hand work for cryptography specialists, one model at a time. HEIR makes that a compiler pass: take a trained model that expects plaintext, compile it to operate on encrypted inputs. Google names recommendations and fraud detection as the target use cases, which is a tell about scale: smaller models, structured inputs, high data sensitivity.

Google is direct about the cost: FHE remains computationally expensive, and it takes recent algorithm and hardware acceleration progress to make even these applications practical. Chat-scale LLM inference on ciphertext is not what is on offer today.

Why a build studio cares

Clients in health and finance ask us for exactly this class of guarantee, and until now the honest answer involved a cryptography specialist and a long timeline. The pitch to a compliance department changes shape: from trust our access controls to the server cannot see the data, cryptographically. The tradeoff stays plain: an FHE inference costs some multiple of the plaintext one, and that multiple decides whether your use case is on the practical side of the line.

Next step: run the HEIR examples on a toy model and measure the latency multiplier for an input shaped like yours. Know that number before anyone promises encrypted inference to a client. If a compliance conversation is pushing your product toward private inference, write to us at hello@gattyworks.com.