In building products I’ve always felt constraints are a way to drive innovation. Whether that’s time, resources, or a crisis. However, I didn’t expect that in the case of DeepSeek’s amazing new AI model it would be US export controls that would lead to innovations that would make it one of the top model in the world. As pointed out in Wired
DeepSeek had to come up with more efficient methods to train its models. “They optimized their model architecture using a battery of engineering tricks—custom communication schemes between chips, reducing the size of fields to save memory, and innovative use of the mix-of-models approach,” says Wendy Chang, a software engineer turned policy analyst at the Mercator Institute for China Studies. “Many of these approaches aren’t new ideas, but combining them successfully to produce a cutting-edge model is a remarkable feat.”