Name: Can an artificial intelligence learn to beat the stock market?
Brand: fastcompany.com
SKU: fastcompany-com-30
Availability: OnlineOnly

critic

Can an artificial intelligence learn to beat the stock market?

preview

advertisementadvertisementBy William D. Cohan On the far side of an office park in a suburb of Seattle, a supercomputer is teaching itself to beat the stock market.advertisementadvertisementThe holy grail of high finance doesn’t look like much: eight rows of servers enshrined in a black metal frame. But inside this austere enclosure, an incredible alchemy is taking place. Four hundred computers blink and hum as market data is digested at a rate of one quadrillion calculations per second, firing order requests to electronic traders in Chicago, 2,000 miles away. Outside the containment, a bank of 10 glowing monitors displays the results as money rolls back in.Even now, as the world economy slumps into a recession, Jeff Glickman and his boutique investment firm, J4 Capital, are quietly taking gains. “Suffice it to say we’re making a profit in this market,” he says.This somewhat understates the miracle that Glickman claims to have performed. When we spoke on March 20, J4 Capital was up nearly 4% for the year, according to internal documents Glickman shared, while the Dow Jones Industrial Average was down nearly 27%—a heroic beat of nearly 31 percentage points. Many other hedge funds were down by double digits and teetering. When we spoke again on May 7, he was approaching a 5% return.advertisementadvertisementMost financial engineers believe that it’s impossible for a machine, left to its own devices, to beat the stock market. The data is too noisy, too random to be predictable. Observable trading records are limited to the past hundred years, and the law of averages is relentless. Any signal that is obvious enough to exploit absent inside information—barrels of oil priced nearly free, for example—will quickly be discovered and eliminated by competitors. While some quantitative hedge funds use algorithms to make high-frequency trades, they must frequently be reprogrammed and refined.All this competition leaves a slim margin for profit. An exceptional trader would be thrilled with a 51% success rate—similar to the house edge at a Las Vegas blackjack table. , perhaps the most profitable quant firm in the world, has generated a vast fortune by leveraging bets with these odds. J4 Capital, which has only two other employees, claims to have a success rate of nearly 60%.Glickman himself knows little about finance. The 59-year-old computer scientist has never worked on Wall Street or for any big bank. For that reason, his supercomputer doesn’t leverage its bets or trade in derivatives, which limits, for now, the amount of money J4 can make. Nor did Glickman write an investment algorithm to tell the machine which inputs to use. Instead, Glickman says, he created an autodidactic “superintelligence” that could reprogram itself.advertisementThe hedge fund world is full of incredible claims, only some of which are true. When asked for comment, two academics with expertise in algorithmic finance expressed skepticism that J4’s innovations were as revolutionary as described, though neither expert was familiar with the firm. Glickman, who holds multiple patents in image processing, pattern recognition, and networking technology, is insistent that his artificial intelligence is the real deal.The mathematics are recondite, but Glickman does his best to explain. The software he runs is a type of theorem prover, a nondeterministic algorithm that can look at a data set and generate a hypothesis to interpret what it sees. Similar to how the human brain breaks information into in order to form heuristics about the world, Glickman’s AI tests theorems with increasing levels of mathematical abstraction. The result, he says, “turns out to be extraordinarily potent.”The trick, of course, is that market drivers are always changing. “Everybody’s sort of observed this informally—that sometimes the market couples itself to gold, and sometimes it couples itself to oil, or some other commodity,” says Glickman. “Sometimes the market is fearful of things that are happening in the world. Like today the market is concerned about Iran and Iraq. At other times, Kim Jong Un could fire a missile over Japan, and the market might sink 3%, and another day it pays no attention to it.”advertisementThe movement of markets can appear random. But at the end of the day, most investors are getting their information from the same sources—oil consumption and strike prices, coronavirus infections and Wall Street Journal headlines. Could it be that with enough processing power, it’s possible to discern signals in the noise?Glickman uses the word “random” advisedly, as if the chaos of the universe is just an illusion, concealing a fundamental, if inscrutable, higher order. “It’s kind of an intellectual cop-out,” he says, “It’s when something becomes so complex that the human mind is overwhelmed by the information content, and the human mind can’t possibly ever understand it.”But that doesn’t mean that some other intelligence won’t. “Despite the fact that you or I might perceive it as being random, there’s nothing random about it,” he continues. “There’s just an overwhelming amount of complexity that’s beyond comprehension for humans, but within the ability of a massive supercomputer to comprehend.”advertisement* * *Glickman got the computer bug early. One day, when he was around six, his mother asked him to change the batteries in a transistor radio. He wondered instead about the electronics inside. His mother was impressed by his curiosity, so she enlisted a family friend who was an engineer at Sperry Univac, an early computer company, to teach her son everything he knew. “By the time I was seven, I was building my own circuits,” Glickman says. “And by the time I was nine, I had built my own computer.”Like many prodigies, Glickman experienced something of an abbreviated childhood. An honorable mention for a Westinghouse science award caught the attention of a group of physics professors, including John Bardeen, a winner of two Nobel prizes, and Wolfgang Johannes Poppelbaum at the University of Illinois Urbana-Champaign; at 16, Glickman was offered his own research lab at the university, funded by the U.S. Navy. There was one condition: He would have to teach a graduate-student seminar in semiconductor physics. “I was petrified,” Glickman remembers, but he took the assignment in stride. “It didn’t strike me as absurd,” he continues. “Today, it does.”advertisementHe got his bachelor’s degree from the University of Illinois but never completed his graduate studies. He was more focused on turning his ideas into viable businesses. He started a company in the early 1980s, Thumb Scan, which secured some of the earliest patents for biometric and fingerprint processing. He started a consulting business, working for Ford and General Motors. But it was the military-industrial complex that provided Glickman the most edifying opportunities.The Department of Defense was interested to know, for instance, how an artificial intelligence would model the military capability of foreign powers; Glickman worked the problem. Later, the military wanted a program that could use radio signals to identify which missile silo in an encrypted network was the command-and-control center. Another time, he was asked to apply machine learning to improve aerial analysis of enemy infrastructure. The Pentagon didn’t want to waste 800-pound bombs on a concrete bridge when a 500-pound bomb would do.These were “esoteric and important problems for the military,” he says, and the work paid well. But it was an assignment to reverse engineer a new random number generator—to predict the next number from an apparently random sequence—that was a turning point in how he conceived of AI. His team was only halfway successful at the project, but it got Glickman thinking about the relationship between what mathematicians call Brownian motion—the random motion of particles suspended in a fluid—and the fluctuation of the stock market.advertisementFor years, he circled the problem. Around 2000, a friend in Seattle had given Glickman a copy of Edward Thorp’s bestselling 1967 book, Beat the Market, which claimed to show how option markets could be beaten, and wondered if what Thorp posited could still be done. A few weeks of study later, Glickman determined that Thorp’s loophole had been closed. But he was intrigued. He wondered if machine learning could be applied to the stock market—and he just as quickly hit a wall. “No matter how hard I tried,” he confesses, “it would not work on the stock markets.”It wasn’t until 2004 that Glickman realized he would need a new genus of software-based AI, a theorem prover that could reprogram itself to generate new models for financial data. Between 2005 and 2010, he worked on the project, getting closer and closer to a product that could “reliably predict” the direction of individual stocks. But he still had not decided definitively how to deploy his creation. He was itching to start a new business. Should it be in education? Could he revolutionize the field of medicine?In the end, he decided to manage money, since doing so would require no manufacturing, no large team or physical infrastructure. “That doesn’t necessarily mean it was going to be easiest to do, because it certainly hasn’t been,” he says. “It’s been extremely difficult.” He found a partner, Steve Jacobs, “just another one of those early young geniuses” who got his MBA at 19, and together they began working to build a scalable platform.advertisementFinally, on the first day of June 2015, Glickman flipped on his AI and let it run a full day of calculations. He had hoped he would be able to predict how the S&amp;P 500 would trade the next day, but wasn’t entirely sure how the AI would reach its conclusion. The trouble with most deep learning models and neural networks is what computer scientists call the : machine intelligence bears little resemblance to human cognition, involving millions or even billions of calculations that can make it practically impossible to understand how an AI arrives at any particular decision.We scratched our heads, and we went, ‘Oh, my God. We have no idea what it did.’”Glickman had decided early on that he needed to build a “white box.” Yet when he looked through the logs to understand what had happened, he was mystified. “We lifted the lid at the end of the day on June 1st, and we looked at the output, and we scratched our heads, and we went, ‘Oh, my God. We have no idea what it did.'”It took more than a year to figure out what the AI had done in those first eight hours. “It starts building out its theories,” Glickman explains. “It first had discovered that there was something called algebra, and algebra was useful in explaining the data. And so, it decided to keep it. And then it discovered geometry. And then it discovered trigonometry, and then it discovered calculus. And then it discovered differential equations. And then it discovered partial differential equations.”advertisementFinally, he tells me, the AI discovered the “upper reaches of mathematics”—the details of which he declines to share.Soon, Glickman let the AI start making test trades on its own—slowly at first, then faster and faster. With every additional trade the machine could make each day, the volatility of the outcome declined. The possibility of beating the stock market was no longer theoretical. His homemade supercomputer, it seemed, had cracked the code.* * *advertisementThe story of J4 Capital is another kind of black box problem. Even if it didn’t require an advanced degree in mathematics to understand what Glickman purports to do, the specifics of his technology are proprietary. It’s difficult to evaluate the extent to which his AI is truly automated—Glickman describes automation as a “spectrum”—or whether it is appropriate to call it a “superintelligence.”&nbsp;If he really is the next Jim Simons, it’s too soon to say.Or perhaps not. Glickman was quarantined at home when we last spoke on the phone. His only full-time employee had been in New Jersey. Yet his beautiful machines keep right on humming. Back at the end of January, amid the first coronavirus outbreak in Washington State, Glickman began testing what would happen if he left the supercomputer alone. “We ran for two weeks with nobody going into the office,” he says. Once he saw that it working just fine unsupervised, he closed down his operation.The system keeps making its bets, even in the dark. “We knew in great detail exactly how it was going to perform,” he continues. “It is precisely on point with the forecasted behavior.”advertisementThe trading infrastructure is largely automated too. The supercomputer now interacts on its own with J4’s broker-dealer; it can put on trades and take them off, shut itself down and reboot as necessary. Early on, Glickman hired a cloud engineer to build a proprietary, scalable platform to increase capacity alongside customer demand. “We can trade London,” he says. “We could trade Hong Kong. We could have 1,000 or even 10,000 customers. And if we needed to trade 15,000 equities, we could.”For the moment, J4 Capital is relatively small. Glickman has raised $10 million in capital from friends and family to build his business, including the 400 servers that form the supercomputer. He charges clients a 2% management fee and 20% of gains—standard practice for a hedge fund, though he has no desire to build the next BlackRock or Renaissance Technologies. Unlike traditional hedge funds, J4 doesn’t have a lock-up period where investors can’t withdraw their money.In a filing made with the State of Illinois, at the end of January, J4 reported that it had a mere $7.2 million under management at the end of 2019, after being in business for four months. But Glickman says he expects to have more than $100 million of assets under management soon, at which point he will be required to file with the Securities and Exchange Commission.advertisementWhile his supercomputers are printing money, Glickman remains at home, focused on the more prosaic aspects of running an investment business: finding new investors so that he can buy more supercomputing power, and trying to bring more assets under management. He’s continuing to refine the technology. He’s transitioning to Charles Schwab as his broker. He’s putting in a new billing system.Most important, he says, is that he’s not satisfied that his machines just trade equities. J4’s technology is close to being able to handle foreign exchange. He wants to be able to trade derivatives. He wants to be able to trade bonds and other credit products. He may soon move beyond trading in the financial markets to solving problems, using AI, in supply-chain logistics—an area where, he says, his clients are asking for his help.Glickman is largely indifferent to how his creation is used. “Our ambition is as simple as one could possibly imagine,” he says. “To build out a business and make money. We’re not a Wall Street company. We just happen to be playing in their space. We’re fundamentally a technology company. We’re a technology company. We’re going to build out a nice profitable business, and we’re going to profit from it, just like any other good tech company would do.”William D. Cohan, a former Wall Street senior mergers and acquisitions banker, has written four bestselling books about Wall Street.advertisementadvertisementadvertisementadvertisementadvertisement

advertisementadvertisementBy William D. Cohan On the far side of an office park in a suburb of Seattle, a supercomputer is teaching itself to beat the stock market.advertisementadvertisementThe holy grail of high finance doesn’t look like much: eight rows of servers enshrined in a black metal frame. But inside this austere enclosure, an incredible alchemy is taking place. Four hundred computers blink and hum as market data is digested at a rate of one quadrillion calculations per second, firing order requests to electronic traders in Chicago, 2,000 miles away. Outside the containment, a...

public

Benjamin Landy

Husband. Father. Best-selling author of six books. Seventh: November 2022. Founding partner, @Pucknews. Air Mail writer-at-large. Occasional NYT Op-Ed Writer.

William D. Cohan

Fast Company is a monthly American business magazine published in print and online that focuses on technology, business, and design. It publishes eight print issues per year. Fast Company was launched in November 1995 by Alan Webber and Bill Taylor, two former Harvard Business Review editors, and publisher Mortimer Zuckerman.

Can an artificial intelligence learn to beat the stock market?

No article scores yet.

CRITIC REVIEWS

PUBLIC REVIEWS