Sunday, 19 February 2012

I'm scared of AI, because prions

I'm not a biologist, or a biochemist, but I find prions cool. Here's what they are (according to Wikipedia):
A prion is an infectious agent composed of protein in a misfolded form. This is in contrast to all other known infectious agents (virus/bacteria/fungus/parasite) which must contain nucleic acids (either DNA, RNA, or both). The word prion, coined in 1982 by Stanley B. Prusiner, is a portmanteau derived from the words protein and infection. Prions are responsible for the transmissible spongiform encephalopathies in a variety of mammals, including bovine spongiform encephalopathy (BSE, also known as "mad cow disease") in cattle and Creutzfeldt–Jakob disease (CJD) in humans. All known prion diseases affect the structure of the brain or other neural tissue and all are currently untreatable and universally fatal.

So in essence its a knot of protein that will eat your brain and leave you dead. Also, they don't really fit the definition of something living. Ideal. Particularly when you find out that these vicious brain eating incurable monsters can evolve:
Scientists have shown for the first time that "lifeless" prion proteins, devoid of all genetic material, can evolve just like higher forms of life.

The Scripps Research Institute in the US says the prions can change to suit their environment and go on to develop drug resistance.

So, that might be an issue. Then I read this:
“There’s this whole world below 650 milliseconds. It’s like landing on another planet,” said Neil Johnson, a complex systems specialist at the University of Miami and co-author of the study, released Feb. 7 on arXiv. “It’s an enormous part of the market which is out of human reach. We have a glimpse of the kind of ecology that’s going on down there.”
Neil Johnson is talking about high speed automated trading. Probably the most bizarre part of the financial sector you could possibly imagine. Its an entire world of 'bots' working completely outside human control, operating on a millisecond level to turn less money into more money. They compete of course, and they do so in an environment which is basically not understood.

Until recently, trading was the preserve of humans. Imagine a stock market and you likely envision a loud, crowded trading floor, a scene out of Wall Street. But in 1998, after the U.S. Securities and Exchange Commission authorized the first electronic exchanges, computer trading programs entered markets as equals to humans.

The programs are designed to trade enormous volumes of stocks, bonds and other financial instruments at superfast speeds, taking advantage of second-to-second fractional price shifts and market trends. It’s now estimated that high-frequency computer trading accounts for 70 percent of all equity trades. While some activity does occur at speeds with which humans can interact, much of it falls beyond the limits of human response time.

(One new computer chip built specifically for high-frequency trading can prepare trades in .000000074 seconds; a proposed $300 million transatlantic cable is being built just to shave 0.006 seconds off transaction times between New York City and London.)

In the early years of computer trading, algorithms were profitable and concerns rare. Designers and investors took their money and didn’t think much about what Johnson and co-authors call “ultrafast machine ecology.” After the 2010 flash crash, however, mainstream economists wondered if high-frequency trading systems might sometimes get weird and unpredictable. A $4.1 billion automated sale was ultimately blamed for triggering that crash, and economists started asking questions about the new, hazy relationships between machines and markets.

“We are certainly witnessing one of the major transitions in the history of financial markets,” said automated trading researcher John Cartlidge of the University of Bristol, who was not involved in new study. “Economic theory has always lagged behind economic reality, but now the speed of technological change is widening that gap at an exponential rate. The scary result of this is that we now live in a world dominated by a global financial market of which we have virtually no sound theoretical understanding.”

If that last sentence doesnt give you pause, go back and look at it again.

So what's the connection between the prions and the AI eating the stock market? Well, both are unliving systems, which shouldn't be able to evolve, but due to a mixture of environment and human intervention are going to. The AI systems will be improved by humans to better compete in their environment, and will behave in increasingly complex ways as their competition is also improved.

I'm not saying that Skynet will be born in the stock market. If we develop human type AI, that'll come from an entirely different field of research. What we might create is wholly autonomous pieces of software whose existence is predicated on activity which may or may not be to human advantage, and which is susceptible to emergent properties.
In philosophy, systems theory, science, and art, emergence is the way complex systems and patterns arise out of a multiplicity of relatively simple interactions. Emergence is central to the theories of integrative levels and of complex systems.
Of course you might say "well, these things arent intelligent". Problem is, ants arent intelligent, nor are bees. They're just complicated little knots of organic computer code, designed to co-operate and compete, strapped to a piece of flesh and let loose. These AI's are strapped money and let loose in an environment where their only object is to make their pile of money bigger. That is their survival imperative.

So whats the bottom line? Well, nothing too sinister. We've just created an entirely new ecosystem, with evolving creatures in it, put it in a sealed container we can't easily examine, and then poured a great deal of the world's money into it. How that can go wrong I don't know.

1 comment:

  1. There's a Neal Stephenson or Daniel Suarez book in there somewhere, unless I get off my lazy arse and beat them to it :)