This Army of AI Robots Will Feed the World

 We actually are seeing early success as a machine now exists to thin lettuce rows.  Yet the difficulties are daunting as well. It is all coming though.  

The hand drudgery of farming must be eliminated and by adaptive robots that trade tools for the task at hand.  These so far are tractor based which will likely remain as the standard for most agriculture.

However i do want to also see an agriculture drone platform able to work around trees with an extendable arm.  This needs to be able to do pruning and also harvest efficiently prime fruit.  All this allows a wider range of particularly larger trees..

This Army of AI Robots Will Feed the World

And it could do it while eliminating herbicides, replenishing topsoil, and reducing carbon consumption. If all goes to plan.

At Blue River Technology headquarters, a prototype learns to distinguish weeds from crops. 

Photographer: Justin Kaneps for Bloomberg Businessweek
By Amanda Little 

January 11, 2018, 2:00 AM PST 

Jorge Heraud is in a California lettuce field, and he’s about to lose his mind. It’s a balmy, cloudless day in October 2014. Salinas Valley stretches out around him like a Hidden Valley Ranch commercial, its endless rows of emerald leaves pushing up through the black soil. Heraud has come here to test Potato, a robot you might call the agricultural equivalent of an Apple 1 prototype circa 1976. The machine is trying to thin baby lettuce plants so the hardiest ones have space to mature. If you’re imagining a C-3PO-style bipedal contraption with pincerlike hands that do the yanking, Potato isn’t that. It looks like a huge metal Pez dispenser laid sideways on a rack hitched to the back of a tractor. The robot “sees” the seedlings via cameras mounted on the rack. In milliseconds it identifies the strongest plants and zaps the weaklings with jets of fertilizer so concentrated it’s deadly.
Or that’s what Heraud’s machine is supposed to do, but right now it’s on the fritz. Robots like controlled environments, and Potato’s delicate equipment isn’t responding well to the heat, dust, and the vibrations from the tractor. Electrical components are short-circuiting, nozzles are failing, and dirt is gumming up the cooling fans. About every half-hour, all day long, the monitors on the PCs that guide Potato freeze into blue screens of death.

Heraud’s agony deepens as the failures mount. For months his team has been testing beta versions of Potato, each with a salad-themed moniker—Caesar, Cobb, Chicken, Wedge, Jello. All are first-generation renderings of a product they’ve officially named LettuceBot, which Heraud has already begun leasing to farmers—prematurely, apparently.

In two days he has to face his investors at a board meeting. They’ve plowed $13 million into his startup, and they want to hear he has a reliable product. A Peruvian-born engineer with deep-set, blue-green eyes and a stolid demeanor, Heraud internalizes stress. Lately his skin has been breaking out into prickly rashes; he’s had insomnia and heartburn. LettuceBot isn’t even what he’d originally pitched to his investors. He’d envisioned a robotic weeder that could perform far more complex tasks and radically reduce herbicide use. Such a machine would disrupt a $28 billion industry dominated by agrochemical companies including Syngenta, Bayer, BASF, DowDuPont, and Monsanto, while also salvaging soil microbiome, saving countless aquatic and amphibious species, and restoring purity to the world’s waterways. He’d named the company Blue River Technology with these goals in mind.

When Heraud confesses his field-test failures to the board, they don’t vote to oust him as he’d feared, but instead encourage him to turn things around. Over the next year, he and his team of 20 engineers launch a 24/7 troubleshooting offensive they call “the surge.” They take turns sleeping on cots in the closet of their Silicon Valley office. They call in husbands and wives to turn wrenches and clamp tubes. They redesign fans, build mounts, change materials, and reformulate chemicals. Heraud consumes Tums by the fistful. By late 2015 they have a glitch-free LettuceBot that can handle the elements. They expand their contracts with farmers in Salinas and Yuma, Ariz. By early 2017, about a fifth of all the lettuce grown in the U.S. has been thinned by a LettuceBot.

Heraud and his investors are buoyed by the success, but other news excites them more. Microchip maker Nvidia Corp. has released a computing platform with outsize processing power. It’s designed for navigation in self-driving cars, but it also means that a farming robot can crunch a lot more data captured by mobile cameras. Heraud may be able to build that weeding robot after all. But what he can’t possibly imagine as his team begins to cobble together their first dream machine is that in September 2017 the green-and-yellow-tractor company, Deere & Co., will acquire Blue River for $305 million. And the oldest brand in agriculture will be on board with Heraud’s most idealistic, even extravagant goal of all—not just of slashing agrochemical applications worldwide, but of fundamentally transforming the way we grow food.

Blue River’s Sunnyvale, Calif., headquarters is located in a glass-and-steel building down the street from Juniper Networks, Lockheed Martin’s space division, and Yahoo! “Welcome to agriculture 2.0,” Heraud deadpans, gesturing at the generic cubicles and gray carpet tiles that make up his office space. Of the company’s 72 employees, only a handful, including Heraud and his co-founder, Lee Redden, have any dirt-under-the-fingernails growing experience; the rest are mostly software and mechanical engineers with degrees from Harvard, Stanford, Oxford, and Caltech. One of few clues that this is a farming company is the “I [HEART] SOIL” sticker on Heraud’s ThinkPad—along with a reminder of the enemy, a framed photograph of a yellow Cessna crop-duster spewing herbicides on an oceanic cornfield.

Heraud, 47, grew up in Lima, the math-loving only child of an electrical engineer and an elementary school teacher. By the age of 5 he was spending his free time adding columns of numbers in the phone book. Weekends and afternoons he tagged along with his dad at his company, Digita, which specializes in factory automation. In the summers, he stayed on his grandparents’ farm north of Lima, where his family grew 200 acres of tomatoes and rice.

He loved the fun parts of farm life—driving tractors and ATVs, raiding the soursop orchards, gathering eggs from the coop, and eating his grandmother’s cakes and pies. But the drudgery seemed inane. He was up by 5:30 a.m. and in the fields with his cousins by 6 a.m., pulling weeds. “I understood early on that a farm, even a small one, is basically a big outdoor factory,” he says. 

“There’d be dozens of us kids in the fields bending and picking, bending and picking weeds. I must have been 7 when I first thought, This is a job for machines.”

Heraud excelled at school and by 14 was designing software for his dad. He went to Pontificia Universidad Católica de Perú, a South American hub for mathematicians, and continued working on the side, heading a project to automate a chicken feed factory. Stanford soon snatched him up with a scholarship to its electrical engineering master’s program. After graduating, he took an engineering job at Trimble Inc., an early GPS-based technology company. Heraud led the team of engineers that designed the first self-driving tractor, a technology now used to produce about 80 percent of the developed world’s food.

In 2008 he became Trimble’s director of business development, buying companies that made things such as digital sensors that measure soil moisture, until he realized he wanted his own enterprise. He returned to Stanford to get an executive MBA. While there, he posted “Let’s solve agriculture’s biggest problems,” on the university intranet. Redden, a 24-year-old Nebraskan Ph.D. student in robotics, replied. Redden also grew up working summers on a farm, in his uncle’s 6,000 acres of cornfields, and was a whiz kid like Heraud. By 15 he was working as a car mechanic and had a side business building and fixing motorcycles, four-wheelers, and go-karts. At Stanford, he’d built dozens of robots that could perform everything from pingpong training to infant CPR. “But they all just sat on a laboratory shelf collecting dust,” he says. “I wanted to do something that could live in the world.”



Blue River Technology CEO Jorge Heraud. 

Photographer: Justin Kaneps for Bloomberg Businessweek Heraud researched the scourges of agriculture: hypoxic dead zones in the Gulf of Mexico and Baltic Sea, the colony collapse of bees, soil degradation, and human health problems from allergies to cancers. “Everything tied back to the blind, rampant, broadcast spraying of chemicals,” Heraud says. He and Redden figured they could teach machines to differentiate between crops and weeds, then eliminate the weeds mechanically or with targeted doses of nontoxic substances. The two first considered hot foam, laser beams, electric currents, and boiling water. They’d market the robot to organic farmers, who spend heavily on chemical-free weeding methods including mechanical tillage, which can be both fuel-intensive and damaging to soil.

After months of research, they faced a disappointing truth: There was no way around herbicides. “Turns out zapping weeds with electricity or hot liquid requires far more time and energy than chemicals—and it isn’t guaranteed to work,” Heraud says. Those methods might eliminate the visible part of a weed, but not the root. And pulling weeds with mechanical pincers is a far more time-intensive task for a robot than delivering microsquirts of poison. Their challenge became applying the chemicals with precision.

In the early days of the startup, Heraud had pitched his company to the investment divisions of Monsanto and Syngenta, the giants of the industry he was planning to eradicate—or at least thin—with his weeding robot. He wanted access to their chemists and botanists and the credibility the association would give him among mainstream farmers who could field-test his prototypes.

The response was tentative. “We loved Jorge’s Trimble background—smart guy—but at first there was some starry-eyed idealism,” says Syngenta Ventures’ investment director, Gabriel Wilmoth, who passed on the first investment round but kept tracking the evolution of the LettuceBot. By the third round, he bought in. Kiersten Stead, an investment director at Monsanto Growth Ventures, also provided some funding. The backing was nominal by Big Ag standards—a few million dollars—and partly a way to keep tabs on the young competition. It was also, perhaps, an admission of defeat.

Weeds are elegant masters of adaptation and procreative success, and the Genghis Khan of weeds—the one most hellbent on total domination—is pigweed, aka Palmer amaranth. It can grow as high as 10 feet in the shape of a ponderosa pine, with a stalk the width of a corncob. A single plant can produce a million seeds, and a pigweed-infested field will spew hundreds of millions, raising the probability that a mutation of the plant will come along that can resist an herbicide. “To a farmer, pigweed’s like a staph infection resistant to every antibiotic,” Heraud says.

For decades chemists at Monsanto, Syngenta, and other agrochemical companies struggled to make products that are molecularly “selective,” meaning lethal to weeds but not to crops. Failing that, they genetically engineered crops, beginning with Roundup Ready cotton, corn, and soy, to tolerate the herbicide glyphosate, so it could be sprayed over entire fields. The solution worked until it led to the overuse of glyphosate and, in turn, superweeds. In 2006 an Arkansas cotton farmer noticed that the Roundup he was spraying on his fields wasn’t killing the pigweed the way it used to. Two years later there were 10 million acres of Roundup-resistant weeds in the U.S., and by 2012, 30 million acres. Today there are 70 million acres, an area of land roughly the size of Nevada.

Chemical companies have responded by reformulating old, high-potency chemicals such as Dicamba and 2,4-D, but this approach has brought its own host of problems. Dicamba has caused chemical drift, damaging 3 million acres of neighboring crops in the past 18 months. Pigweed, meanwhile, has kept on dropping its tiny genetic bombs by the trillions throughout America’s farmlands.

If robots can prevent herbicides from having any contact with crops, it means that 18 classes of chemicals previously considered too damaging to be widely sprayed suddenly become viable. “We’re both ratcheting down the volume of chemicals that need to be used, but also expanding how many types can be used,” Heraud says. In other words, Blue River’s success might be the worst thing that could happen to the herbicide industry, or it could open up an avenue to sell new products.

“It’s not either/or—should we do technology or agro-ecology, sustainable farming or industrial farming. It’s both/and” 

It’s a steamy, early summer day in Marianna, Ark., and we’re deep in the heart of cotton country. Marianna looks like many small towns in the Arkansas Delta—population 4,000, median income $24,000—a farming community suffering from low crop prices. Many of the houses at the city center, once-beautiful, gingerbread-trimmed Victorians, are abandoned, their porches sunken and windows broken, kudzu crawling in, evidence of the one resource this town remains rich in: weeds.

Heraud has come to the weed capital of the world to test See & Spray, his first robot weeder, in cotton fields owned by 37-year-old Nathan Reed, a third-generation Marianna farmer who cultivates cotton, corn, rice, and soybeans on 6,500 acres. A dome of fabric that looks like a huge white hoop skirt protrudes off the back of Blue River’s tractor to protect the robot from dust and rain. Eight computers are stacked sideways beneath the domed cover, and above the shrouded robot three large tanks are filled with water dyed electric blue—a faux herbicide for the test run.

A software engineer is in the cab of the tractor looking at a laptop that displays a composite view of the ground beneath the robot gathered by 16 cameras. It shows cracked brown soil with cotton seedlings poking up about 3 inches and a random assortment of weeds that, to the untrained eye, are indistinguishable from the cotton plants. The robot does the differentiating for us. See & Spray is scanning the plants, Heraud explains, and within 30 milliseconds—about a tenth of the time it takes for you to blink your eyes—it’s distinguished the cotton from the weeds and decided how much and where to spray. The screen shows circles around the cotton plants and squares—dozens of them overlapping—around the weeds.



A Blue River prototype sprays water on the red plants. 

Photographer: Justin Kaneps for Bloomberg Businessweek Psst pssst pst psssssst pst—tiny bursts of blue “herbicide” are shot from 128 nozzles across eight rows of cotton plants. Patches of blue ink land on clumps of weeds in perfect rectangles, some the size of a sheet of paper, others a thumbnail. “There’s a misfire—you woulda murdered my cotton plant,” Reed jokes, pointing to a seedling shot with blue.

“That’s why we don’t use the red dye,” Heraud says. “It’d look too gory.”

In its early days in Yuma, the LettuceBot did indeed murder entire fields of lettuce. Its nozzles sprang leaks and dripped hyperconcentrated fertilizer on acre after acre of seedlings. Heraud got on a plane and went to make things right with the affected farmers. His team fixed the problem by adding an automatic abort function to nozzles that drip for more than five seconds, then they thinned the farmers’ next 100 acres for free.

On Reed’s field we notice a lot of blue-spattered cotton plants, while the weeds next to them are untouched. The machine is getting confused because some of the cotton is runty and withered—not as healthy as the cotton See & Spray is programmed to recognize. The robot needs to be fed first hundreds, then thousands, and eventually millions of images of cotton to learn the many variations of the plant, how its leaves change shape and texture over time, how they look when they’re sickly and healthy, and during all stages of growth. The robot’s ability to draw from this image archive and make distinctions and decisions is “deep learning.”

The Blue River team built the memory of See & Spray by going to a cotton farm in Australia, hitching a video camera to a modified shopping cart, and spending three months pushing it around different fields, uploading about 100,000 images of cotton. But the Arkansas cotton, struggling in a wet, cold spring, isn’t looking enough like the Australian cotton for 100 percent accuracy. Each day for a fortnight, Heraud’s team will take tens of thousands of new cotton images, and each day the robot will become more accurate.

For now, though, See & Spray is making toddleresque mistakes. Suddenly, Heraud slaps his thigh. “Nailed it!” he shouts, breaking his characteristic composure. He’s looking at a cotton plant surrounded by a nasty weed. The machine has outlined the weed in blue liquid and spared the struggling seedling at the center.

Where Heraud sees lowered chemical use, Reed sees savings. Because of his location, herbicides account for about 40 percent of his operating costs—more than $500,000 a year. On an acre of cotton, he typically uses about 20 gallons of herbicide. After several weeks of trials, it appears the See & Spray robot can manage his weeds with 2 gallons per acre. A robotic weeder is also a huge advantage for no-till agriculture, a practice Reed has adopted in recent years. Tilling is a way to manage weeds without chemicals, but it also erodes and dries out soil, disturbs the microbiome, kills earthworms, and releases trapped carbon. Avoiding tillage eliminates those fuel costs and reduces irrigation needs. See & Spray also liberates Reed from an economic hamster wheel; to use broadly applied sprays like Roundup, each season farmers must buy expensive seeds genetically modified to resist the chemicals. A robot that targets only the weed will allow Reed to buy nonengineered seeds, which cost roughly 75 percent less. But Reed, like many farmers, is struggling to get by. A robotic weeder will be an option only if Heraud can lease his robots at a competitive price.

As See & Spray was busy crunching data in Reed’s fields, Heraud was hatching a plan to produce an affordable machine. “We’d been seriously courting Jorge for months and following his progress for years,” says John Teeple, Deere’s director for advanced technology. In September the farm equipment multinational bought the three-year-old Silicon Valley startup. “It was clear that Blue River was becoming the industry leader in robotics and machine learning and that this for us would be a perfect synergy,” he says.

Heraud had no ambivalence about giving up his independence. “Last week we were a tiny company that might succeed or fail,” he says after the acquisition. “A young startup with six lettuce thinners and two weeding prototypes. Any more scary scenarios”—like the leaky nozzles on the LettuceBots—“could be a fatal blow.” He estimates that Blue River will release its first See & Spray bots in the U.S. in early 2020 and in Europe in 2021—several years sooner and on a much larger scale than it could have without Deere’s army of mechanical engineers, forge factories, and 10,000 dealers around the world.

His next step, with Deere’s backing, will be to move Blue River’s robots beyond herbicides to fertilizers, the culprits behind toxic algae blooms, which are killing fish and making lakes unswimmable. Farmers typically spend up to 10 times more annually on fertilizers than weed killers—about $150 billion a year. But the shift is a big leap for a robot. It must gather a range of visual signals—the colors, sizes, and textures of a plant’s leaves—and from this data extrapolate the plant’s health and how much nourishment it needs. “It’s a ton more processing power, but it’s doable,” Heraud says.

The next link in this technological chain could be a kind of agricultural Swiss Army knife: a robot that can apply not only herbicides and fertilizers but also insecticides, fungicides, and water all at once, delivering only as needed.

The implication of plant-by-plant—rather than field-by-field—farming is not just the prospect of vast reductions in chemical usage. It could also, in theory, end monocropping, which has become the new normal—cornfields and soybean fields as far as the eye can see—and has given rise to the kind of high-calorie, low-nutrient diets that are causing heart disease, obesity, and Type 2 diabetes. Monocrops also leach soil nutrients and put food supplies at risk, because single-crop fields are more susceptible to blight and catastrophe. Modern farmers have been segregating crops in part because our equipment can’t handle more complexity. Robots that can tend plants individually could support intercropping—planting corn in with complementary crops such as soybeans and other legumes.



Miniature Blue River weedbot and John Deere tractor. 

Photographer: Justin Kaneps for Bloomberg Businessweek 

Danielle Nierenberg, president of Food Tank, a think tank that advocates for sustainable agriculture, isn’t particularly comforted by this vision of AI farming. “Lots of questions need to be asked, like which chemicals will be deployed by these bots?” she says. “And which of the many problems inherent with industrial agriculture will persist even as we cut back on herbicides?”

One of those problems is the potentially coercive power of monopolies. Deere is cast as a villain in the right-to-repair movement, in which urban and rural DIYers are fighting for laws limiting the use of proprietary software and hardware that make it almost impossible for individuals to fix their own gadgets or machines—a problem that could economically devastate an owner of a $200,000 AI-enabled tractor. Deere’s ability to make farmers dependent on the usage and, increasingly, the maintenance of its specialized equipment bears relation to Monsanto’s system of locking farmers in to its herbicides and seeds. If Blue River’s sophisticated robots succeed as the company hopes, it could mean that farmers, and our food supply, are even more dependent on a handful of corporations. There’s also the possibility, however remote, that a software-dependent food system may become vulnerable to hackers who could manipulate the dosages of toxic chemicals on the fields.

Heraud prefers not to fixate on worst-case, sci-fi scenarios. “It’s not either/or—should we do technology or agro-ecology, sustainable farming or industrial farming,” he says. “It’s both/and. We need all solutions.” He brings me back to the connection he made as a kid between farms and factories. “A hundred years ago, factories were a nightmare, spewing black smoke, with terrible working conditions and people dying. A lot of agribusiness is in that state right now, with massive inefficiencies, harmful chemicals, a huge carbon impact. But compare that with modern factories, designed to be smart, automated, safe for the environment and humans, ergonomics on every single job. They’ve turned around.” The happy paradox, he insists, is that “robots don’t have to take us away from nature—they can help us restore it.”

(Corrects the acres of crops destroyed by chemical drifts over the past 18 months in the 18th paragraph.

)