The emerging 737 Max Investigation

What we know so far is not much except that the software responded to a failing sensor by going nose down. So far, so good. Except that i think it is rubbish. Something way more subtle happened here.

The reason i am going there is that there are always a finite number of obvious failure modes and all engineering builds in fail safes.  This must be absolutely true for these sensors with an actual failure history.  It should be as simple as the pilot hitting an over ride and taking back control then and there.

That did not happen here and the evidence is that they were actively trying to do just that.  This event happened twice.  How did an over ride system get blocked?

I also do not think this is a scandal.  All the procedures are surely fine and certainly good enough to produce safe planes all the time.  I really think that we need to investigate the possibility of deliberate sophisticated sabotage here.  The mere fact that the pilots were utterly unable to get control back screams PLANNED FAILURE as the first investigatory target... 

.

The emerging 737 Max scandal, explained 

 
It’s more than bad software.
By Matthew Yglesias@mattyglesias

matt@vox.com

Mar 29, 2019, 9:10am EDT

An employee walks down a stairway leading to a Boeing 737 MAX airplane on March 14, 2019, in Renton, Washington. Stephen Brashear/Getty Images

https://www.vox.com/business-and-finance/2019/3/29/18281270/737-max-faa-scandal-explained

Boeing executives are offering a simple explanation for why the company’s best-selling plane in the world, the 737 Max 8, crashed twice in the past several months, leaving Jakarta, Indonesia, in October and then Addis Ababa, Ethiopia, in March. Executives claimed on March 27 that the cause was a software problem — and that a new software upgrade fixes it.


But this open-and-shut version of events conflicts with what diligent reporters in the aviation press have uncovered in the weeks since Asia, Europe, Canada, and then the United States grounded the planes.


The story begins nine years ago when Boeing was faced with a major threat to its bottom line, spurring the airline to rush a series of kludges through the certification process — with an underresourced Federal Aviation Administration (FAA) seemingly all too eager to help an American company threatened by a foreign competitor, rather than to ask tough questions about the project.


The specifics of what happened in the regulatory system are still emerging (and despite executives’ assurances, we don’t even really know what happened on the flights yet). But the big picture is coming into view: A major employer faced a major financial threat, and short-term politics and greed won out over the integrity of the regulatory system. It’s a scandal. 

 
The 737 versus 320 rivalry, explained


There are lots of different passenger airplanes on the market, but just two very similar narrow-body planes dominate domestic (or intra-European) travel. One is the European company Airbus’s 320 family, with models called A318, A319, A320, or A321 depending on how long the plane is. These four variants, by design, have identical flight decks, so pilots can be trained to fly them interchangeably.


The 320 family competes with a group of planes that Boeing calls the 737 — there’s a 737-600, a 737-700, a 737-800, and a 737-900 — with higher numbers indicating larger planes. Some of them are also extended-range models that have an ER appended to the name, and, as you would probably guess, they have longer ranges.


Importantly, even though there are many different flavors of 737, they are all in some sense the same plane, just as all the 320 family planes are the same plane. Southwest Airlines, for example, simplifies its overall operations by exclusively flying different 737 variants.


Both the 737 and the 320 come in lots of different flavors, so airlines have plenty of options in terms of what kind of aircraft should fly exactly which route. But because there are only two players in this market, and because their offerings are so fundamentally similar, the competition for this slice of the plane market is both intense and weirdly limited. If one company were to gain a clear technical advantage over the other, it would be a minor catastrophe for the losing company.


And that’s what Boeing thought it was facing.
The A320neo was trouble for Boeing


Jet fuel is a major cost for airlines. With labor costs largely driven by collective bargaining agreements and regulations that require minimum ratios of flight attendants per passenger, fuel is the cost center airlines have the most capacity to do something about. Consequently, improving fuel efficiency has emerged as one of the major bases of competition between airline manufacturers.


If you roll back to 2010, it began to look like Boeing had a real problem in this regard.


Airbus was coming out with an updated version of the A320 family that it called the A320neo, with “neo” meaning “new engine option.” The new engines were going to be more fuel-efficient, with a larger diameter than previous A320 engines, that could nonetheless be mounted on what was basically the same airframe. This was a nontrivial engineering undertaking both in designing the new engines and in figuring out how to make them work with the old airframe, but even though it cost a bunch of money, it basically worked. And it raised the question of whether Boeing would respond.


Initial word was that it wouldn’t. As CBS Moneywatch’s Brett Snyder wrote in December 2010, the basic problem was that you couldn’t slap the new generation of more efficient, larger-diameter engines onto the 737:


One of the issues for Boeing is that it takes more work to put new engines on the 737 than on the A320. The 737 is lower to the ground than the A320, and the new engines have a larger diameter. So while both manufacturers would have to do work, the Boeing guys would have more work to do to jack the airplane up. That will cost more while reducing commonality with the current fleet. As we know from last week, reduced commonality means higher costs for the airlines as well.


Under the circumstances, Boeing’s best option was to just take the hit for a few years and accept that it was going to have to start selling 737s at a discount price while it designed a whole new airplane. That would, of course, be time-consuming and expensive, and during the interim, it would probably lose a bunch of narrow-body sales to Airbus.


The original version of the 737 first flew in 1967, and a decades-old decision about how much height to leave between the wing and the runway left them boxed out of 21st-century engine technology — and there was simply nothing to be done about it.


Unless there was.
Boeing decided to put on the too-big engines anyway


As late as February 2011, Boeing chair and CEO James McNerney was sticking to the plan to design a totally new aircraft.


“We’re not done evaluating this whole situation yet,” he said on an analyst call, “but our current bias is to move to a newer airplane, an all-new airplane, at the end of the decade, beginning of the next decade. It’s our judgment that our customers will wait for us.”


But in August 2011, Boeing announced that it had lined up orders for 496 re-engined Boeing 737 aircraft from five different airlines.


It’s not entirely clear what happened, but, reading between the lines, it seems that in talking to its customers Boeing reached the conclusion that airlines would not wait for them. Some critical mass of carriers (American Airlines seems to have been particularly influential) was credible enough in its threat to switch to Airbus equipment that Boeing decided it needed to offer 737 buyers a Boeing solution sooner rather than later.


Committing to putting a new engine that didn’t fit on the plane was the corporate version of the Fyre Festival’s “let’s just do it and be legends, man” moment, and it unsurprisingly wound up leading to a slew of engineering and regulatory problems.
New engines on an old plane


As the industry trade publication Leeham News and Analysis explained earlier in March, Boeing engineers had been working on the concept that became the 737 Max even back when the company’s plan was still not to build it.


In a March 2011 interview with Aircraft Technology, Mike Bair, then the head of 737 product development, said that reengineering was possible.


“There’s been fairly extensive engineering work on it,” he said. “We figured out a way to get a big enough engine under the wing.”


The problem is that an airplane is a big, complicated network of interconnected parts. To get the engine under the 737 wing, engineers had to mount the engine nacelle higher and more forward on the plane. But moving the engine nacelle (and a related change to the nose of the plane) changed the aerodynamics of the plane, such that the plane did not handle properly at a high angle of attack. That, in turn, led to the creation of the Maneuvering Characteristics Augmentation System (MCAS). It fixed the angle-of-attack problem in most situations, but it created new problems in other situations when it made it difficult for pilots to directly control the plane without being overridden by the MCAS.


On Wednesday, Boeing rolled out a software patch that it says corrects the problem, and it hopes to persuade the FAA to agree.


But note that the underlying problem isn’t really software; it’s with the effort to use software to get around a whole host of other problems.



Recall, after all, that the whole point of the 737 Max project was to be able to say that the new plane was the same as the old plane. From an engineering perspective, the preferred solution was to actually build a new plane. But for business reasons, Boeing didn’t want a “new plane” that would require a lengthy certification process and extensive (and expensive) new pilot training for its customers. The demand was for a plane that was simultaneously new and not new.


But because the new engines wouldn’t fit under the old wings, the new plane wound up having different aerodynamic properties than the old plane. And because the aerodynamics were different, the flight control systems were also different. But treating the whole thing as a fundamentally different plane would have undermined the whole point. So the FAA and Boeing agreed to sort of fudge it.
The new planes are pretty different


As far as we can tell, the 737 Max is a perfectly airworthy plane in the sense that error-free piloting allows it to be operated safely.


But pilots of planes that didn’t crash kept noticing the same basic pattern of behavior that is suspected to have been behind the two crashes, according to a Dallas Morning News review of voluntary aircraft incident reports to a NASA database:


The disclosures found by the News reference problems with an autopilot system, and they all occurred during the ascent after takeoff. Many mentioned the plane suddenly nosing down. While records show these flights occurred in October and November, the airlines the pilots were flying for is redacted from the database.


These pilots all safely disabled the MCAS and kept their planes in the air. But one of the pilots reported to the database that it was “unconscionable that a manufacturer, the FAA, and the airlines would have pilots flying an airplane without adequately training, or even providing available resources and sufficient documentation to understand the highly complex systems that differentiate this aircraft from prior models.”


The training piece is important because a key selling feature of the 737 Max was the idea that since it wasn’t really a new plane, pilots didn’t really need to be retrained for the new equipment. As the New York Times reported, “For many new airplane models, pilots train for hours on giant, multimillion-dollar machines, on-the-ground versions of cockpits that mimic the flying experience and teach them new features” while the experienced 737 Max pilots were allowed light refresher courses that you could do on an iPad.


That let Boeing get the planes into customers’ hands quickly and cheaply, but evidently at the cost of increasing the possibility of pilots not really knowing how to handle the planes, with dire consequences for everyone involved.
The FAA put a lot of faith in Boeing


In a blockbuster March 17 report for the Seattle Times, the newspaper’s aerospace reporter Dominic Gates details the extent to which the FAA delegated crucial evaluations of the 737’s safety to Boeing itself. The delegation, Gates explains, is in part a story of a years-long process during which the FAA, “citing lack of funding and resources, has over the years delegated increasing authority to Boeing to take on more of the work of certifying the safety of its own airplanes.”


But there are indications of failures that were specific to the 737 Max timeline. In particular, Gates reports that “as certification proceeded, managers prodded them to speed the process” and that “when time was too short for FAA technical staff to complete a review, sometimes managers either signed off on the documents themselves or delegated their review back to Boeing.”


Most of all, decisions about what could and could not be delegated were being made by managers concerned about the timeline, rather than by the agency’s technical experts.


It’s not entirely clear at this point why the FAA was so determined to get the 737 cleared quickly (there will be more investigations), but if you recall the political circumstances of this period in Barack Obama’s presidency, you can quickly get a general sense of the issue.


Boeing is not just a big company with a significant lobbying presence in Washington; it’s a major manufacturing company with a strong global export presence and a source of many good-paying union jobs. In short, it was exactly the kind of company the powers that be were eager to promote — with the Obama White House, for example, proudly going to bat for the Export-Import Bank as a key way to sustain America’s aerospace industry.


A story about overweening regulators delaying an iconic American company’s product launch and costing good jobs compared to the European competition would have looked very bad. And the fact that the whole purpose of the plane was to be more fuel-efficient only made getting it off the ground a bigger priority. But the incentives really were reasonably aligned, and Boeing has only caused problems for itself by cutting corners.
Boeing is now in a bad situation


One emblem of the whole situation is that as the 737 Max engineering team piled kludge on top of kludge, they came up with a cockpit warning light that would alert the pilots if the plane’s two angle-of-attack sensors disagreed.


But then, as Jon Ostrower reported for the Air Current, Boeing’s team decided to make the warning light an optional add-on, like how car companies will upcharge you for a moon roof.


The light cost $80,000 extra per plane and neither Lion Air nor Ethiopian chose to buy it, perhaps figuring that Boeing would not sell a plane (nor would the FAA allow it to) that was not basically safe to fly. In the wake of the crashes, Boeing has decided to revisit this decision and make the light standard on all aircraft.


Now, to be clear, Boeing has lost about $40 billion in stock market valuation since the crash, so it’s not like cheating out on the warning light turned out to have been a brilliant business decision or anything.


This, fundamentally, is one reason the FAA has become comfortable working so closely with Boeing on safety regulations: The nature of the airline industry is such that there’s no real money to be made selling airplanes that have a poor safety track record. One could even imagine sketching out a utopian libertarian argument to the effect that there’s no real need for a government role in certifying new airplanes at all, precisely because there’s no reason to think it’s profitable to make unsafe ones.


The real world, of course, is quite a bit different from that, and different individuals and institutions face particular pressures that can lead them to take actions that don’t collectively make sense. Looking back, Boeing probably wishes it had just stuck with the “build a new plane” plan and toughed out a few years of rough sales, rather than ending up in the current situation. Right now the company is, in effect, trying to patch things up piecemeal — a software update here, a new warning light there, etc. — in hopes of persuading global regulatory agencies to let its planes fly again.


But even once that’s done, Boeing faces the task of convincing airlines to actually go buy its planes. An informative David Ljunggren article for Reuters reminds us that a somewhat comparable situation arose in 1965 when three then-new Boeing 727 jetliners crashed.


There wasn’t really anything unsound about the 727 planes, but many pilots didn’t fully understand how to operate the new flaps — arguably a parallel to the MCAS situation with the 737 Max — which spurred some additional training and changes to the operation manual. Passengers avoided the planes for months, but eventually came back as there were no more crashes, and the 727 went on to fly safely for decades. Boeing hopes to have a similar happy ending to this saga, but so far it seems to be a long way from that point. And the immediate future likely involves more tough questions.
A political scandal on slow burn


The 737 Max was briefly a topic of political controversy in the United States as foreign regulators grounded the planes, but President Donald Trump — after speaking personally to Boeing’s CEO — declined to follow. Many members of Congress (from both parties) called on him to reconsider, which he rather quickly did, pushing the whole topic off Washington’s front burner.


But Trump is generally friendly to Boeing (he even has a Boeing executive serving as acting defense secretary, despite an ongoing ethics inquiry into charges that he unfairly favors his former employer), and Republicans are generally averse to harsh regulatory crackdowns. The most important decisions in the mix appear to have been made back during the Obama administration, so it’s also difficult for Democrats to go after this issue. Meanwhile, Washington has been embroiled in wrangling over special counsel Robert Mueller’s investigation, and a new health care battlefield opened up as well.


That said, on March 27, FAA officials faced the Senate Commerce Committee’s Subcommittee on Aviation and Space at a hearing called by subcommittee Chair Ted Cruz (R-TX). Cruz says he expects to call a second hearing featuring Boeing executives, as well as pilots and other industry players. Cruz was a leader on the anti-Boeing side of the Export-Import Bank fight years ago, so perhaps he’s more comfortable than others in Congress to take this on.


When the political system does begin to engage on the issue, however, it’s unlikely to stop with just one congressional subcommittee. Billions of dollars are at stake for Boeing, the airlines that fly 737s, and the workers who build the planes. And since a central element of this story is the credibility of the FAA’s process — in the eyes of the American people and in the eyes of foreign regulatory agencies — it almost certainly isn’t going to get sorted out without more involvement from the actual decision-makers in the US government.