Sixteen Years Ago - The Unthinkable - April 20, 2010

Note: Following is an excerpt from my book, Into The Storm:

Sometime during the morning of April 21, 2010, I was sitting in my office at BP’s Westlake 4 Houston headquarters when I saw the first images of Transocean’s drilling rig, the Deepwater Horizon, flash across my computer monitor. And even though I knew that rig well, the only thing recognizable through the flames and billowing smoke were its four distinctly shaped support columns. The reality of what I watched was difficult to process. A catastrophic blowout had taken place the night before, resulting in an explosion and subsequent fire. We’d learn later that eleven men lost their lives and that the well was out of control, dumping oil into the Gulf of Mexico at an unprecedented rate. The Horizon had been working in BP’s Macondo field fifty miles off the coast of Louisiana in the Gulf of Mexico. The event would eventually come to be known internationally as the BP Horizon Oil Spill, one of the most devastating environmental catastrophes ever to occur in the Gulf of Mexico.

Years earlier, while working for Reading & Bates Drilling, I had led the operations team during the design and construction phase for its sister rig, the Deepwater Nautilus. About a year into the project, and after several months of reviews with a company called Vastar, a dynamically positioned version of the Nautilus was selected for their new 5th generation newbuild, and the Deepwater Horizon was born. Both rigs were built in South Korea, completed one year apart. The company representative leading the effort for Vastar, and the man responsible for selecting the Nautilus design, was named Don Weisinger. Years later, Don, the Horizon, and I all wound up working at BP.

Gabriel García Márquez wrote a famous book, One Hundred Years of Solitude, that’s set in a fictional, doomed town called Macondo. Following is part of the Wikipedia summary detailing how the novel ends:

As Aureliano reads the manuscript, he feels a windstorm starting around him, and he reads in the document that the Buendía family is doomed to be wiped from the face of the Earth because of it. In the last sentence of the book, the narrator describes Aureliano reading this last line just as the entire town of Macondo is scoured from existence.

I’m not sure how names of oil and gas exploration fields are chosen, but whoever came up with this one might not have been aware of its context. Maybe it was meant as an homage to a great South American writer. But I'm fairly certain whoever signed off on that name never actually read the book.

Shortly after the explosion and subsequent efforts to mitigate what was quickly becoming an event so unprecedented that there were real questions as to whether or not several of the largest companies in the world would survive, an article was published in the New York Times describing the offshore drilling industry as one where workers could find the last great American blue-collar jobs. At the time I found that to be an insightful observation….almost. The reality of just what these people do and the skills required is quite a bit more complicated. If it is one of the last true great blue-collar industries left, there is a legitimate reason. Working offshore on a highly complex floating vessel and attempting to drill tens of thousands of feet into the Earth in thousands of feet of water is one of the most technologically challenging  pursuits imaginable. In reality, it has more in common with NASA than US Steel. I suppose one of the main differences is that in space things don’t corrode, and there are no hurricanes.

The whole world watched in shock as the images flashed across every television and Internet news story that day in April 2010. Even after all this time, it’s still hard to comprehend the facts of what happened. The Horizon had finished a very difficult well in the Macondo field and was preparing for a move to a new location. Thinking the well was secured, they were in the process of displacing the drilling mud in the marine riser when gas entered the well bore and a blowout occurred. Once the gas reached the surface, it immediately ignited resulting in an explosion.

Soon after, I was showing the security guards my badge and entering the third floor of BP’s Westlake 4 building, the same building where my office was located several floors above. I'd been working for BP as a consultant for nearly a decade, and had formed my own consulting company several years ealier.

I suddenly remembered something I hadn’t thought about before. Days earlier I was approached in my office by a man who was about to travel to the Horizon to temporarily fill in for one of the BP rig supervisors. He was experienced but had mostly worked on land rigs up until that point. Before heading out, he asked me for a brief orientation on some of the subsea systems, particularly the marine drilling riser. His name was Bob Kaluza, and had agreed to work a five-day hitch to fill in. While writing this, I looked him up and see now he was due to return home the day after the blowout occurred. As a result of his four days onboard the Deepwater Horizon, he, along with another BP superintendent Donald Vidrine (portrayed as an overly evil villain by John Malkovich in the movie), spent years fighting prosecution by the government and were eventually acquitted. But both lost their jobs and nearly went to prison, not to mention almost losing their lives that night. Those four days out of the thousands the Horizon had operated—what were the odds?

As I walked through the doors, I saw a mix of industry personnel, many of whom I knew from the past and present, some from companies outside of BP. A live wall-sized video feed displayed images of the blowout preventer and mangled drilling riser at the bottom of the ocean. Multiple remotely operated vehicles (R.O.V.s—subsea robots) silently navigated around the site, some manipulating hydraulic hoses attached to the B.O.P. as oil continued to flow. The scene was surreal.

Everything I’m recounting in this chapter is a matter of public record. I’ve referred to government reports for the exact timeline of some events, but it’s not my intention to include much of what occurred outside of what I experienced personally. For anyone interested, the information is easily accessible online. Specific events and timelines are well documented.

Needless to say, attempts to gain control of the B.O.P. were the first priority, but alternative efforts to kill the well were already being planned. Prior to arriving, I’d been told that at one point, it was briefly believed the team may have been successful in firing the shear rams, since there was a noticeable jerk of the giant B.O.P. stack. That would have been a true game-changer. But nothing happened as a result of whatever had occurred.

Oil flowed through the broken drill pipe, as well as the torn riser. There was also flow from the riser mud boost line, which is a smaller auxiliary line attached to the riser main tube used to help transport cuttings to the surface when drilling. We were divided into three groups. Ours was led by a man named Gordon Birrel, a British BP manager who showed real leadership during this nightmare. When putting this part of the book together, I wasn’t surprised to find out he’s now part of BP’s senior executive group. Our team’s project manager was a man named Trevor Smith who showed equally impressive ability in maximizing the group’s cumulative skills sets.

During that time there was an influx of government officials, some of them cabinet-level. Early on, Secretary of the Interior Ken Salazar gave our team a pep talk, then told the press scrum outside the building that the government had their boot on BP’s throat. The company was red meat for a very hungry and rightfully outraged public. At one point sometime later, Secretary Salazar even threatened to push BP out of the way. I’m not entirely sure what the replacement would be, although I suppose it sounded good to the press and the public. But the company was doing everything humanly possible both from a technological and financial standpoint.

I also remember sitting next to the Energy Secretary, Dr. Steven Chu, a Nobel Prize-winning physicist, typing furiously on his laptop. He appeared to be communicating with someone via email and I remember wondering if it was President Obama. Secretary Chu would eventually make a very important decision that helped end the nightmare during the initial stages of installing a cap that temporarily sealed the well. But unfortunately that was much farther down the road.

I was assigned the responsibility of coordinating the effort to plug the boost line. The obvious intention was to reduce flow, but also to measure the pressure inside the line which some believed might help in calculating the actual volume of oil flowing. The estimated amount was constantly being debated by several government experts, as well as BP engineers. Those estimates varied considerably.

Some of the construction personnel proposed an overshot device commonly used on pipelines that would fit around the outer diameter then seal. But it was bulky, too heavy, and this had to be installed by an R.O.V., basically a subsea robot. I had real doubts it would work. But we had two overshots put on a jet and flown to Houston. I don’t remember much about why I was so convinced it wouldn’t work. Probably something to do with its weight or possibly its size relative to the gap between the boost line and the main riser tube. There may not have been adequate clearance. For whatever reason, I just remember it had failure written all over it and we needed a win. Also, it wasn’t like we could easily try different options until one worked. Preparing was a tedious process and took days, not only to develop the procedures, but for the customization of the R.O.V. tools required. If it failed, we’d have to start over.

The whole oil industry universe was on call, and I spent all of that first night talking to vendors from around the world, trying to find something that would work better than the overshot. Finally, I discovered a mechanical plug online and knew that was it. I can’t even remember its main application, but it certainly wasn’t sealing riser boost lines. It was designed to be inserted into the inner diameter of various pipe sizes, then mechanically expanded to seal. We arranged to have two of them hot-shotted to the R.O.V. support vessel and started developing the installation procedure. By the time we finally got everything in place to start, I had been awake for days, so it’s all blurry at this point.

I don’t remember the exact details of what was required, only that the line had to be sawed evenly on the end first. This was also done by an R.O.V.—Looking at the government’s timeline, BP issued a press release that one of the three leaks had been plugged on May 5. That would have been the boost line. All I remember is that it was early evening, and I have no idea what day of the week, since by then all the days had melted together. Recounting all this it sounds like the installation of the plug would have been a fairly simple endeavor but that certainly wasn’t the case at the time, and we spent several days walking through the procedures as well as risk assessment exercises. Luckily, it all came together, mainly due to the skill of the R.O.V. operator from 5,000’ above. It was a small win but something sorely needed at that stage.

During that time I had my last email conversation with Don Weisinger. He had sent a frantic message reminding me that all those years ago we had designed the Horizon’s B.O.P. control pods to be removable, and maybe that provided an option to gain some degree of control. He ended the message with, Do something, try anything! The realization that the rig we’d both had so much in common with had literally exploded into one of the most tragic nightmares imaginable was not lost on either of us. The fact that this was happening at all was almost inconceivable. That it was the Horizon just wouldn't sink in.

Later that year, on Christmas Day, Don died of a heart attack. He was fifty-seven. When it happened, I couldn’t bring myself to talk about it with anyone. Writing this here is the closest I’ve ever come. The disturbing reality that Don went to his grave the same year the Horizon blew up and sank to the bottom of the Gulf of Mexico adds an even deeper layer of emotional complexity to this tragedy and is something that will haunt me on some level for the rest of my life.

Meanwhile, BP had been working around the clock to fabricate three containment domes, each a different size. By now the attempt to function the B.O.P. was looking like less of a viable option. The idea was to lower one of the domes on top of the leaking well and channel the flow of oil to the surface. But this concept ran into problems due to methane hydrates, a common issue when working in deepwater. Basically, they’re made up of methane gas molecules locked inside frozen water crystals. It looks like ice and forms at low temperatures and high pressures deep in the ocean. Hydrates can be very problematic for B.O.P.s as well. The containment dome concept was, for the time being, put on hold. Later, it would be revisited but with little success.

The next attempt was something known as a junk shot. Basically this is just what it sounds like: shredded tires, golf balls, knotted rope, almost anything you can imagine would be pumped into the B.O.P. below the shear rams in an effort to clog it up and stop the flow, or at least reduce it to a trickle. As primitive and strange as it sounds, this technique has been pretty successful throughout the world on land rigs with surface B.O.P.s, but the pressure and temperature at 5,000’ of water here complicated things and greatly reduced the chances of the junk shot working. There were also concerns about several right angles inside the B.O.P. that were considered problematic.

Based on gamma-ray inspection, there was a belief that at least some of the pipe rams had closed, but the shear rams, the top set, for some reason had not. A lot of work had gone into preparing for the junk shot and the materials used not as random as it sounds. Extensive testing had been done to determine what types of items might work best in this particular application. I was told by one of the experts that in surface B.O.P. applications, lamp wire is often successful due to its combination of bendable copper and rubber.

Just preparing for the operation required a significant plumbing effort performed by R.O.V.s controlled on the surface by operators on workboats. These operators were highly skilled and I believe there were as many as seven or eight R.O.V.s working at any given time. The area was congested, and the precision required was incredible. The effort was all coordinated by a BP R.O.V. expert from our location. I remember one particular image of several R.O.V.s working simultaneously on one particular fitting with an amazing level of dexterity. Someone said what we were all thinking, that we needed to get a picture of this.

If the junk shot was successful in stopping or even slowing the flow significantly, heavy drilling mud would then be forced down into the well (called a top kill) followed by cement. Ultimately though, it didn’t work—a huge disappointment. Focus returned to the containment domes and ways to control the hydrate problem. At some point one, or possibly both of the control pods for the B.O.P. were disconnected and brought to the surface, something that was possible since they’d been designed for that by our team a decade earlier, based on Don Weisinger’s request. However, I wasn’t involved in that effort.

As I mentioned in an earlier chapter, Tony Hayward was the CEO of BP at the time, having taken over from John Browne three years earlier. Browne had gotten caught up in controversy over improprieties around using company resources for a romantic interest which eventually led to perjury charges. Hayward had been seen as a pragmatic and no-nonsense replacement. There had been a very proactive internal BP public relations campaign that seemed intended to cultivate a more progressive image of a new generation leader, one who was more engaged and accessible than his predecessor.

As it became clear nothing in the arsenal of technology that existed at the time was going to end the nightmare quickly, Hayward’s presence became more high-profile in the media. The outraged American public demanded accountability, and Hayward was BP’s public face. Even a CEO with the most savvy public relations skills would have struggled with a situation like this, but the enormity of the vitriol Hayward faced was equaled only by a strange inability to communicate the right message and tone to government officials as well as the American public. Each interaction seemed to result in clips played repeatedly by the media, dragging him and the company into a more negative light. It certainly didn’t help when President Obama said in an interview with NBC that he would have sacked Hayward if it were up to him.

The finger-pointing between BP, Transocean, and Halliburton (the third party that had provided the cement services for the well) had already started. Meantime, it became obvious to all the magnitude of what had happened could ultimately result in an epic management purge. The only question was how high up the ladder that purge would climb. It was impossible to tell. There was no precedent for what was happening. I’ll never forget walking into that dark room (dark so we could see the projection TV images of the flowing well) around 2:00 a.m., having been awake more hours than I can remember, and hearing one of Hayward’s top lieutenants pronouncing that he’d been informed if we weren’t able to accomplish the specific task at hand, he would be asked to resign. The dozen or so of us who were there at first thought it was a poor attempt at a joke. But it became clear to me that he was serious, and he wasn’t smiling.

Although we were successful that night with some minor task, he was forced out and sent home within the next few days. Since he was technically just below Tony Hayward, it was pretty clear to all accountability had escalated and there was only one rung on the ladder to go. Although the exact reason the man was dismissed wasn’t announced, it seemed like a sacrificial move to appease those who were calling for a senior level head to roll, possibly a result of Hayward sensing that his own career was about to be swallowed by the cursed hole. As it turned out, the sacrifice didn’t work.

At some point I started getting pulled into meetings outside of the Horizon mitigation effort, focusing on the PDQ B.O.P.. Obviously, the potential ramifications of what had happened were starting to reverberate to other operations, and there was none bigger than Thunder Horse. By then drilling in the entire Gulf of Mexico had been suspended. I spent the next months pretty much waiting for the inevitable changes to the federal regulations that would eventually impact Thunder Horse. I won’t go into the technicals here but multiple modifications were implemented to the B.O.P.s of most, if not all, deepwater rigs in the Gulf of Mexico as a result of new federal guidelines.

Sometime in July, a functioning containment cap was finally installed that temporarily stopped the flow while two relief wells were being drilled by separate rigs, the Development Driller III being the first to reach the target. During this time, Secretary Chu made a critical decision regarding pressure build-up below the cap that led to its ultimate success. I had not realized that until researching the event for this book as I’d long since moved back to the PDQ operation by then to focus on it. The containment cap had held, but nobody knew if it would eventually leak.

At this point, the only way to ensure the well was forever dead was to plug it and the only way to do that was to drill a relief well. There is undoubtedly an interesting story around the engineers who led the effort and navigated the relief well’s trajectory with the level of precision required to hit a target the size of a dinner plate from a floating drilling rig through 13,000’ of formation under 5,000’ of water. The task was spearheaded by John Wright, Bill McElduff  and Donal Fitterer, each part of an elite club of less than a dozen engineers in the world who specialize in relief wells. I often wonder if the drill crew encountered any of the same issues that the Horizon did since the Macondo well had been so problematic even before the blowout. The well was finally plugged in mid-September—a cement stake permanently shoved into its dark heart.

On October 1, Bob Dudley assumed the CEO position. The total estimated cost to BP at that time was $40 billion. Dudley had endured his own stint in corporate hell while in Russia overseeing BP’s assets there and had nearly been forced to flee the country, rumored to have incurred the wrath of Vladimir Putin himself. There were even stories that he had been poisoned while there, although I don’t believe that was ever substantiated. Years earlier, Dudley had been a candidate for the CEO job as well, but it had gone to Hayward. I’ve often wondered what might have happened had those decisions been reversed; if Dudley would have been able to navigate the situation without the benefit of hindsight from Hayward’s missteps. Or if Hayward might have found himself stepping in for Dudley.

Either way, after everything that had happened, Dudley did seem better equipped to handle the delicate public relations balancing act that would be required if the company was to regroup. It didn’t hurt that he was American and possibly more relatable to some who had been directly affected by the environmental damage to the Gulf Coast.

Needless to say, employee morale was in shambles. Many exited as soon as they could find new jobs. A series of comprehensive reorganizations ensued throughout the company as everyone braced for the aftermath. And the change to the industry, particularly the Gulf of Mexico, was seismic.

Even now, over sixteen years later, the scope of it all seems impossible. The idea that a movie starring famous actors would someday reenact the Horizon’s tragic demise like some modern-day Greek tragedy almost seems to bend reality.