NOAA Fisheries Science: A Guess, a Coin-Flip, and a Kick in the Teeth

Protecting our national fish stocks from ruin by overfishing is a national imperative that requires both good management and solid science.  NOAA, the agency responsible for managing our fisheries, is incapable of even adequate management, as shown by a recent independent review by Preston Pate.

At a 4/13/11 Senate subcommittee hearing (video from minute 98), Senator Ayotte of New Hampshire challenged NOAA Administrator Dr. Jane Lubchenco: "NOAA has been relying on ... incomplete and sometimes non-existent [fisheries] data."  Dr. Lubchenco responded with the expected evasion and the familiar patronizing talking point about "best available science information that we have."

Dr. Lubchenco, what if your best available science is not very good?  What if, in fact, it is flawed?  This is not a rhetorical question, as there is evidence that the vaunted NOAA fisheries science is indeed unsound.

Every year, by law, NOAA puts out a fisheries report to the Congress.  This is NOAA's chance to show off in the very best light.  Table 1 of NOAA's annual report to Congress on the 2009 Status of U.S. Fisheries covers 522 fish stocks.  Twenty-three percent of the most important 230 stocks have no data, being termed either "unknown" or "undefined."  Is this what we have to show for forty years of effort -- a three-quarter understanding of our important fish stocks?  Overall, of the 522 stocks, 51% have no data.  No data on over half of the fish stocks, and this is our best available science?

Of course, the annual fisheries report does not say anything about the plight of the fishermen.  NOAA instituted a management system known as catch shares into New England fisheries in May 2010.  At the same time, NOAA reduced the annual catch limits (ACL) from the previous year by some 30%-50%.  The results have been as precipitous as they were predictable: ten percent of the boats are doing well under the new system, but over half the fleet never left the dock for lack of adequate allocations.

Massachusetts Governor Patrick petitioned Commerce Secretary Locke for emergency relief.  The governor provided Locke with a study by the Massachusetts Marine Fisheries Institute (MFI) showing that allocations could be raised by at least 30% without violating the governing Magnuson-Stevens Act (MSA).  Locke turned him down.

The MFI Study is, in a manner of speaking, an exposé of the process of determining the fisherman's ACL.

In addition to the MFI Study, this essay draws upon the work of two of the principals behind the study: Dr. Brian Rothschild and Dr. Steven Cadrin, both of UMass/Dartmouth and both highly respected in the fisheries sciences.  The conclusions herein are the author's and not necessarily those of Dr. Rothschild or Dr. Cadrin.

Before peeling the onion a couple of layers, I need to introduce you to the lingo.  Scientists protect their job security with impenetrable notation and nomenclature.  The fisheries scientists are particularly and uniquely esoteric.  Sorry, but it is science.

Maximum Sustainable Yield (MSY) is the highest catch that can be taken while maintaining a sustainable population.  There are two other terms of interest: fishing mortality (F) and biomass (B), the total weight of the population.  The fishing mortality associated with MSY is FMSY and the biomass associated with MSY is BMSY.  Dr. Cadrin, in a presentation (video) to the New England Fisheries Management Council (NEFMC), described FMSY and BMSY as the "scientific underpinnings of fisheries science." 

NOAA data shows FMSY as unknown for twelve of the nineteen stocks covered by the MFI Study.  Unknown underpinnings are unsatisfactory. 

NOAA's workaround for the missing data is appalling.  They use what Dr. Rothschild in a paper titled "The Overfishing Metaphor" refers to as "generally arbitrary substitutes (so-called "proxies") [that] are contrived to replace the optimization target."  Contrived, arbitrary substitute data?  No wonder Dr. Lubchenco dodged Senator Ayotte's question.

"Proxies" are at best little more than a guess.  Solid science is not based on guesses.

In the seven cases where a proxy was used in place of a known FMSY, the result was some 30% lower than the known FMSY.  This 30% comes right off the top of the fisherman's ACL.

The MFI Study identified a second source of scientific uncertainty in the stock assessment itself.  The report states, "Several models had substantial uncertainty manifest by retrospective inconsistency."  How's that for a mouthful?

I think it means that the assumptions about fisheries sampling techniques that were made in the 1970s are breaking down with time.  Dr. Rothschild puts it this way: "It is pretty clear that the data and models are wildly divergent (e.g., retrospective patterns)."

As I understand it, the mathematical model for a given stock (called the Base Case Model) developed in the early days of NOAA gave reasonably consistent results year-to-year.  Sometime in the 1990s, the data became "inconsistent," and some scientists felt it necessary to revise the modeling.  A competing model of the stock referred to as the Split Survey Model was developed.  The Split Survey Model gives consistently lower results than the Base Case Model. 

For Georges Bank yellowtail flounder, the quantitative difference is more than two to one, with the Split Survey being the lower.  Two to one is a significant difference.

The following is from the MFI Study: 

Although 'base case' models have diagnostic problems, they are the simplest analysis of all available data, ... By comparison, split survey models imply substantial increases in survey efficiencies (some greater than 100%); and adjusted models account for a potential bias that is not understood and may not persist. 

Phrases such as "efficiencies greater than 100%" (i.e., perpetual motion), "bias that is not understood," and "conditions that may not persist" do not inspire confidence.  The phrase "diagnostic problems" is also quite low on my comfort meter, for that matter.

Faced with two competing models -- at least one of which, if not both, had to be wrong -- NOAA made an inexplicable management error.  They allowed the scientists to present both models to a peer review.  Good business practices would have required in-house consensus before going public.  The peer review "could not distinguish whether either of them was right or wrong," (Cadrin video, previously linked) proving once again that Murphy is a NOAA manager.

If the proxy for FMSY represents a guess, then picking one of two competing models is a coin-flip.  However, NOAA has arbitrarily decided to always use the Split Survey model (which might well be technically incorrect) so as to be risk-averse.  Voilà: another significant lop-off from the fisherman's ACL.

How is it that the vaunted NOAA science seems to rely on unsound scientific principles -- a guess and a coin-flip -- to reach decisions that drive fishermen into unemployment and bankruptcy?  The answer is risk-avoidance.

At a 3/8/11 Senate subcommittee meeting (two-hour video here), Senator Snowe asked Dr. DeMaster, the NOAA fisheries scientist, what would be a comfortable risk.  He said that risk by law should not exceed 50%, but that he, as a professional, found 50% too high, or "not risk averse enough."  That term again. 

"Risk-averse" implies a willingness to accept a reduced payback to mitigate risk.  When Dr. DeMaster advocates mitigating the overfishing risk, does he understand that the reduced payback he is so graciously willing to accept has to be borne by the fishermen?  I think not.  But the folks at NOAA do not think in terms of fishermen -- just in terms of fish.

Punishing the fisherman while championing the fish has long been a NOAA tradition.  NOAA science cannot measure FMSY, so the organization penalizes the fisherman by slashing his ACL in the name of reducing the risk of overfishing.  NOAA uses the model that gives the worst-case results for the fisherman's ACL in selecting from "widely divergent" models, again to reduce the risk of overfishing.  The reductions in ACL are so severe that little if any risk of overfishing remains, but NOAA is not done yet.  Because the vaunted NOAA fisheries science is so "uncertain," NOAA management imposes another 25% reduction in ACL as an "uncertainty buffer."

Now, that's a kick in the teeth.

Mike Johnson is a concerned citizen, a small government conservative, and a live-free-or-die resident of New Hampshire.  E-mail mnosnhoj@comcast.net.

Protecting our national fish stocks from ruin by overfishing is a national imperative that requires both good management and solid science.  NOAA, the agency responsible for managing our fisheries, is incapable of even adequate management, as shown by a recent independent review by Preston Pate.

At a 4/13/11 Senate subcommittee hearing (video from minute 98), Senator Ayotte of New Hampshire challenged NOAA Administrator Dr. Jane Lubchenco: "NOAA has been relying on ... incomplete and sometimes non-existent [fisheries] data."  Dr. Lubchenco responded with the expected evasion and the familiar patronizing talking point about "best available science information that we have."

Dr. Lubchenco, what if your best available science is not very good?  What if, in fact, it is flawed?  This is not a rhetorical question, as there is evidence that the vaunted NOAA fisheries science is indeed unsound.

Every year, by law, NOAA puts out a fisheries report to the Congress.  This is NOAA's chance to show off in the very best light.  Table 1 of NOAA's annual report to Congress on the 2009 Status of U.S. Fisheries covers 522 fish stocks.  Twenty-three percent of the most important 230 stocks have no data, being termed either "unknown" or "undefined."  Is this what we have to show for forty years of effort -- a three-quarter understanding of our important fish stocks?  Overall, of the 522 stocks, 51% have no data.  No data on over half of the fish stocks, and this is our best available science?

Of course, the annual fisheries report does not say anything about the plight of the fishermen.  NOAA instituted a management system known as catch shares into New England fisheries in May 2010.  At the same time, NOAA reduced the annual catch limits (ACL) from the previous year by some 30%-50%.  The results have been as precipitous as they were predictable: ten percent of the boats are doing well under the new system, but over half the fleet never left the dock for lack of adequate allocations.

Massachusetts Governor Patrick petitioned Commerce Secretary Locke for emergency relief.  The governor provided Locke with a study by the Massachusetts Marine Fisheries Institute (MFI) showing that allocations could be raised by at least 30% without violating the governing Magnuson-Stevens Act (MSA).  Locke turned him down.

The MFI Study is, in a manner of speaking, an exposé of the process of determining the fisherman's ACL.

In addition to the MFI Study, this essay draws upon the work of two of the principals behind the study: Dr. Brian Rothschild and Dr. Steven Cadrin, both of UMass/Dartmouth and both highly respected in the fisheries sciences.  The conclusions herein are the author's and not necessarily those of Dr. Rothschild or Dr. Cadrin.

Before peeling the onion a couple of layers, I need to introduce you to the lingo.  Scientists protect their job security with impenetrable notation and nomenclature.  The fisheries scientists are particularly and uniquely esoteric.  Sorry, but it is science.

Maximum Sustainable Yield (MSY) is the highest catch that can be taken while maintaining a sustainable population.  There are two other terms of interest: fishing mortality (F) and biomass (B), the total weight of the population.  The fishing mortality associated with MSY is FMSY and the biomass associated with MSY is BMSY.  Dr. Cadrin, in a presentation (video) to the New England Fisheries Management Council (NEFMC), described FMSY and BMSY as the "scientific underpinnings of fisheries science." 

NOAA data shows FMSY as unknown for twelve of the nineteen stocks covered by the MFI Study.  Unknown underpinnings are unsatisfactory. 

NOAA's workaround for the missing data is appalling.  They use what Dr. Rothschild in a paper titled "The Overfishing Metaphor" refers to as "generally arbitrary substitutes (so-called "proxies") [that] are contrived to replace the optimization target."  Contrived, arbitrary substitute data?  No wonder Dr. Lubchenco dodged Senator Ayotte's question.

"Proxies" are at best little more than a guess.  Solid science is not based on guesses.

In the seven cases where a proxy was used in place of a known FMSY, the result was some 30% lower than the known FMSY.  This 30% comes right off the top of the fisherman's ACL.

The MFI Study identified a second source of scientific uncertainty in the stock assessment itself.  The report states, "Several models had substantial uncertainty manifest by retrospective inconsistency."  How's that for a mouthful?

I think it means that the assumptions about fisheries sampling techniques that were made in the 1970s are breaking down with time.  Dr. Rothschild puts it this way: "It is pretty clear that the data and models are wildly divergent (e.g., retrospective patterns)."

As I understand it, the mathematical model for a given stock (called the Base Case Model) developed in the early days of NOAA gave reasonably consistent results year-to-year.  Sometime in the 1990s, the data became "inconsistent," and some scientists felt it necessary to revise the modeling.  A competing model of the stock referred to as the Split Survey Model was developed.  The Split Survey Model gives consistently lower results than the Base Case Model. 

For Georges Bank yellowtail flounder, the quantitative difference is more than two to one, with the Split Survey being the lower.  Two to one is a significant difference.

The following is from the MFI Study: 

Although 'base case' models have diagnostic problems, they are the simplest analysis of all available data, ... By comparison, split survey models imply substantial increases in survey efficiencies (some greater than 100%); and adjusted models account for a potential bias that is not understood and may not persist. 

Phrases such as "efficiencies greater than 100%" (i.e., perpetual motion), "bias that is not understood," and "conditions that may not persist" do not inspire confidence.  The phrase "diagnostic problems" is also quite low on my comfort meter, for that matter.

Faced with two competing models -- at least one of which, if not both, had to be wrong -- NOAA made an inexplicable management error.  They allowed the scientists to present both models to a peer review.  Good business practices would have required in-house consensus before going public.  The peer review "could not distinguish whether either of them was right or wrong," (Cadrin video, previously linked) proving once again that Murphy is a NOAA manager.

If the proxy for FMSY represents a guess, then picking one of two competing models is a coin-flip.  However, NOAA has arbitrarily decided to always use the Split Survey model (which might well be technically incorrect) so as to be risk-averse.  Voilà: another significant lop-off from the fisherman's ACL.

How is it that the vaunted NOAA science seems to rely on unsound scientific principles -- a guess and a coin-flip -- to reach decisions that drive fishermen into unemployment and bankruptcy?  The answer is risk-avoidance.

At a 3/8/11 Senate subcommittee meeting (two-hour video here), Senator Snowe asked Dr. DeMaster, the NOAA fisheries scientist, what would be a comfortable risk.  He said that risk by law should not exceed 50%, but that he, as a professional, found 50% too high, or "not risk averse enough."  That term again. 

"Risk-averse" implies a willingness to accept a reduced payback to mitigate risk.  When Dr. DeMaster advocates mitigating the overfishing risk, does he understand that the reduced payback he is so graciously willing to accept has to be borne by the fishermen?  I think not.  But the folks at NOAA do not think in terms of fishermen -- just in terms of fish.

Punishing the fisherman while championing the fish has long been a NOAA tradition.  NOAA science cannot measure FMSY, so the organization penalizes the fisherman by slashing his ACL in the name of reducing the risk of overfishing.  NOAA uses the model that gives the worst-case results for the fisherman's ACL in selecting from "widely divergent" models, again to reduce the risk of overfishing.  The reductions in ACL are so severe that little if any risk of overfishing remains, but NOAA is not done yet.  Because the vaunted NOAA fisheries science is so "uncertain," NOAA management imposes another 25% reduction in ACL as an "uncertainty buffer."

Now, that's a kick in the teeth.

Mike Johnson is a concerned citizen, a small government conservative, and a live-free-or-die resident of New Hampshire.  E-mail mnosnhoj@comcast.net.