There’s an old saying among lawyers: When you have the facts on your side, pound on the facts. When you have the law on your side, pound on the law. When you have neither, pound on the table.

At first glance, that seems to be Samuel Moyn’s counsel in a widely shared Boston Review essay, “Resisting the Juristocracy.” “In the face of a solid conservative majority for years to come,” what are progressives to do? Looking for help from Chief Justice John Roberts as the new “swing” vote, Moyn reasonably concludes, is hopeless. There is little more reason for optimism in fanciful plans to pack the Court or impeach Kavanaugh. The problem is not the personnel who comprise the Court, Moyn argues; the problem is that we have “a cult of the higher judiciary,” a cult that progressives themselves have done much — even more than conservatives — to enshrine and insulate from democratic politics.

Robbed of the power to shuffle the composition of the Court, in other words, progressives need to de-consecrate the Supreme Court, to shift power back towards legislative bodies. Here is, I think, the key paragraph outlining his argument for a future progressive strategy:

In the face of an enemy Supreme Court, the only option is for progressives to begin work on a long-term plan to recast the role of fundamental law in our society for the sake of majority rule — disempowering the courts and angling, when they can, to redo our undemocratic constitution itself. This will require taking a few pages from the conservative playbook of the last generation. It is conservatives who stole the originally progressive talking point that we are experiencing “government by judiciary.” It is conservatives who convinced wide swathes of the American people that it is the left, not the right, that too routinely uses constitutional law to enact its policy preferences, no matter what the text says. The truth is the reverse, and progressives need to take back the charge they lost. To do so, they need to abandon their routine temptation to collude with the higher judiciary opportunistically. Progressives must embrace democracy and its risks if they want to avoid the stigma of judicial activism that still haunts them from the past.

As advice, that seems to me sound. But as history, Moyn’s account of why progressives find themselves in this predicament is only half-accurate, and its inaccuracy may have serious and harmful consequences for the way progressives strategize and for the way all parties understand the Supreme Court’s place in American life.

After recommending that progressives “embrace democracy and its risks,” Moyn spells out a brief history of how progressives brought the “stigma of judicial activism” upon themselves. Liberals gifted conservatives with this talking point by relying “on black-robed power to enact their preferences” in cases focused on the right to privacy — most importantly with regard to abortion and LGBT rights. “[L]iberals entered an unholy alliance with Kennedy, Kavanaugh’s predecessor, to advance gay and women’s rights on a libertarian rationale — defending the free choice of individuals independent of state control — even though that rationale mostly serves business interests in most areas of law.”

In other words, Moyn is arguing that it was largely the desire to advance the causes of women and sexual minorities that drew liberals and progressives to throw themselves on the ark of the Court. Progressives thereby endangered their broader agenda by failing to win majority support among the whole population for those causes before trying their luck with “black-robed power.” It was almost a simple matter — even as abortion and gay marriage attained majority support in the wake of the Court’s decisions — for conservatives to turn this short-circuiting of the democratic process into a cudgel with which to beat progressives. Progressives’ willingness, in words that Moyn quotes from Roberto Unger, “to obtain from judges, under the cover of improving interpretation, the advances popular politics fail to deliver,” effectively ceded to conservatives the claim to represent the “real” majority of U.S. voters.

There is an undeniable measure of truth to this narrative, but only as a radically truncated history of the relationship between progressivism, the Court, and conservative backlash. The progressive love affair with judicial remedies over legislative solutions is much older than Roe v. Wade (or Griswold v. Connecticut), and the older part of its history still matters greatly.

Before briefly tracing that history, though, I want to explain why it matters, and why I am concerned by a narrative that identifies women’s and LGBT rights as the causes which tipped progressivism into its fatal reliance on judicial fiat.

By singling out cases that protect the rights of women and sexual minorities as (initially) countermajoritarian and improvidently rushed into law, Moyn positions those groups and those rights as in some way existing on the fringe of a progressive bloc and the progressive agenda: he is in effect arguing that their vulnerability is an indication of their attenuated connection to a progressive core. He comes close to saying this directly when he grimaces, “The endangerment or even loss of precedents that the left cares about (such as Roe) is going to be a grievous blow, and no one should celebrate that outcome. But if it is going to happen anyway, then it is time to pivot to a democratic strategy to protect what we care about.”

Clearly, Roe is still within the ambit of “what we care about” for Moyn, and I’m not trying to question his commitment either to women’s or LGBT rights. But I do feel that the history he has provided — which singles out those cases that guaranteed these rights as the source of conservatism’s politically successful accusation of judicial activism — has an intrinsic logic that leads to treating those rights as somehow more marginal and more tendentiously grounded than, say, the rights of labor or environmental regulations or the rights of racial and religious minorities.

It is dangerous to single out women’s and LGBT rights first because when conservatives have spoken of judicial activism over the past fifty years, they have included issues such as school prayer, desegregation, health care, environmental and workplace safety protections, and federal oversight over state-run elections. This is doubly true because “judicial activism” has been only one prong of a broader assault on the legitimacy of liberal/progressive victories in the twentieth century. Alongside the attempt to undermine the validity of cases like Roe or Obergefell (or Engel v. Vitale), conservatives have also poured enormous intellectual resources into attacks on administrative law as fundamentally unconstitutional, frequently using the same kind of logic: i.e., that liberals and progressives have achieved their gains by evading the hard work of achieving popular mandates and legislative solutions. Nor have they stopped at the Capitol’s doors: as we’ve seen with the right’s polymorphous assault on voting rights, it matters little whether the basis of a law comes from a legislative act, an administrative code, or a Supreme Court decision — all may be under threat.

My point is not that progressives ought to continue to follow a strategy of trying to hold a bare majority on the Court or to hope to lure a swing vote with tailor-made legal reasoning in order to achieve a particular substantive end (e.g., the legalization of gay marriage). That is not a sustainable or prudent strategy and never was. But progressives must realize that their Court-worship is a much older phenomenon and consequently that a far broader array of rights and protections are entangled with “black-robed power.”

The origins of that entanglement are not to be found within the Warren Court nor should they be dated to FDR’s ill-conceived court-packing plan — a solution which, even had it worked, would not have put a dent in the supremacy of judicial review but only made the importance of a single Justice’s vote marginally smaller.

Instead, we need to look back to the progressive veneration of one particular Justice as the seed of more than a century of liberal Court-worship: Oliver Wendell Holmes, Jr. The personal hold that Holmes had over many of the most influential Progressives was profound, and they evangelized on his behalf almost as much as they spread the gospel of progressivism. When Louis Brandeis was appointed to the Supreme Court, the Progressive movement had a second jurist-apostle to idolize, and again, personal admiration as much as political comradeship fueled the cult of Brandeis and solidified a new eagerness to pursue juridical remedies over legislative solutions.

Much of this history can be found in biographies of these figures; a recent book, The House of Truth: A Washington Political Salon and the Foundations of American Liberalism , by Brad Snyder, collects many of the details, often demonstrating in elaborate detail the immense trust that Progressives placed in the wisdom and character of Holmes and Brandeis. At no point did the dangerous effects of this childlike reverence become more clear than in the painful episode of Sacco and Vanzetti, when both Holmes and Brandeis repeatedly denied progressives’ pleas for a stay of execution. Up until the very hour of electrocution, progressives like Felix Frankfurter (later a Supreme Court Justice himself) were sure that Holmes would come through.

But there are other sources as well: a generation of scholarship on administrative law has demonstrated how much the development of the “administrative state” has owed to its veneration of judicial impartiality and how deliberately judges have tried to shape administrative agencies’ internal operations in the image of the judiciary. The best monograph illustrating these dynamics is Daniel Ernst’s Tocqueville’s Nightmare: The Administrative State Emerges in America, 1900-1940, and the key figure in that history is another Supreme, Chief Justice Charles Evans Hughes.

These histories precede the narrative Moyn lays out by decades, but the significance of that backdating lies in the way they shift our understanding of what is at risk in this new dispensation of the Supreme Court and what progressives must do to rethink their relationship with the judiciary. In fact, the judiciary may be only part of the problem: what becomes clear when we re-focus on Holmes and Brandeis as the origins of progressives’ reverence for the Court is that this reverence has more often been personal rather than institutional in nature — it has the taste more of hero-worship than Court-worship.

We can still, of course, see that hero-worship in the cult figure that Ruth Bader Ginsburg has become for many on the Left — even if she is (I would say) much more worthy of our devotion than Oliver Wendell Holmes, Jr.

If hero-worship is one part of the problem, though, a crippling self-consciousness about constitutional legitimacy may be another part — as Moyn indicates in an aside. Ironically, for many of the original Progressives, this was not the case — they were far more willing to think of the Constitution as an instrument rather than as Holy Writ.

Moyn is surely right to stress the need for an open examination of progressivism’s reliance on the judiciary. But we must have our history right before we can truly make sense of what all is wrong, and what all there remains to do.

Andrew Seal teaches economic history at the University of New Hampshire. This essay was originally published by the Society for U.S. Intellectual History