On May 23, 1861 — six weeks after the first shots of the Civil War were fired at Fort Sumter — Virginia voters ratified the state’s secession ordinance. Virginia no longer considered itself part of the United States of America.
That very night, three men — Frank Baker, Shepard Mallory, and James Townsend, who had been put to work building fortifications in Sewell’s Point as slaves of Col. Charles K. Mallory — escaped and made their way to Fort Monroe, which was still flying the U.S. flag in Hampton, Virginia.
It was a big risk. They couldn’t be certain what would happen when (and if) they reached their destination. At the time, the Fugitive Slave Act of 1850 was still in force. It stated in part that any slaves who escaped across state lines into free states were still legally considered the property of their slaveholders and must be returned. Escaped slaves had shown up at Fort Monroe before and been denied asylum. When Baker, Mallory, and Townsend set out that night, they could only hope that political shifts in the country had changed something in their favor. If they were caught or returned, they could expect harsh punishment.
The next day, Maj. John Cary of the 115th Virginia Militia arrived at the fort under a white flag. Speaking on behalf of Col. Mallory, Cary asked that the slaves be returned under the provisions of the Fugitive Slave Act.
The man in charge of Fort Monroe — who was ultimately responsible for responding to Cary — was Maj. Gen. Benjamin F. Butler. A Massachusetts lawyer and politician with little military experience, Butler had assumed command of the fort only a day before Baker, Mallory, and Townsend arrived.
At the time, Butler was no abolitionist — he had, in fact, supported Jefferson Davis as the Democratic Party presidential candidate in 1860 — but he understood that most of the men under his command opposed slavery. Returning the escapees to bondage would certainly sow discontent and discourage loyalty among his men. He faced a dilemma: Abide by his Constitutional duty and abandon the principles of his men, or grant asylum and give his enemies legal ground to retaliate.
Luckily, Butler’s legal experience led him to a third possibility.
When Cary asked what Butler intended to do with the escaped slaves, Butler, as recorded three decades later in his autobiography, replied, “I intend to hold them.”
“Do you mean, then, to set aside your Constitutional obligation to return them?” Cary asked.
Butler pointed out that the Fugitive Slave Act applied only between states in the Union. “I mean to take Virginia at her word, as declared in the ordinance of secession passed yesterday,” he said. “I am under no constitutional obligation to a foreign country, which Virginia now claims to be.”
Furthermore, Butler added that he would keep the men “as contraband of war, since they are engaged in the construction of your battery and are claimed as your property. The question is simply whether they shall be used for or against the government of the United States.”
But he did ostensibly offer a loophole: “Though I greatly need the labor which has providentially come to my hands, if Col. Mallory will come into the fort and take the oath of allegiance to the United States, he shall have his negroes, and I will endeavor to hire them from him.”
He knew Mallory would never switch sides, and so Cary rode back to camp empty-handed.
Legally speaking, Butler’s maneuver didn’t free these three men but made them property of the federal government. That word contraband was, until this time, primarily used in maritime law to describe smuggled goods. Butler’s argument treated these three “confiscated” men as “enemy property that was being used for hostile purposes,” on the same level as cannons or muskets seized from an enemy combatant. A military company was under no legal or ethical obligation to return war-making apparatus to the enemy.
Here at the beginning of the war, many in the North were more concerned with holding the Union together than with abolishing slavery. For abolitionists, though, the end of slavery was front and center. But on that morning, Butler provided a solid rationale that brought the two outlooks together while sidestepping longstanding debates about the morality of slavery.
The action quickly became known as Butler’s “Contraband Decision.” Ironically, it’s unlikely that Butler actually used the word contraband when speaking to Cary on that May morning. But that word appeared when the story was reported in the New York Tribune, was reprinted elsewhere, and ultimately caught the public’s attention.
The word found a new life in the North, and contraband quickly became a widespread slang term for an escaped slave. And the word was everywhere: in news reports, political cartoons, and popular culture. The Philadelphia composer Septimus Winner composed a piano solo called “Contraband” and dedicated it to Butler. Louisa May Alcott in 1863 published the short story “My Contraband,” about a white nurse and a freed slave in the Union Army. Painters like Vincent Colyer, Thomas Waterman Wood, and Winslow Homer crafted artistic depictions of freed slaves and named the works Contraband.
Why did the word (a meme by today’s standards) become so popular? To some in the North, referring to escaped slaves as contrabands was a way of thumbing their collective noses at Southern slaveholding rebels. But it served a more important social role as well: Northerners who were reluctant to call former slaves freemen found a more palatable alternative in contrabands — people who were not quite free, but not enslaved either.
Some objected to the term, rightly claiming that calling men and women contrabands was dehumanizing and demeaning — Frederick Douglass, for example, was not a fan. But contraband certainly was a step up from slave.
Word of the “Contraband Decision” spread as quickly through the Confederacy as the Union, especially among the whispered voices of the enslaved working the plantations. Within 24 hours of Baker, Mallory, and Townsend finding safety at Fort Monroe, half a dozen more slaves escaped their captors and trekked to Fort Monroe’s gate. They, too, were given refuge as contraband of war.
Soon, to those in bondage in the South, the word contraband became synonymous with freedom. And knowing that a life outside of slavery was possible, slaves were emboldened to pursue their own emancipation.
Though it stood on a peninsula entirely within the Confederate State of Virginia, Fort Monroe and the surrounding village of Hampton remained under Union control throughout the Civil War. It became an outpost of freedom. To thousands of men, women, and children, escaping to Fort Monroe to become contraband was a huge step toward, ultimately, freedom and American citizenship.
Within only three months of the “Contraband Decision,” nearly 1,000 men, women, and children had fled bondage and reached the fortress. There, they found shelter, food, some education, and — for the first time — wages for their hard work (though often enough they didn’t receive them).
Capt. Charles B. Wilder was Fort Monroe’s assistant quartermaster and superintendent of the contrabands, and it is through his records that we know that, by war’s end, more than 10,000 self-emancipated people had found their first true taste of liberty and begun new lives there.
Since then, Fort Monroe has been called “Freedom’s Fortress” and “Ellis Island for African Americans” — millions of African Americans today can trace their family’s history through this single location. In 1997, one of those descendants, Gerri Hollins, spearheaded the effort to create the nonprofit Contraband Historical Society, whose mission is “to research, preserve, and promote the history, legacy, and contributions of the formerly enslaved, who were considered ‘Contraband’ of war.” Through the CHS’s Contraband Descendants Connection, they help people research and share their ancestors’ stories through the gates of Fort Monroe.
Fort Monroe was decommissioned in September 2011, and a proclamation by Pres. Barack Obama in November of that year designated it a national monument.
Though laud for the “Contraband Decision” usually falls to Maj. Gen. Butler, the credit should be shared with those three enslaved men who escaped from a plantation in the hope that someone with more political clout would support their bid for freedom: Frank Baker, Shepard Mallory, and James Townsend. “Those three men,” says William Wiggins, historian of the CHS and a board member of the Fort Monroe Casemate Museum, “they were the spark that ultimately led to the end of enslavement.”
Though they likely gave little thought to what achieving their own freedom might mean for the enslaved throughout the country, their trek, and Butler’s legal interpretation, caused a ripple that would reach to the highest echelons of government. “What they did was to force the federal government to make a decision,” says Wiggins.
Butler’s argument became the official stance of the U.S. government, a stance later codified in the First (1861) and Second (1862) Confiscation Acts, which authorized the seizure of Confederate property, including slaves. Soon, contraband camps began appearing wherever Union forces set down.
According to historian James McPherson, the “Contraband Decision” “turned out to be the thin edge of a wedge driven into the heart of slavery.” Ultimately it led to the Emancipation Proclamation and the 13th Amendment, abolishing slavery in America.
“But the spark, the catalyst, was those three men,” Wiggins says. “And so I give them credit along with Maj. Gen. Benjamin Franklin Butler and his actions. Those men had names.”
Andy Hollandbeck is the Post’s senior managing editor and language columnist.
This article appears in the January/February 2023 issue of The Saturday Evening Post. Subscribe to the magazine for more art, inspiring stories, fiction, humor, and features from our archives.
Become a Saturday Evening Post member and enjoy unlimited access. Subscribe now