US Civil War Intelligence

The Birth of American Air Intelligence

W.E. Linde Intelligence, Uncategorized, US Civil War, US Civil War Intelligence, US History, US Military History , , , , , , , , , , , ,

I started my Air Force career as an imagery intelligence analyst (also known as a “1N1X1” or a “squint”). It was my job to examine images taken by any of a plethora of “overhead” military platforms (U2 spy planes, as one example) and determine what was going on in them. American intelligence collection capability is truly amazing, and the origins of airborne intelligence collection were simultaneously humble and revolutionary.

The value of airborne intelligence gathering was pretty well evident in the earliest days of aviation. Only months following the first human’s ascension in a hot air balloon into the sky near Versailles   in 1783, the French incorporated a balloon unit into its military.  However, despite the notable contributions made during successive European battles, by 1802 the aérostiers were retired.[i] So by the time the US Civil War erupted in 1861, manned balloons had remained almost entirely outside of military operations for the better part of six decades.

At the outset of the conflict, experienced balloonists (called “aeronauts”)  like Thaddeus Lowe, John La Mountain, and John Wise petitioned the US government for the opportunity to serve the country with the unique capabilities afforded by their aerial platforms. And while La Mountain was able to establish balloon operations in beleaguered Fort Monroe, Virginia, Lowe distinguished himself from his professional competitors by demonstrating the unique capabilities offered by aerial intelligence directly to President Lincoln. On 16 June, 1861, Lowe ascended in a balloon which was tethered near the White House. He then, using a telegraph machine in the basket, telegraphed what he saw directly to Lincoln’s office.

“This point of observation commands an area near fifty miles in diameter. The city with its girdle of encampments presents a superb scene,” reported Lowe in what could be described, using today’s military concepts, as an exercise intelligence report.[ii] Lincoln was impressed, and appointed Lowe head of the Union’s new Balloon Corps.

June 1861 telegraph to Lincoln from Lowe_smaller
Telegraph message from Thaddeus Lowe to Abraham Lincoln, 16 June 1861.

From the earliest days of the war through early 1863, the Balloon Corps demonstrated the unique capability afforded by aerial intelligence.  An early example of this is actually on display at the National Museum of the United States Air Force, in Dayton, Ohio. Set up in the front of the Early Years Gallery is a hand-drawn map. In April 1861, Union forces at Fort Monroe in Virginia were isolated thanks to Virginia’s secession. John La Mountain managed to get his equipment and a single balloon into the old fort, which was bracing for a possible assault similar to Fort Sumter. Major General Benjamin Butler, in command at Monroe at the time, needed intelligence, and La Mountain was the man to get it for him. On 10 August, La Mountain ascended in his balloon to 3500 feet. From that vantage, he was able to identify troop camps and naval activity. He also provided General Butler with this map, possibly the first example of aerial intelligence mapping.[iii]

20180909_163710.jpg
Map drawn by John La Mountain, while observing from his balloon, 10 August 1861. (Exhibit at National Museum of the United States Air Force. Photo of exhibit mine).

The use of these military balloons not only advanced military intelligence collection capabilities, but the Union army was forced to devise new technologies to deploy the assets in the field. Thaddeus Lowe created a mobile hydrogen gas generator, as well as directed the conversion of a Navy vessel into a specialized balloon deployment asset. This vessel, the USS George Washington Parke Custis – a coal barge- was fitted with the special hydrogen generator, and the deck was cleared to allow for balloon inflation. This gave the Union the ability to tow the balloon along the Potomac and adjacent waterways, expanding the range and flexibility of aerial intelligence collection.[iv]

Balloon Ship
USS George Washington Parke Custis, a converted coal barge, was used by Thaddeus Lowe to increase the Balloon Corps’ effectiveness. It also represented military innovation and expanded intelligence capabilities.

Arguably then, the first military aviation platforms commissioned by the US Army were intelligence collection platforms. Yes, they were used for artillery spotting, but one of the primary drivers for President Lincoln to approve the creation of a Balloon Corps was the promise of real-time intelligence collection and transmission to commanders on the ground. It was not uncommon for an officer (at times, the commander) to ascend with Lowe to get a sense of the land and enemy disposition. The Confederates were vexed by the balloons, and tried to destroy them whenever they were observed rising.

Photographs would not be used with balloons, although some experiments of aerial photography (using kites and balloons) had been conducted by civilians around this time. However, the methodical use of professional intelligence gathering by specially trained aeronauts during the US Civil War is clear milestone (if not the first milestone) in the evolution of American aerial intelligence capabilities. A whole new dimension of warfare was emerging.

 

[i] I’m not going to pretend that my research into French ballooning goes beyond the reading of a few secondary sources at this point. Charles M. Evans, in War of the Aeronauts, gives a brief overview of the earliest days of ballooning as he lays the ground work for his in-depth telling the use of balloons in the US Civil War. But I found a fascinating and concise article on the subject in All the Year Round, a British periodical and literary journal edited by none other than Charles Dickens. All the Year Round, Volume 1; Volume 21 (27 Feb, 1869) pp297-299.

[ii] Lincoln, Abraham. Abraham Lincoln papers: Series 1. General Correspondence. 1833 to 1916: Thaddeus S. C. Lowe to Abraham Lincoln, Sunday,Telegram from balloon. 1861. Manuscript/Mixed Material. https://www.loc.gov/item/mal1031300/.

[iii] Evans, Charles M, War of the Aeronauts, a History of Ballooning (Stackpole Books, Mechanichsville, PA, 2002), 96-98.

[iv] U.S. AIRCRAFT CARRIERS: THE FORERUNNERS, NavSource Online Naval History (https://www.navsource.org/archives/02/forerunners/cv-forerunners.htm), Last update: 17 September 2006, accessed 23 Sep 2018.

The Frenzied First Days of War: the Path to the Battle of Big Bethel

W.E. Linde Intelligence, US Civil War, US Civil War Intelligence, US History, US Military History , , , , , ,

Tensions ran high in Virginia in the last week of May, 1861. After the heady rush of victory that electrified the South following

Fort Monroe, located at the tip of the Virginia Peninsula, remained in Union hands throughout the Civil War.
Fort Monroe, located at the tip of the Virginia Peninsula, remained in Union hands throughout the Civil War.

the surrender of Fort Sumter and the seizing of Harper’s Ferry, anticipation of the next clash with the North grew. It was no secret that new recruits were pouring into Washington, and on the 24th, Union forces swept unopposed into Alexandria. Was this how Union General Irvin McDowell would strike the Old Dominion? Or would General Patterson try to leap from Maryland in an attempt to crush General Johnston on the way to Richmond? Or would the Northerners, as then-Colonel John Magruder anxiously wondered, invade via the coast, using the still Northern-held Fort Monroe on the tip of Hampton, VA to stage operations?

On the same day the US Army raised the Federal flag over Alexandria, Magruder received an alarming message from a man “said to be reliable” who had just arrived at West Point, Virginia. Hampton, he said, had been overrun by some 2500 Union troops. Magruder’s next steps demonstrated a levelheadedness in regards to military intelligence that many lacked, both Northern and Southern, in these early days of war. First, he asked that two small craft be dispatched to Jamestown Island, which juts into the James River some 30 miles northwest of Newport News and Hampton, in order to establish communication between the mainland and the area supposedly under Union control. He then also requested that cavalry forces be sent to him at once. “No reliable information can be attained without them,” Magruder wrote.

John B. Magruder. A graduate of West Point, "Prince John" demonstrated a certain affinity for operational intelligence operations.
John B. Magruder. A graduate of West Point, “Prince John” demonstrated a certain affinity for operational intelligence operations.

Within hours of sending this dispatch, additional intelligence revealed that the “reliable” man was actually quite mistaken. In a dispatch that sounded much less frenzied than the first, Magruder relayed that less than half the number of Union troops (1000) had landed at Hampton, and then only briefly. In addition, a small scouting party had been dispatched to Newport News. Only one company of enemy cavalry was noted. Magruder’s quick correction revealed two things: first, his awareness of the need to rapidly convey intelligence up to leadership in Richmond, and second, that he wasn’t letting his ego interfere with the execution of this responsibility. This is in contrast to Lieutenant Colonel A.S. Taylor in Alexandria, who earlier that month took nearly 4 days to communicate the intelligence that prompted his rapid retreat from Alexandria.

The Union withdrawal would be short-lived, however. Days later on the 27th of May, Major General Benjamin Butler lead US troops back into Newport News. Two weeks later, North and South clashed in what would be dubbed the Battle at Big Bethel. Here too, as we will see, Magruder and his subordinate commanders would demonstrate quickly maturing operational and tactical intelligence capabilities that helped blunt Union advantages in troop strength and materiel.

“Shall I arrest Colonel Taylor?” Confusion in Alexandria, May 1861

W.E. Linde Intelligence, US Civil War, US Civil War Intelligence, US History, US Military History , , , , , , , ,

In the first days of May, 1861, with the echoes of Fort Sumter scarcely faded, Confederate forces stationed at Alexandria, Virgina, directly across the river from the Union capital, waited nervously for the next move. Flush with zealous patriotism, the new nation’s leadership in Richmond was adamant that not an inch of sacred Southern soil be yielded without a heavy price in blood. It was easier said than done.

Alexandria from Pioneer Mill, looking north-west (1865)
Alexandria from Pioneer Mill, looking north-west (1865)

Leading the Confederate soldiers in Alexandria was Lieutenant Colonel A.S. Taylor. With orders to stand ground “unless pressed by overwhelming and irresistible numbers”, he conducted a sober accounting of his fighting strength as compared to the swelling army across the Potomac. Two companies of “raw Irish recruits” armed with “altered flint-lock muskets of 1818, and without cartridges or caps…”; a company with 86 musket-armed troops; an additional 52 men in various states of armament (including 15 with no weapons at all); two companies of about 160 men armed with minie rifles, but only from five to nine cartridges each; and two companies of cavalry, one with 40 troops armed with carbines but “limited” ammunition, and one with no weapons except Colt revolvers. So with less than 400 soldiers, Taylor was tasked with holding or delaying the inevitable Federal invasion.

Probably about the same time as he received his orders, which had been delivered on 5 May, Taylor received intelligence from one Mister J.D. Hutton, who until recently worked as a cartographer for the War Department. From Mr. Hutton, Taylor was dismayed (but certainly not surprised) to learn that the Yankees intended to occupy Alexandria within days, either the 7th or the 8th of May (in fact, it would be a few more weeks before Union troops organized enough to even venture across the river). With forces arriving almost daily around Washington, and two steamers only a few miles downriver seemingly ready to move troops, the colonel decided the threshold for “overwhelming and irresistible numbers” had been met. He quickly gathered up his small command, and retreated to Springfield, 10 miles west of Alexandria.

The commanding general of the Potomac Department, General Philip St. George Cocke, was furious that Taylor had not

Philip St. George Cocke (1809–1861). After perceived slights from General P.G.T. Beauregard and General Lee following the Battle at Bull Run, General Cocke took his own life in December, 1861.
Philip St. George Cocke (1809–1861). After perceived slights from General P.G.T. Beauregard and General Lee following the Battle at Bull Run, General Cocke took his own life in December, 1861.

only retreated, but was in such a hurry that he didn’t bother to communicate his intentions. It’s this break down in command and control, as well as in the convoluted handling of intelligence, that is highlighted here. Taylor received credible intelligence from a trustworthy source, but his first instinct was not to disseminate this up the chain of command in order to coordinate a response or seek reinforcements. Cocke’s order to remain in place was clear that this should be done: “keep up your communications with the various parts in your rear, so as to call every resource to your aid and support in making a gallant and fighting retreat, should you be forced to it, and can stand at all without danger of uselessly sacrificing your command.” It wasn’t until 7 May that Cocke located Taylor, and up to this point he was still completely ignorant of why Alexandria had been abandoned. Late on the 6th, in a dispatch to Richmond, Cocke wrote that he had not “been able, from any other source, except that furnished me by the arrival of Mr. Skinner, direct from Alexandria…to learn the cause of that movement; and, so far as I am informed up to this moment, there was no proper or justifiable cause whatsoever for any such movement. After waiting for further intelligence and receiving none, and duly considering and weighing all the circumstances and bearing of that movement with the information before me, I have ordered the return of the troops, as communicated by telegram, a duplicate of which has just been transmitted to the general-in-chief.” Cocke even requested permission to arrest Taylor, but was talked out of it by Robert E. Lee, who instead asked for a reason the forces evacuated.

By the 9th, Taylor had explained his reasons, and this seemed to placate Cocke. On 13 May he forwarded Taylor’s written statement to Richmond. The request to arrest him was absent from this communication. But the lesson here is that, although strategic intelligence around the overall disposition and intent of the US military was becoming clear, operational intelligence operations were off to a rough start, at least in the northeast corner of Virginia. Unfortunately for the Union, more competent leadership would soon arrive, and intelligence preparations were to be put in place over the next three months that proved critical to the Confederate victory at Bull Run.

The Union’s Intelligence Chiefs: The Bureau of Military Information

W.E. Linde Intelligence, US Civil War, US Civil War Intelligence, US History, US Military History , , , , ,
Brandy Station, Va. Col. George H. Sharpe, John G. Babcock, unidentified, and Lt. Col. John McEntee, Secret Service officers at Army of the Potomac headquarters
Brandy Station, Va. Col. George H. Sharpe, John G. Babcock, unidentified, and Lt. Col. John McEntee, Secret Service officers at Army of the Potomac headquarters

The first such organization of its kind in a modern military, the Bureau of Military Information sought to provide the commander of the Army of the Potomac with contemporary all-source intelligence. Despite its rapid maturation and success, it was abandoned after the war. For a comprehensive history of the evolution of the Union’s intelligence capabilities, read Edwin C. Fishell’s excellent The Secret War for the Union, The Untold Story of Military Intelligence in the Civil War.

General Joseph Hooker
General Joseph Hooker

General Joseph Hooker:

Quickly after taking command of the Army of the Potomac in January 1863, General Hooker ordered the establishment of an organization to “organize and perfect a system for collecting information as speedily as possible.” The order was directed to General Marsena R. Patrick, the Provost Marshall.

General Marsena R. Patrick
General Marsena R. Patrick

As part of his duties as Provost Marshall, General Patrick was responsible for the disposition and interrogation of prisoners and defectors. Originally concerned specifically with the security of Washington, Hooker’s mandate to create a “secret service” was the first step toward creating an institutionalized intelligence service for the military.

Maj. Gen. Daniel Butterfield, Chief of Staff to General Hooker
Maj. Gen. Daniel Butterfield, Chief of Staff to General Hooker

General Hooker’s Chief of Staff. Butterfield’s vision and administrative skills were critical to establishing an efficient intelligence reporting system.

Gen. George H. Sharpe, the first head of the Bureau of Military Information
Gen. George H. Sharpe, the first head of the Bureau of Military Information

The first chief of the new military information bureau, Sharpe would oversee the coordinated intelligence operations of espionage, prisoner interrogations, cavalry reconnaissance, the Union Signal Corps, newspaper intelligence gathering, and balloon and signal tower surveillance. (See Fishell, The Secret War for the Union, p297)

John G. Babcock (pictured at top, in group photo)

While a private in the Union army, Babcock’s gift for cartography caught the attention of General McClellan. After a stint working for the infamous Alan Pinkerton, General Burnside offered Babcock Pinkerton’s job once the McClellan spy chief left with his former boss. Babcock accepted, and was hired as a civilian. He stayed on once the Bureau of Military Information was established. (See Fishell, The Secret War for the Union, pp 154, 257-258).

The talents of these leaders were instrumental in the creation of an efficient intelligence organ for the Union.The quality of the Bureau’s reporting was quickly evident. In this dispatch, dated June 7, 1863, sent to General Butterfield, then-Col Sharpe outlines Confederate force disposition, assessed intent, and enemy troop strength.

“Letter of intelligence to Butterfield and Lincoln”, 7 June, 1863

The above image is only the first page (full letter and transcript can be found by clicking the image, stored by the Library of Congress), but within it Sharpe provides an updated threat assessment of Confederate General J.E.B. Stuart’s forces since the battle at Chancellorsville (which ended 6 May, 1863). “I estimated them then at 4700 men in the aggregate, for duty. We now estimate the same at 7500 men for duty.” Equally as impressive is how Hooker sought to qualify other intelligence in the same letter, rather than present sketchy information as more credible than could be vouched for (as was often the practice by many a general before). “We have considerable reason to believe that two brigades of cavalry have recently arrived from the direction of North Carolina not heretofore connected with General Stuarts command. We can of course give no estimate of their force; but it would not be safe to put them down at less than 1500 men to a brigade.” The Bureau began to make rapid strides in making American intelligence operations more professional, more analytical, and more reliable.

WEL

Lieutenant Jones’ Intelligence Network and the Evacuation of Harper’s Ferry, 18 April 1861

W.E. Linde Intelligence, US Civil War, US Civil War Intelligence, US History, US Military History , , , , , , , , , ,

The burning of the United States arsenal at Harper's Ferry, 10 P.M. April 18, 1861

“Up to the present time no assault or attempt to seize the Government property here has been made, but there is decided evidence that the subject is in contemplation, and has been all day, by a large number of people living in the direction of Charlestown; and at sun-down this evening several companies of troops had assembled at Halltown, about three or four miles from here on the road to Charlestown, with the intention of seizing the Government property, and the last report is that the attack will be made to-night. I telegraphed this evening to General Scott that I had received information confirming his dispatch of this morning, and later to the Adjutant-General that I expected an attack to-night. I have taken steps which ought to insure my receiving early intelligence of the advance of any forces, and my determination is to destroy what I cannot defend, and if the forces sent against me are clearly overwhelming, my present intention is to retreat into Pennsylvania.”

First Lieutenant Roger Jones, Mounted Rifles, U.S. Army, reporting on the situation at Harper’s Ferry, 18 April, 1861.

Days after the fall of Fort Sumter, the vulnerable military outpost at Harper’s Ferry was on high alert. Lieutenant Jones had been growing increasingly pensive as reports arrived of groups of Southern troops arriving in the vicinity intent on seizing government property. His concerns were quite legitimate. The Union army at this point was still woefully undermanned for any significant combat operations (perhaps explaining why a First Lieutenant was commanding the arsenal at Harper’s Ferry). Additionally, the historic arsenal was surrounded by high ground: the Maryland Heights to the east, and the Loudon Heights to the south. Harper’s Ferry was, in the words of historian James M. McPherson, a “trap waiting to be sprung by any force” that could place artillery at those locations.[i]

Harper's Ferry, photographed immediately after its evacuation by the rebels. 1861Both the Union and the Confederacy would find defending Harper’s Ferry difficult. This photo was taken after Confederate forces found the position untenable as well.

Of course, much has been written about these earliest days of the war. What I want to draw attention to here are these words of Lieutenant Jones above: “I have taken steps which ought to insure my receiving early intelligence of the advance of any forces…” This statement demonstrates the reality of military intelligence operations during the Civil War era. For most of the conflict, intelligence operations were the responsibility of the commanding officer. There was no institutional support for such activities, and the successful use of intelligence rested almost exclusively on the skill and disposition of the officers in charge. If the officer had little ability or faith in intelligence (as demonstrated by many military leaders of the time), then his operations usually would benefit his forces little, or actually impede battlefield success.

It would appear the Lieutenant Jones’ home-spun intelligence network was effective in this case. Shortly after the above dispatch was sent to Washington, Jones became convinced that he could not defend Harper’s Ferry. “Immediately after finishing my dispatch of the night of the 18th instant,” he informed Winfield Scott, ”I received positive and reliable information that 2,500 or 3,000 State troops would reach Harpers Ferry in two hours…” He set out immediately to destroy some 15,000 arms within the arsenal in an attempt to deny their use by the rebels, then evacuated his command to Carlyle Barracks, Pennsylvania. But the next four years of combat would reveal just how inconstant Union and Confederate leaders would be in the effective direction of intelligence operations.

WEL

[i] James M. McPherson, Crossroads of Freedom: Antietam (New York: Oxford University Press, 2002), 106-110.

Quotes of Lt Jones taken from the official records of the Union and Confederate armies. / Series 1 – Volume 2.

“12000 strong…they intend to seize the capitol” – the pitfalls of human intelligence, 11 April 1861

W.E. Linde Intelligence, US Civil War, US Civil War Intelligence, US Military History , , , , , ,
Unknown to Abraham Lincoln, Thursday, April 11, 1861 (Plan to seize Washington) (Library of Congress)
Unknown to Abraham Lincoln, Thursday, April 11, 1861 (Plan to seize Washington) (Library of Congress)

In early April, 1861, the possibility of civil war was growing by the day. Seven southern states had already seceded, and tensions in Baltimore and Virginia were poised to make things even worse (the attack on Fort Sumter would commence on 12 April). Intelligence gathering was a lost art, and insurrection outside the gates of Washington was of utmost concern to the new president. Scarcely a month following his inauguration in March, a poorly written intelligence warning was sent to President Lincoln:

“Baltimore April 11. 1861 “Dear Friend I take this metod of informing you that you better prepair yourself for an asailing mob that is organizing in Baltimore as far as i can inform myself is about 12000 m. strong they intend to seize the capitol and yourself and as they say that they will tar & put cotton on your head and ride you and Gen Scot on a rail this secret organation is about 70000 m members in Maryland and Virginia and thay can be all brought to gether in five days, the person that rits this was a member and is bound by a strong oath which if they now ho i was i wold not be suffer to live but justis to you and my country make me do this”

There’s no record (that I am aware of) that Lincoln reacted to (or even read) this letter. Clearly, no such force attempted to seize Washington, but such intelligence certainly contributed to the mounting wariness of a possible attack. A week later (19 April), US soldiers did clash with a mob in Baltimore, but this was hardly an organized attempt to overthrow the government. Rather, southern-sympathizers were outraged that Union troops were passing through their city, and tensions ultimately exploded into a riot. Also of note, on 1 July, the Union army, by order of Winfield Scott, arrested Baltimore’s chief of police and the city’s police commissioners in order to “carry consternation into the ranks of our numerous enemies about you [General Nathaniel Banks].” This dramatic move was based on the recent discovery of supposed secret armories, and the suspect allegiance of certain key members of Baltimore’s civilian leadership.

Ascertaining the credibility of intelligence is always a tricky endeavor. However, making matters worse for Lincoln (and the forming Confederacy, for that matter), was that there was no institutional intelligence capability to help provide such guidance. Intelligence capabilities (and the skills needed to use them) were as lacking as the general military readiness of the North and the South – and probably more so. The consequences of this were many. Among them was a hyper-sensitivity to possible threats to Washington, as Lincoln and Scott demanded significant forces be always held back to protect the city, which at times frustrated generals who felt they had better use of the army. But such weakness in the Union’s intelligence apparatus arguably contributed to the early defeat of Union forces at Bull Run, as military leaders  – scarcely more competent in intelligence operations – were largely left to their own devices and cunning to collect and assess intelligence.

WEL