Taney was a former Supreme Court justice who ruled that all slaves, even free slaves, could not be United States citizens.
Roger Brooke Taney served as a Chief Justice between 1836-1864.
Annapolis' tribute to Taney was erected at the State House in 1872.
"While we can not hide from our history - nor should we - the time has come to make clear the difference between properly acknowledging our past and glorifying the darkest chapters of our history", the governor said in a statement. The ruling also said that blacks, whether slaves or not, could never be US citizens, according to the website of American National Biography Online.
The decision to remove the statue was made on Wednesday, after three out of four members of the State House Trust having backed the move.
But in a statement released on Tuesday, Hogan contended that taking the statue down was "the right thing to do".
North Korea Says It Won't Fire Missiles At Guam, After All
The president added, "They will be met with fire, fury and frankly power the likes of which this world has never seen before". North Korean media is only reporting that Kim Jong Un has reviewed the plans and they might carry them out.
In advance of the late-night removal, a team of workers inspected the statue and took measurements at midday on Thursday.
The Senate president said in his letter that voting by email was "just plain wrong" and that the matter was "of such outcome that the transparency of a public meeting and public conversation should have occurred".
In 1996, on the opposite side of the State House, the state unveiled a statue of Baltimore native Thurgood Marshall, the first black Supreme Court justice.
Miller also noted that Taney was personally against slavery - calling it "a blot on our national character" - and had freed his own slaves early in his life.
The statue was reportedly lowered into a truck and driven away to a storage facility.
One of the members who voted on the decision, criticized the fact that the vote wasn't held in public. According to Wood and Cox of the Baltimore Sun, Senate President Thomas Mike V. Miller wrote a letter to Governor Hogan arguing that the statue should stay in place.
Another confederate statue in Maryland is on its way out.