President Donald Trump, who is trying to position himself as a standard-bearer for law and order with less than five months to go before November's presidential election, tweeted calls by police Saturday to identify more than a dozen other demonstrators who took part in the action.
On Monday evening, a group of protesters attacked the statue of former President Jackson, a slaveowner who led the United States from 1829 to 1837, which stands in Lafayette Park next to the White House.
They wrapped ropes around the monument and tried to knock it down.
Based on video footage, four were identified tying or pulling the ropes, or passing a hammer to another demonstrator.
Aged between 20 and 47, they were charged on Friday with "destruction of federal property," a charge punishable by one to 10 years in prison.
One of them was arrested on Friday and presented to a judge on Saturday, while the other three were named but have not yet been apprehended, the Washington prosecutor's office said in a statement, AFP reported.
"These charges should serve as a warning to those who choose to desecrate the statues and monuments that adorn our nation's capital: your violent behavior and criminal conduct will not be tolerated," wrote Acting US Attorney Michael R. Sherwin.
Since the death of George Floyd, an African-American killed by a white police officer on May 25, Americans have faced a critical reckoning on the country's racism problem.
Floyd's death ignited mass civil unrest across the country. On the sidelines of those protests, several statues, including of Confederate generals and pro-slavery leaders, have been targeted or pulled down.
Trump, who is running for a second term, on Friday signed an executive order pledging to enforce prosecution for protesters who vandalize public memorials.
The Republican president "will never allow violence to control our streets, rewrite our history, or harm the American way of life," the White House said at the time.