By Amy Westfeldt The Associated Press

NEW YORK – Construction workers began clearing gravel and debris from the site of the World Trade Center towers yesterday, the first step in building a permanent memorial to victims of the Sept. 11 terrorist attacks.

Some family members of those killed rallied at the site, protesting that the project would destroy a piece of history and dishonor their loved ones.

Without the fanfare that usual accompanies groundbreakings, trucks carrying lumber and other equipment rolled down a ramp to the site, and workers started shoveling earth off the north tower footprint.

Stefan Pryor, president of the Lower Manhattan Development Corp., the agency in charge of rebuilding ground zero, said construction was finally under way “after a monumental public planning process.”

But at street level, 100 family members, including a group that has sued to stop construction, said the memorial would destroy the trade center’s historic foundation and fail to honor their loved ones.

“There is hope that we will be able to stop the LMDC from trampling the footprints,” said Anthony Gardner of the Coalition of 9/11 Families, which sued on Friday to stop construction.

Gardner’s attorney, Alan Fuchsberg, said a state judge set a Wednesday hearing in the case. A leading national preservation group recently echoed the group’s concerns in a letter to rebuilding officials.

The memorial, titled “Reflecting Absence” by architect Michael Arad, was chosen two years ago from more than 5,200 proposals. It marks the fallen towers near their footprints with two stone reflecting pools at street level, surrounded by trees. The pools will go 70 feet below ground, where visitors can find the names of the nearly 3,000 people killed in the 2001 attacks and the 1993 trade center bombing.

The construction that began yesterday will take about six weeks, followed by the creation of more than 100 concrete footings to support the memorial. It is scheduled to be complete in 2009.