The demonstration in the flashpoint city of Jos coincided with the start of a three-day fast ordered by the authorities in central Plateau state in a symbolic commitment to reconciliation between Muslims and Christians.
With recriminations still flying around over a weekend massacre of Christians in villages on the fringes of Jos, locals said they would pray for an end to the bloodshed and they had no faith in the security services.
"We are mourning because of the children that were killed on Sunday, we are coming as a mass to cry out," said 32-year-old Rebecca Adiwu as she joined in the mass protest in central Jos.
Some carried Bibles, others wooden crosses, and some held the branches of mango trees in a sign of solidarity.
"We do not want soldiers! No more soldiers!" the protesters chanted, waving their Bibles and crosses in the air. Scene: 'Enough is enough' say Nigerian women
Helen Laraba, a 26-year-old tailor who was among the women in black, also vented her anger at the military which has been accused of failing to respond to reports that gangs of machete-wielding Muslims had gone on the rampage.
Troop reinforcements are now patrolling the city and the surrounding villages but locals said it was too little, too late.
"They said they would come and protect us, but they didn't do anything for us," said Laraba.
"We are going to church to cry (over) the killing of innocent people."
Women and children bore the brunt of the three-hour killing spree in the early hours of Sunday morning. The exact toll is unclear with police saying 109 people died while the local information commissioner put the figure at 500.
It was the latest in a long chapter of sectarian violence and came as locals were still trying to come to terms with Muslim-Christian clashes in Jos in January which left several hundred dead.
Jonah Jang, the governor of Plateau, announced late Wednesday a three-day state-wide fast "to forgive our sins and bring peace".
"I am already fasting. It's a symbolic commitment," said 36-year-old accountant Michael Kwakfut.
"It's for the healing of our land, because of (the) ... things that we have done and that have annoyed God."
Police have arrested 49 herdsmen from the mainly Muslim Fulani ethnic group for the killings and said they had confessed to have acted in revenge for attacks in January which left more than 300 mainly Muslims dead.
Announcing the fast, Jang said it was time to put an end to the violence that has long plagued the state.
"It is time to forgive and allow peace to reign. We must sheathe our swords. The responsibility of achieving peace is a collective one," he said.
The country, Africa's most populous, is divided almost in the middle between the two faiths.
Plateau straddles the mainly Muslim north and predominantly Christian south.
The latest attacks also caused about 8,000 people to flee their homes, according to the Red Cross.