Four soldiers and one attacker were killed on Tuesday when hundreds of men wielding pistols and swords assailed troops in Pyaungpit, Maungdaw township, an area populated mainly by Muslim Rohingya.
Troops also gave a toll of seven dead after fighting in the nearby village of Taung Paing Nyar, updating an earlier figure.
"After the incident, troops found seven dead bodies," the state-run Global New Light of Myanmar reported.
"Swords and sticks were found with the bodies."
The military has been scouring the region, not far from the border with Bangladesh, after nine police officers were killed on Sunday in coordinated attacks on three border posts.
The unrest has raised the spectre of a repeat of 2012, when sectarian violence ripped through Rakhine, killing more than 100 people and driving tens of thousands of Rohingya into displacement camps.
Rohingya both in and outside the squalid camps face severe restrictions on their movement and access to basic services, with rights groups calling them one of the world's most persecuted peoples.
Buddhist nationalists have sought to brand the group as illegal immigrants from Bangladesh, despite many tracing their lineage in Myanmar back generations.
A total of six suspects behind the border attacks -- including four who were captured on Tuesday -- are being held by authorities, according to state media.
Authorities have released few details about the attackers or their motives, with some blaming the Rohingya, while others have pointed the finger at Bangladeshi groups.
The clashes mark a dramatic escalation of violence in the region, which has simmered with tension ever since the 2012 unrest that effectively left the state divided along religious lines.
The UN's special advisor on Myanmar, Vijay Nambiar, urged troops and residents to exercise restraint at what he termed a "delicate juncture" for the state.
He also called on civilians to "not be provoked into any kind of response by targeting other communities or religious groups".
Rumours of killings and mass arrests around Maungdaw have spread like wildfire on social media, stoking fear, but details have proved difficult to confirm in the remote and tightly controlled area.
Locals told AFP they were too scared to leave their houses as troops patrolled the streets.
Activists have warned the search for the attackers is being used as a pretext for a crackdown on the Rohingya.
Myanmar's de facto leader, Aung San Suu Kyi, has come under heavy international pressure to carve out a solution for the Rohingya, whose plight has left a dark stain on the country's promising democratic gains.
She recently appointed a commission, headed by former UN chief Kofi Annan, to find ways to heal wounds in the bitterly divided and poor state.