Islamic State militants have reportedly taken over large parts of a Palestinian refugee camp in Damascus on Wednesday, and are fighting with a group inside the camp.

This is the deepest incursion yet by ISIS militants into the Syrian capital. If they gain control of the Yarmouk camp, they can threaten President Bashar Assad's seat of power.

"Fighters from ISIS launched an assault this morning on Yarmouk and they took over the majority of the camp," said Anwar Abdel Hadi, director of political affairs for the Palestine Liberation Organization in Damascus.

The Syrian Observatory for Human Rights, a Britain-based activist group, said ISIS was in control of a "large part" of the camp after fighting with Palestinian groups also opposed to President Bashar Assad's regime.

The Observatory said clashes broke out Wednesday in Yarmouk between IS fighters and members of the Palestinian faction called Aknaf Beit al-Maqdis.

According to the Beirut-based Al-Mayadeen TV, the IS extremists fighters entered the camp from the nearby neighborhood of Hajar Aswad. That southern Damascus neighborhood is controlled by militants.

Yarmouk was once a thriving neighborhood home to 160,000 Palestinian refugees and Syrians but has been caught up in the country's fighting and besieged by regime forces for more than a year.

Only about 18,000 residents are estimated to remain in the camp after many fled the fighting.

The Yarmouk Camp has changed hands several times over the past four years. It was under Assad's control, moved to the hands of moderate Sunni rebels (that don't belong to Islamist organizations) and, at a certain point, was also captured by the al-Qaeda-affiliated Jabhat al-Nusra organization. 

The siege has caused significant shortages of food, water and drugs inside the camp.