Theresa May and European Commission President Jean-Claude Juncker release joint statement following their meeting in Brussels on Thursday.
The European Union states that it will not re-open talks on the Brexit deal.
However, they offer to add new words to the accompanying Political Declaration.
This is unlikely to win over UK MPs who voted last week for a full renegotition of the deal.
UK and EU negotiators will hold new talks and find a way to break the current deadlock.
May and Juncker will meet again before end of February to "take stock" of these discussions.
Theresa May's demand to re-open negotiations on her Brexit deal has been officially rejected by the European Union, after the prime minister traveled to Brussels on Thursday morning.
In a joint statement released by the UK prime minister and European Commission President Jean-Claude Juncker, the two leaders said they had engaged in a "robust but constructive" conversation, but confirmed that the Withdrawal Agreement was not up for renegotiation.
"President Juncker underlined that the EU27 will not reopen the Withdrawal Agreement, which represents a carefully balanced compromise between the European Union and the UK, in which both sides have made significant concessions to arrive at a deal," the two leaders said in a statement.
They added that Juncker was open to adding new words to the accompanying "Political Declaration," which is a non-legally binding statement on aspirations for the future relationship between the two sides.
They also said that both sides would hold fresh talks "before the end of February" in order to find a way to break the deadlock.
Britain is set to leave the European Union on March 29. Here, we take a look at the latest figures behind Brexit.
However, the statement casts further doubt on May's hopes of securing any meaningful changes to the controversial Northern Ireland backstop, as demanded by significant numbers of MPs in Westminster.
The backstop is the insurance policy for making sure there is no return to a hard border between Northern Ireland and the Republic under any circumstances after the UK has left the EU.
A majority of MPs in the UK House of Commons voted last week for May to replace the current backstop with "alternative arrangements."
Conservative MPs fear that the backstop in its current form would leave the UK tied to EU rules for years after Brexit, while the Democratic Unionist Party which props up May's government says it would create unacceptable new checks between Northern Ireland and Great Britain.
Conservative rebels have also indicated that they will not back any deal that does not include the removal or complete change of the backstop arrangements.
The British prime minister this week established a new working group charged with defining what those "alternative arrangements" could be.
The EU's rejection of May's call for a renegotiation comes after the UK opposition Labour leader Jeremy Corbyn wrote to May offering to back her deal if she accepted a closer relationship with the EU after Brexit.
Corbyn said the party would support a deal that was based on a permanent customs union and a closer relationship with the single market.
A spokesperson for May said the prime minister would respond to Corbyn's offer "in due course," but added that "our position on the custom union is well known and that has not changed."
The talks were held in a spirit of working together to achieve the UK's orderly withdrawal from the EU, especially in the context of a shared determination to achieve a strong partnership for the future given the global challenges the EU and the UK face together in upholding open and fair trade, cooperation in the fight against climate change and terrorism and defending the rules-based international system.
The Prime Minister described the context in the UK Parliament, and the motivation behind last week's vote in the House of Commons seeking a legally binding change to the terms of the backstop. She raised various options for dealing with these concerns in the context of the Withdrawal Agreement in line with her commitments to the Parliament.
President Juncker underlined that the EU27 will not reopen the Withdrawal Agreement, which represents a carefully balanced compromise between the European Union and the UK, in which both sides have made significant concessions to arrive at a deal. President Juncker however expressed his openness to add wording to the Political Declaration agreed by the EU27 and the UK in order to be more ambitious in terms of content and speed when it comes to the future relationship between the European Union and the UK. President Juncker drew attention to the fact that any solution would have to be agreed by the European Parliament and the EU27.
The discussion was robust but constructive. Despite the challenges, the two leaders agreed that their teams should hold talks as to whether a way through can be found that would gain the broadest possible support in the UK Parliament and respect the guidelines agreed by the European Council. The Prime Minister and the President will meet again before the end of February to take stock of these discussions.